The Role of Integrated Scientific Advice for the Early Determination of RWE Requirements in HTA and Payer Assessments


Andrea Schmetz, MBA
Senior Consultant
Market Access Consulting

Stephanie Wise, MPharm
Market Access Consulting

Matthew Bending, PhD
Executive Director of HTA Strategy &
UK Practice Lead
Market Access Consulting

Patricia Hurley, PhD
Senior Director
Strategic Regulatory Consulting

Real-world evidence (RWE) is a ubiquitous conversation topic in discussions on the current health technology assessment (HTA) landscape for biotech and biopharma assets. By now it is widely known that RWE is an unavoidable part of drug development and should be given thorough consideration, ideally early on; a simple online search for “the importance of real-world evidence in HTA” reaps thousands of results. High-quality practice guidelines lead the way to explaining how to develop robust RWE, e.g., through the Innovative Medicines Initiative’s (IMI) GetReal project, the International Society for Pharmacoeconomics and Outcomes Research’s (ISPOR) RWE task force, and other instructional articles.1,2 Despite the availability of such guidance, often health technology assessment bodies, payers, and even regulators do not regard submissions of real-world data or evidence as manufacturers hoped they would. The aim of this article is to discuss the current use of real-world data and evidence in HTA and payer appraisals, its potential role in lifecycle management, and how the early dialogue provided by Integrated Scientific Advice (ISA) engagement can be used as a key tool in real-world evidence generation planning.

Click here to download printable PDF

    RWE is defined by the ISPOR task force as being obtained from the process of analyzing real-world data (RWD), which in turn is defined as data gathered outside randomized clinical trials (RCTs), e.g., through routine clinical practice.1 The nature of this data implies that it is less controlled and thus more prone to bias than the data obtained from an RCT.3 As a result, implementation of robust RWE plans can be almost as work intensive as planning for an RCT, but the return on investment is not always as obvious to manufacturers. RWE planning also requires significant internal collaboration and “buy-in” across teams as it affects, at a minimum, the clinical, regulatory, HEOR, and market access functions. In the light of this effort, is developing RWE worth the effort?

      The impact of RWE in HTA decision making as of today has been explored by multiple stakeholders and interested parties, and results show a clear trend towards more use and impact of real-world evidence in HTA decision making, however, they also outline apparent limitations.4,5

      • A conference presentation from 2018 outlines the use and impact of RWE in appraisals by the National Institute for Health and Care Excellence (NICE) in the United Kingdom (UK), the Pharmaceutical Benefits Advisory Committee (PBAC) in Australia, the Haute Autorité de Santé (HAS) in France, and the pan-Canadian Oncology Drug Review (pCODR) in Canada in non-small cell lung carcinoma (NSCLC) drugs, where the authors found that RWE was used as supportive evidence for efficacy in a majority of NICE, PBAC, and pCODR appraisals.5
      • A further research article shows citation of real-world prevalence data in a majority of melanoma drug appraisals and just under half of the 52 reviewed appraisals utilized RWE effectiveness data.6
      • A study from the London School of Economics shows that RWE is being used mainly in accelerated access by regulators and in re-assessment by HTA bodies.7

      While these delineate a clear place for RWE in HTA and regulatory decision making today, it can be seen that currently RWE mainly finds use in cost-effectiveness markets. Furthermore, it emerged that the HTA policies within these cost-effectiveness markets are most favorable regarding the use of RWE in evaluations. Nonetheless, most countries’ policies clearly state that most weight in decision making is to be given to RCT data.8

        An interesting observation that emerged during this review was that markets that do not utilize cost-effectiveness analyses in their HTA appraisal procedures also do not seem to utilize RWE for decision making to a large extent. This observation might merit further investigation, as it might suggest that markets not using cost-effectiveness data are still establishing appraisal processes for RWE, creating ambiguity but also opportunity for manufacturers when developing RWE strategies for the markets with larger potential sales volume.

        Part of the reason for not using RWE might be the lack of perceived rigor required from evidence as outlined in the law or data protection requirements, as is the case in Germany.9 However, given the challenges brought by new medicines with long-term effects, such as CAR-T and other gene therapies, those markets currently not using cost-effectiveness data and RWE may need to start considering RWE more in their drug appraisals.

          It is, by now, commonly known that healthcare systems are under increasing pressure from innovative, long-term effectiveness therapies that are expensive and come with much uncertainty in terms of their long-term impact.10,11 As a response, many stakeholders across healthcare systems are starting to look at lifecycle management of drugs. For example, EUnetHTA is dedicating a whole workstream in a Joint Action to lifecycle management, implying a continuous need for data on every asset launched.12

          Lifecycle management of drugs that incorporates the use of RWE is an increasingly discussed topic as well; large conferences, including the 2019 HTAi conference that Evidera staff attended, gave this topic key importance.13 A main example cited during the conference was CAR T-cell therapies, where long-term evidence and RWE that have emerged since launch suggest that CAR-T may not be as effective in the long term as initially hoped or projected.14 Similar results may become reality with regards to other gene therapies in light of the waning effect over time. This constitutes a “perfect storm” for healthcare systems with robust pipelines of promising technologies, with increased demand for early and equitable access likely resulting in a deepening of the affordability crisis. Two key areas of focus in lifecycle management are suggested to address this:

          • Removal of low-value technology
          • The shift of pricing, reimbursement, and market access (PRMA) decision making to a proactive approach throughout the lifecycle

          The UK health system, including NHS England and the NHS Clinical Commissioners (NHSCC) joint working group, represents an example of a healthcare system acting to remove low-value technology. Following a national consultation in 2017, guidance listing 18 items which should no longer be routinely prescribed in primary care was published; this list was updated in 2019 with updates and additions, clearly showing that products are actively being managed off care pathways.15 Similarly, Germany has just ratified the GSAV (Gesetz für mehr Sicherheit in der Arzneimittelversorgung) law this year, outlining the potential for frequent reassessment throughout products’ lifecycles to ensure continued delivery of relative value in a shifting landscape.16

            As it is evident that RWE is increasingly finding its place and use in HTA appraisals and in market access in general, and healthcare systems will likely see an increasing need for data on pharmacotherapies throughout the product lifecycle, it will be key for manufacturers to establish ways to plan an RWE strategy as early and as efficiently as possible.

            Early Engagement with ISA Can Help Shape RWE Plans

            Integrated Scientific Advice is a multi-stakeholder advice process that brings together regulatory advice (either with country-level agencies or the European Medicines Agency [EMA]) and HTA advice (either with individual country-level agencies or multi-country collaborations).17

            Early engagement with ISA is a valuable strategy to refine and evaluate evidence generation plans and align them with regulators’ and HTA bodies’ needs (See Figure 1). However, a vital question emerges: do regulatory agencies and HTA bodies require a discussion on the plans for developing real-world evidence during scientific advice and to what level are they willing to discuss it?

              Figure 1. Timeline for Inclusion of RWE Generation Strategy and ISA

              Historically, regulatory agencies have often requested RWD and RWE as mandatory post-launch evidence commitments. They are vital tools to enhance existing safety and efficacy data in the long term to satisfy approval requirements.18,19 HTA bodies have been traditionally more hesitant to consider RWE and RWD in initial PRMA assessments and thus did not frequently request or consider it. However, contrary to past experience, a new trend seems to be emerging. Evidera has engaged in several early scientific advice and ISA processes in 2018 and 2019 and a shift has been observed in the form of increased requests for and clarifications on RWE and RWD generation. Particularly in the list of issues provided to manufacturers as part of the EU parallel advice program, more requests on RWE planning, e.g., for observational trials and registries, were seen. It seems requests for RWE and the willingness to discuss its inclusion now come equally from HTA bodies and regulators during EMA-HTA Parallel Consultation.

              Despite the increasing demand and interest in RWE observed, a key challenge in terms of impact on outcomes remains as was summarized by the former head of NICE scientific advice, Dr. Leeza Osipenko, in an interview with Evidera in 2018: “… unfortunately there is a strong move to start using RWE in place of, rather than in addition to, properly collected and analysed data which are needed to establish relative clinical effectiveness of the intervention. RWE often produces more noise than clinically relevant information.”20

              “… unfortunately there is a strong move to start using RWE in place of, rather than in addition to, properly collected and analysed data which are needed to establish relative clinical effectiveness of the intervention. RWE often produces more noise than clinically relevant information.”20

              To avoid having carefully collected RWE or RWD categorized as noise, it will be important for manufacturers to understand requirements in the exact context of their product. General themes that emerged from reviewing scientific advice feedback are in line with good practice guidelines and pointers given by EMA1,18; RWE should provide:

              • Collection of long-term effectiveness and safety outcomes
              • Assurance that manufacturers are prepared to support their therapies throughout the lifecycle
              • Preparation by the manufacturer to actively contribute to the development and improvement of overall disease area outcomes
              • Generation of data that applies to different healthcare systems and treatment patterns as an acknowledgement that variation inside a randomized clinical trial, and thus local applicability of trial data, is limited
              • Demonstration that therapies provide satisfactory relative value in a clinical landscape that is continually evolving, particularly in some disease areas such as cancer

              This review also revealed how HTA bodies do not want RWE or RWD to be used, as emphasised by other stakeholder voices, e.g., members of the Cochrane collaboration21:

              • Evidence collection to circumvent gathering of sufficient clinical trial evidence
              • A tool targeted at collection of evidence on a therapy in isolation from the system, e.g., product registry planning versus integration into a disease registry

                Global Collaboration is Needed to Improve the Value of RWE

                Overall, RWE has established itself as a key part of an asset’s evidence generation plan, but it seems that the exact use and design of RWE to make an impact on regulatory and HTA decision making is yet to be defined. In particular, markets not currently using cost-effectiveness data should hold manufacturers’ attention, as the role of RWE in these markets seems to be in the early shaping process.

                A team at HTAi proposed key focus areas to be developed in order to further the importance and place of RWE in market access, including global collaboration to provide leadership in the form of an  accreditation body and establishment of common legal and methodological frameworks.22 At the HTAi Global Policy Forum conference in January 2019, it could be seen that this work has already begun in the form of EUnetHTA’s Work Package 5, titled “Life Cycle Approach to Improve Evidence Generation.12” The objective of this work package is to help generate robust evidence for health technologies (pharmaceuticals or others) all along the technology lifecycle; it consists of two strands: (A) Early dialogues (initial evidence generation) and (B) Post-launch Evidence Generation and Registries.

                  Scientific advice/early dialogues with HTA bodies and regulatory agencies offer manufacturers an optimal approach for early development of integrated, cross-functional evidence generation plans which, for four key reasons, are particularly applicable to more novel fields like RWE.

                  1. Early cross-functional alignment on requirements can avoid last-minute shifts in evidence planning that can be very costly; a side benefit of needing to write a briefing book.
                  2. Participating in scientific advice generates alignment and marks a willingness to communicate with regulators and HTA bodies, meaning that evidence plans are less likely to be dismissed than those without official consultation.4,23
                  3. Advice will be situational and applicable to the exact asset in question, hence avoiding ambiguity in evidence planning due to interpretation of general guidelines without dialogue.
                  4. It allows manufacturers to enter the conversation on the place of RWE in evidence generation plans outside of political discussions, further helping to shape the current landscape.

                  The global PRMA landscape is seemingly shifting towards demands for iterative demonstration of value in real-world populations in order for therapies to earn and maintain their place in clinical pathways. In such an environment, ISA represents an opportunity to gain early external and internal cross-functional alignment on RWE strategy and goals for new therapies at launch and beyond (See Figure 2).

                    Figure 2. Overview of ISA as a Solution to Current Market Changes


                    1. Berger ML, Sox H, Willke RJ, Brixner DL, Eichler HG, Goettsch W, Madigan D, Makady A, Schneeweiss S, Tarricone R, Wang SV, Watkins J, Mullins CD. Good Practices for Real-World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR-ISPE Special Task Force on Real-World Evidence in Health Care Decision Making. Value Health. 2017 Sep;20(8):1003-1008. doi: 10.1016/j.jval.2017.08.3019. Epub 2017 Sep 15.
                    2. Miksad RA, Abernethy AP. Harnessing the Power of Real-World Evidence (RWE): A Checklist to Ensure Regulatory-Grade Data Quality. Clin Pharmacol Ther. 2018 Feb;103(2):202-205. doi: 10.1002/cpt.946. Epub 2017 Dec 6.
                    3. Garrison LP Jr, Neumann PJ, Erickson P, Marshall D, Mullins CD. Using Real-World Data for Coverage and Payment Decisions: The ISPOR Real-World Data Task Force Report. Value Health. 2007 Sep-Oct;10(5):326-35.
                    4. Gannedahl M, Udechuku A, Bending MW. Initiatives Driving Accelerated Access to Medicines in Europe: Review of Recent Concepts and Developments. Medicine Access @ Point of Care. 2018 March. doi: 10.1177/2399202618761854.
                    5. Ng T, Chawla T, Bending MW. PCN293 – What is the Value of Real World Evidence in Oncology in HTA Appraisals in England, France, Canada and Australia? Value Health. 2018 Oct; 21(suppl 3):S64. doi:
                    6. Makady A, van Veelen A, Jonsson P, Moseley O, D’Andon A, de Boer A, Hillege H, Klungel O, Goettsch W. Using Real-World Data in Health Technology Assessment (HTA) Practice: A Comparative Study of Five HTA Agencies. Pharmacoeconomics. 2018 Mar; 36(3):359-368. doi: 10.1007/s40273-017-0596-z.
                    7. LSE Research Online. Gill J, Kanavos P, Avouac B, Duncombe R, Hutton J, Jahnz-Rozyk K, Schramm W, Spandonaro F, Thomas M. The Use of Real World Evidence in the European Context: An Analysis of Key Expert Opinion. London School of Economics, London, UK.  Available at: Accessed September 4, 2019.
                    8. Makady A, Ham RT, de Boer A, Hillege H, Klungel O, Goettsch W; GetReal Workpackage. Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies. Value Health. 2017 Apr;20(4):520-532. doi:  10.1016/j.jval.2016.12.003. Epub 2017 Jan 27.
                    9. Universitat Hamburg Online. Ertl K. Sozialrechtliches Methodenbewertungsrecht – Offen fur “Big Data” und Real World Evidence? 2018. Available at: Accessed September 4, 2019.
                    10. Deloitte. Analysis 2019 Global Health Care Outlook – Shaping the Future. Available at: Accessed September 4, 2019.
                    11. Goldman DP, Van Nuys K, Cheng WH, Hlavka JP, Pani L, Chassang S, Snowberg E. A New Model for Pricing Drugs of Uncertain Efficacy. NEJM Catalyst. December 13, 2018. Available at: Accessed August 29, 2019.
                    12. eunethta | European Network for Health Technology Assessment. JA3 Work Package 5 – Lifecycle Approach to Improve Evidence Generation. Available at: Accessed August 29, 2019.
                    13. Evidera. Insights from the HTAi 2019 Annual Meeting. Available at: Accessed September 6, 2019.
                    14. Shah NN, Maatman T, Hari P, Johnson B. Multi Targeted CAR-T Cell Therapies for B-Cell Malignancies. Front Oncol. 2019 Mar 12;9:146. doi: 10.3389/fonc.2019.00146. eCollection 2019.
                    15. NHS Clinical Commissioners. Items Which Should Not Routinely be Prescribed in Primary Care: Guidance for CCGs. Version 2, June 2019. Available at: Accessed September 4, 2019.
                    16. Bundesministerium fur Gesundheit. Gesetz fur Mehr Sicherheit in der Arzneimittelversorgung. Available at: Accessed September 4, 2019.
                    17. Bending M. Early Integrated Scientific Advice: A Key to Optimal PRMA? Presented at World Pharma Pricing and Market Access Congress, March 19-20, 2019, in Amsterdam, Netherlands.
                    18. European Medicines Agency. Moseley J. Regulatory Perspective on Real World Evidence (RWE) in Scientific Advice. 17 April 2018. Available at: Accessed September 4, 2019.
                    19. FDA | U.S. Food & Drug Administration. Real-World Evidence. 2019. Available at: Accessed August 29, 2019.
                    20. Osipenko L, Michel S. HTA Scientific Advice – Is it Becoming More Important? The Evidence Forum. 2018 Fall. Available at: Accessed August 29, 2019.
                    21. Schunemann HJ. All Evidence is Real World Evidence. March 29, 2019. thebmjopinion. Available at: Accessed August 29, 2019.
                    22. HTAi. Tunis S, Sampietro-Colom L. Real-World Evidence in the Context of Health Technology Assessment Processes – from Theory to Action. HTAi Policy Forum Series Newsletter. February 2019. Available at: Accessed September 4, 2019.
                    23. European Medicines Agency. Scientific Advice Leads to Stronger Applications from Industry. April 17, 2015. Available at: Accessed September 4, 2019.