GenAI: The Muscle Behind Strong Regulatory Intelligence For Combination Products

By Doug Mead, CP Pathways LLC

Combination products are inherently complex because they involve both drugs and devices. That makes their regulatory testing and submission requirements more complex than those for either type of product alone. For those reasons, keeping up-to-date with all the regulations, guidances, and standards that apply to combination products — and then both interpreting and implementing them — is one of the biggest challenges we face in bringing new drug delivery devices to market.

While guidances and standards include basic development and regulatory requirements and expectations, one of the best regulatory strategies is to research recent FDA approvals to understand precedents and policies that were applied to specific products that are like those your organization is developing. This “precedent research” is critical to understanding regulatory risk for any new product application.

As a consultant specializing in combination products, even I struggle to stay current with new, emerging, and evolving regulatory expectations and how they affect drug delivery devices in development and in submission planning. Yes, keys to keeping up still include attending conferences, benchmarking, and conducting meetings with regulators. However, leveraging new GenAI tools has become an essential, additional approach to strengthening and expanding the skill sets needed for combination product development in an increasingly complex regulatory environment.

When used wisely, GenAI  can provide quick insights and save tons of time. And GenAI tools have improved significantly just in the last year, both with the standard free or low-cost tools, or with the more sophisticated commercial search tools purpose-built for regulatory intelligence.

Point GenAI Tools At Regulatory Databases

GenAi search tools use Large Language Models (LLMs) and massive databases, along with web sources with relevant content, to match user queries, retrieve just the right information from these sources,  and synthesize (i.e., statistically predict) a human-like answer.

In the arena of combination products regulatory intelligence, these capabilities can be pointed at 1) what other companies submitted in their applications for their delivery devices, 2) what questions were asked during the reviews,  and 3) what regulators ultimately accepted before approving the product. FDA’s CDER, CBER, and 510(k) databases, as well as the EMA’s EPAR databases, are important targets for precedent research. These databases include thousands of detailed documents with the regulator’s decisions and comments. Topics of high interest and regulatory risk might include the number of PPQ (Process Performance Qualification) batches required, examples of when a human factors validation study or bioequivalence study was waived, or comments or Information Requests that were made about design verification testing.

Just a few short years ago, the effort to find relevant precedents was overwhelming. Prior to employing GenAI tools, I used Google search terms and the terms that targeted the database of interest (e.g., “drugs@fda.gov” or “Other Reviews”). From a list of likely review memos, I could then search the PDF file for terms of interest. It was time-consuming and difficult to know if the search was sufficiently comprehensive.

GenAI search tools have significantly advanced this precedent research. With a well-crafted query, the search results will find the database, look for specific and alternative terms in the indexed database, and deliver responses that are usually highly relevant to the specific topics searched. Hallucinations (i.e., false or misleading results) are still possible. For that reason, going to the source files listed in the response is critical for checking accuracy.

Optimizing GenAI Searches Requires Recognizing Limitations

Of course, there are limitations to the capabilities of these tools. As noted above, GenAI searches can yield incorrect or misleading results. For example, a poorly worded question about stability might generate drug product results or even oral dosage form results. In that query scenario, it would be wise to be specific about exactly what kind of stability information is being sought — for instance, functional stability vs. drug product/substance stability, or context on the dosage form, kitted devices, or stability of sources.

Also problematic can be the terminology for combination products. LLMs can still get quite confused because of that. For example, the term prefilled pen, used by some health authorities, could be interpreted by an LLM as either a pen injector (e.g., an insulin pen) or an autoinjector. As a partial solution, commercial GenAI search tools often include an interface algorithm specifically developed for regulatory applications. In my work with Rhizome AI, one of several commercial tools on the market, we asked thousands of questions about drugs, devices, and combination products to identify misinterpretations. Then, we implemented software corrections to deliver the most relevant and accurate responses.

Another limitation is that, for various reasons, some information you might like to know simply isn’t available. For example, it may be redacted in the database. Device performance specifications or acceptance criteria, as well as supplier information, are usually redacted. However, test results are not usually redacted.

Also, “deep dive” or comprehensive searches eat up search time (i.e., compute time), and some tools either truncate database search time and results or take a very long time to generate a result.

Lastly, company IT departments may also need to find a way to allow the careful use of an enterprise version of a GenAI search tool that can access outside databases while preserving information security.

4 Keys To  Effective GenAI Regulatory Searches

Even with these shortcomings, GenAI tools remain immensely powerful. Again, I strongly recommend spending the time to hone your capabilities in conducting precedent research.

To help you in these efforts, I offer four best practices shaped by my own experience with these tools:

1. Search the FDA’s Drugs@FDA database for review memos and correspondence related to delivery devices and human factor studies. 

Targeting this site can include the introductory phrase: “Based solely on the drugs@FDA database, what has the FDA commented on related to…”   Similar phrases can isolate content in CBER reviews, EMA EPARs, CTIS clinical protocols, labeling (USPI) or approval letters, or international government combination product regulations and guidances.

2. Educate your LLM query with some introductory information by including one or two sentences about the topic of interest.

For example, you could start a query about “control strategy” (which could be interpreted as two separate words), with a quick sentence about its meaning. For example, “A control strategy for an autoinjector may include incoming acceptance of components, assembly process controls or in-process controls, design verification testing, or release testing. Based solely on the content in the drugs@FDA database, what deficiencies has the FDA cited for a control strategy for an autoinjector in the last four years?”

Even with this wording example, one publicly available GenAI took over five minutes to execute with some reasonable results, while reporting some difficulties opening the large PDF memo files and noting significant redacted content it could not include. Commercial GenAI regulatory search tools  have usually downloaded, indexed, and applied Optical Character Recognition (OCR) to image pages and store the documents in their servers. These regulatory intelligence subscription services and their regulatory databases have a significant advantage in providing the most comprehensive, relevant, and accurate results very quickly.

3. Ask multiple questions with increasingly specific wording.

Query responses will often include, at the end of the initial response, one or more alternative search strategies or a way to refine the query to find the desired content. The search activity becomes more like a conversation to find the information you really want — again, if it’s available and correctly predicts alternative search strategies. For example, the search tool can ask whether you want only part of the response, whether the search should be repeated to find more examples, or whether you want a table format for the responses.

4. Always check or review the sources referenced in the responses.

A GenAI tool aims to provide a definitive answer, but it can still exaggerate or misinterpret a document, failing to take the overall context into account. Probably the most useful strategy is just identifying the relevant sources for a careful reading of that context. For example, the search tool can interpret an FDA challenge as a requirement, whereas the company provided a justification for their approach that the FDA reviewer ultimately accepted. That kind of nuanced back-and-forth between the reviewer and the company is easily misinterpreted by GenAI tools.

Start Strengthening Your Regulatory Intelligence Now

As noted above, because combination product development and regulatory expectations are only generally explained in guidances and many standards, GenAI searches of precedents and decisions in regulatory documents for approved products can help refine development plans and submission content to minimize regulatory risk. However, the use of these tools isn’t a panacea. Often there will be inconsistencies in reviews or reviewer-dependent concerns, or changes in review policies when safety signals seen by the FDA or other health authorities come and go. These inconsistencies are often easily found and readily apparent. Then, the time can be spent assessing these precedents and developing test programs and submission strategies most likely to be successful.

In summary, the skills needed to effectively use these tools are two-fold: 1) ask clear questions, and then 2) confirm the responses by reviewing the source information. With those foundational approaches driving your searches, GenAI tools should become part of our regulatory intelligence arsenal, whether you are a developer of delivery devices or are a regulatory professional.

Doug Mead is Principal Consultant and President of CP Pathways LLC, a combination product consultancy. He helps combination product companies with their regulatory strategies and works closely with delivery device teams and regulatory staff to prepare submissions in line with the latest regulatory expectations.