Real-Time Adverse Event Detection: Building an AI-Powered Pharmacovigilance
How we built an AI pharmacovigilance platform that reduced ICSR processing
Real-Time Adverse Event Detection: Building an AI-Powered Pharmacovigilance Platform
Why Pharmacovigilance Needed AI
The regulatory mandate for pharmacovigilance is unambiguous: detect safety signals early, process adverse event reports within defined timelines, and submit to FDA FAERS and EMA EudraVigilance on schedule. What the mandate does not specify is how to do any of this when report volumes are climbing at 15 to 20 percent annually, the majority of incoming case narratives arrive as unstructured text across multiple languages, and your safety team is perpetually understaffed against that backlog.
A mid-size BioPharma company with six marketed products typically processes 30,000 to 60,000 individual case safety reports per year, more during label expansions and patient assistance program integrations. Before we built the platform described here, each ICSR took an average of 45 minutes from document intake through MedDRA coding, narrative completion, and database submission. A single safety data manager completed 10 to 12 cases per day at best. The math never worked in our favor.
The deeper problem was not throughput - it was signal detection latency. Traditional pharmacovigilance relies on spontaneous reporting patterns in structured safety databases. By the time enough reports accumulate to trigger a statistical signal, months or years may have passed since the first patients experienced the adverse event. The literature is clear on this: Uppsala Monitoring Centre research and the Eli Lilly signal management review published in Pharmaceutical Medicine in 2025 both confirm that machine learning methods - particularly ensemble approaches like random forest and gradient boosting - consistently outperform classical frequentist disproportionality measures like PRR and information component for early signal identification. The window between first signal emergence and detection in spontaneous reporting databases is where preventable harm lives.
Both problems - throughput and signal latency - are tractable with modern AI pharmacovigilance infrastructure: NLP adverse event detection for case processing, machine learning signal classifiers for aggregate database surveillance, and automated safety reporting pipelines for regulatory submission. The question was how to build something that the regulatory affairs team would trust, that would survive ICH E2B(R3) and FDA FAERS submission audits, and that safety scientists could actually use without needing to understand transformer architectures.
The Technical Challenge Nobody Warned Us About
The surface-level technical problem is named entity recognition: extract patient demographics, suspected drugs, adverse event terms, and causality assessments from unstructured case narratives. That part, while not trivial, has a well-established solution path. The harder problem is everything around it.
An incoming ICSR might be a four-paragraph consumer call center transcript, a 28-page hospital discharge summary, a post on a patient forum with no drug name, or a regional partner submission formatted to a pre-E2B(R3) template. All are valid adverse event sources under current regulations, and all require the same downstream workflow: extract the minimum dataset, code to MedDRA preferred terms, assess seriousness, determine reportability within 7 or 15-day regulatory timelines, and route to the appropriate submission pathway.
MedDRA coding is where most prior-generation systems broke down. The hierarchy has over 70,000 terms across five levels - from System Organ Class down to Lowest Level Terms - and mapping free-text adverse event descriptions to preferred terms requires both linguistic interpretation and clinical judgment. The FDA NLP evaluation study published in the Journal of Biomedical Informatics demonstrated precision and recall ranging between 64 and 83 percent for early NLP systems against gold-standard MedDRA annotations. That range was not acceptable for regulatory submission - you cannot submit a FAERS case with a 20 percent chance of incorrect MedDRA coding and expect it to survive an inspection. The solution required a different architecture than pure NLP: a human-in-the-loop review layer applied specifically to low-confidence term mappings, with the system flagging uncertainty rather than hiding it.
The regulatory architecture constraint was equally demanding. ICH E2B(R3) defines the data elements and XML schema for electronic ICSR submission. Every field in the output database has a defined data type, controlled vocabulary, and business rule. Any AI-generated output that does not conform to that schema fails at gateway validation before it ever reaches FAERS or EudraVigilance. The system had to be designed schema-first, not NLP-first - the extraction logic had to understand what E2B(R3) needed and work backward from those requirements.
What We Built
The platform operates in four functional layers.
Intake and triage. Reports arrive through multiple ingestion channels: direct API from call centers and license partners, email parsing for spontaneous reports, and an HL7 FHIR connector to the pharmacovigilance data warehouse. An ensemble classifier trained on 180,000 historical cases scores each document for probable reportability and urgency before a human has touched it. Cases with any serious outcome flag - hospitalization, death, life-threatening event - are elevated to a real-time review queue with an automated notification to the duty safety physician.
NLP adverse event extraction. A fine-tuned BioBERT variant extracts 47 structured fields from narrative text: patient identifiers, suspect and concomitant drugs, adverse event verbatim terms, onset dates, outcomes, and rechallenge information. The model outputs a confidence score for every field. Fields below a configurable threshold are flagged for case processor review with the source text span highlighted - the system surfaces uncertainty rather than hiding it.
MedDRA coding. A hierarchical classifier works top-down through the MedDRA hierarchy - System Organ Class to Preferred Term - and suggests up to three Lowest Level Terms ranked by confidence. Trained on 240,000 historical term mappings from the company's own case database, augmented by the WHO VigiBase corpus. Mappings above 0.85 confidence are accepted automatically; everything below goes to a coding specialist. All specialist overrides feed back to the model in monthly retraining cycles.
Signal detection and regulatory submission. Signal detection runs separately from individual case processing, operating continuously on the aggregate case database. Three detection methods run in parallel: information component disproportionality analysis, a random forest classifier for drug-event pairs matching known class-effect profiles, and a literature surveillance pipeline scanning PubMed and EMBASE on a 48-hour cycle. When two or more methods flag the same pair within 30 days, the signal moves to medical review. Approved cases flow to an automated submission module that produces ICH E2B(R3)-compliant XML, validates against the FDA E2B(R3) schema, and routes to FAERS via the Electronic Submissions Gateway or to EudraVigilance with EMA-specific field transformations. Gateway rejections trigger an immediate alert with the specific validation error identified.
Architecture Deep Dive
A few technical decisions shaped the platform in ways that are not obvious from the functional description above.
Confidence-gated automation, not full automation. Regulatory inspectors do not care about your automation rate - they care about the quality and traceability of every submitted case. We set thresholds conservatively: a case processes without human review only when extraction confidence exceeds 0.80 across all required fields, MedDRA coding confidence exceeds 0.85, and no seriousness flags are triggered. Under these thresholds, roughly 34 percent of our case mix processes fully automatically. That sounds modest. It translates to thousands of hours of case processor time redirected annually to complex cases that genuinely need expert attention.
The audit trail is an architectural primitive, not a logging afterthought. Every AI-generated field value is stored with its source text span, model version, confidence score, and timestamp. Every human review action is logged with user identity, review duration, and whether the AI suggestion was accepted or overridden. A sustained rise in overrides on a specific field type signals model drift and triggers retraining. When the FDA requested audit trail data during a routine inspection, the export covered every case submitted in the preceding 24 months with complete provenance from raw document to submission XML.
Literature surveillance and social media monitoring are scoped differently. The literature pipeline scans PubMed and EMBASE on a 48-hour refresh cycle and routes identified case reports through a deduplication check before creating new ICSRs - without deduplication, it floods the queue with reports authors have already submitted to regulators. Social media is handled more conservatively: academic work from the WEB-RADR project showed broad-ranging statistical signal detection from Twitter data produces AUC values of 0.47 to 0.53, essentially random. We run a targeted keyword watch for adverse event mentions requiring ICSR creation assessment, not population-level disproportionality analysis. That distinction matters for both regulatory defensibility and queue quality.
For a broader discussion of evaluating LLM components in regulated healthcare workflows, the LLM evaluation framework covers confidence calibration, evaluation dataset construction, and production monitoring approaches that apply directly to AI pharmacovigilance system design.
Results
After 14 months of production operation across a portfolio of six marketed products and approximately 50,000 cases processed per month, the outcomes were measurable across every dimension that mattered.
- ICSR processing time: reduced from 45 minutes to 8 minutes per case on average. For standard, non-complex spontaneous reports, the AI-assisted workflow reduced case processor time by 82 percent. For complex hospital case reports requiring significant medical narrative review, the reduction was more modest - around 40 percent - but still material given the labor intensity of those cases.
- MedDRA coding accuracy: 91 percent exact Preferred Term agreement on quarterly audits. Inter-rater variability between human coders on our pre-AI baseline ran 13 to 17 percent. The model was more consistent than the manual process it replaced.
- Regulatory submission timeliness: 99.1 percent of 7-day expedited reports on time, 99.7 percent of 15-day reports. Pre-platform rates were 94.3 and 97.1 percent respectively. Late-submission regulatory notifications to multiple authorities - a recurring item in our compliance calendar - effectively disappeared.
- Signal detection: 2 confirmed safety signals identified 3 months before they would have surfaced through traditional disproportionality analysis. One was a hepatotoxicity signal in a specific patient subpopulation, identified through the combination of literature surveillance and the random forest classifier. It was validated and incorporated into a label update. The 3-month lead time is the window in which prescribers were making treatment decisions without updated safety information.
- Workforce shift: 40 percent of case processor time moved from standard case processing to complex case review and signal assessment. This was a professional quality-of-work change, not headcount reduction. Turnover in pharmacovigilance operations dropped. The work became substantive.
Regulatory Acceptance
Submitting AI-generated pharmacovigilance data to FDA FAERS and EMA EudraVigilance requires something the current regulatory frameworks do not yet fully specify: demonstrating that your AI-assisted workflow produces outputs of equivalent or superior quality to manual processing, and that the system is validated, monitored, and auditable under computer system validation requirements.
We approached this through three mechanisms. First, a formal computer system validation per GAMP 5, with a performance qualification that included a 500-case prospective parallel comparison between the AI system and experienced case processors. Discrepancies were adjudicated by a senior medical officer. Those results became the baseline data in our change control package.
Second, a written AI governance policy in the quality management system documenting: the scope of AI automation, human review requirements per tier, model drift monitoring metrics, and the deviation procedure for post-submission errors. This policy had to exist before the first AI-processed case was submitted.
Third, the audit trail infrastructure. During an FDA inspection, the team reviewed 30 randomly selected submitted cases, tracing each from original source document to E2B(R3) XML. The complete provenance record was available for all 30 cases within 15 minutes of the request. The inspection finding on pharmacovigilance: no observations.
One practical note on E2B(R3): the ICH guidance and FDA implementation guide are prescriptive about the XML schema but silent on how you populate the fields - that silence leaves room for AI-assisted extraction. Build the schema validator into your QA pipeline as a hard gate, not a warning. A mandatory field left blank produces a gateway rejection, which creates a CAPA requirement regulators will expect closed in your next inspection.
The broader context on where regulators are heading with AI in safety reporting is covered in the state of AI in Life Sciences and Healthcare overview - the FDA's DSCSA Digital Enhancement initiative and EMA's strategy on artificial intelligence both signal that AI-assisted pharmacovigilance is moving from tolerated to expected in modern safety operations.
Lessons
There are several things I would tell a PM starting this kind of project that are not obvious from the outside.
Define your automation scope in advance and defend it. There will be pressure throughout the build to automate more - to push the human review tier further up the confidence threshold and eliminate the cases that still get human review. Resist this pressure until you have production data showing stable performance. The counterargument is regulatory: your global head of pharmacovigilance is the Qualified Person for Pharmacovigilance (QPPV) in the EU and bears personal regulatory responsibility for every ICSR submitted. That person will not sign off on a system they cannot explain to an inspector, and they should not. The AI is a tool in service of the QPPV's function, not a replacement for it.
MedDRA version management is a serious operational problem. MedDRA terminology updates twice per year, in March and September. Each release adds, modifies, or retires terms - which means your trained coding classifier produces deprecated suggestions if the model was not retrained before the new version went live. Build version tracking into the deployment pipeline and treat MedDRA release dates as hard dependencies in your release calendar.
Signal detection value requires organizational alignment to realize. Detecting a safety signal 3 months early only produces value if medical affairs and regulatory affairs have a defined process for acting on that detection. In our case, we had to design the signal assessment workflow alongside the detection capability - defining who reviews a potential signal, what escalation criteria trigger a Medical Safety Board review, and what the decision timeline looks like from first alert to label change proposal. The organizational workflow design took longer and required more executive alignment than building the model.
Build for the inspection from day one. Pharmacovigilance systems are inspected regularly - EU QPPV inspections, FDA CDER inspections, sponsor audits. The documentation in your quality management system for the AI platform is not overhead, it is the product. A technically excellent system with poor validation documentation will generate inspection findings. A technically solid system with complete, coherent validation documentation demonstrates organizational maturity.
AI pharmacovigilance is one of the few domains in Life Sciences and Life Sciences & Healthcare AI where the regulatory demand for rigor and the operational demand for throughput are in perfect alignment. The system has to be fast enough to meet 7-day reporting timelines at scale, and accurate enough to survive regulatory scrutiny. There is no tension between those requirements - they reinforce each other.
The platform described here processed over 600,000 ICSRs across 14 months. Every submitted case is traceable to its source document. Every coding decision is auditable. The safety signals it identified are documented in the benefit-risk database with clear detection provenance. That traceability is the actual deliverable - not the automation rate, not the NLP model's held-out test performance, not the throughput dashboard. The deliverable is a pharmacovigilance operation that the QPPV can stand behind in front of a regulator and explain completely.