Federal health research platforms have a specific challenge with AI: the data is sensitive, the accuracy bar is high, and the compliance requirements are real. You cannot send protected health information to a commercial API, and you cannot publish AI-generated content to a national audience without review. But the volume of work, spanning literature, publications, content, and code, is large enough that ignoring AI entirely leaves real efficiency on the table.

We are integrating AI across several workstreams on a federal health research information platform. In each case, AI is in the loop but a human reviewer is always the last step before anything reaches the platform or its users.

Publication discovery

The platform surfaces health research publications to a national audience of clinicians, researchers, and policymakers. The volume of relevant literature across PubMed and other sources is high, and manual review processes struggle to keep pace with publication rates in active research areas.

We are using LLMs to assist with identifying and surfacing relevant publications: analyzing abstracts, flagging relevance to specific topic areas, and reducing the manual triage burden on the editorial team. The goal is better coverage, not automation of editorial judgment. Recommendations are reviewed before anything goes to the platform.

The model used for this workstream runs on-premises. Research content does not leave the environment.

LLM comparative evaluations

Before committing to a model for any content task, we run structured evaluations against real platform outputs. General benchmarks (MMLU, HumanEval, etc.) are not useful for this. What matters is whether a model produces accurate, consistent, and appropriately scoped outputs for the specific tasks the platform needs.

Our evaluation process scores models against a set of real examples from the platform: publication summaries, topic classifications, content refinements. We evaluate multiple models at each task, score outputs blind, and track consistency across runs. Cost per task is factored in alongside quality: the right answer at 10x the inference cost is not the right answer for production.

This approach has changed which models we use for different tasks. The model that performs best on creative text generation is not the same one that performs best on structured classification, even within the same platform.

AI-assisted development

The platform runs on Drupal, a large, complex codebase with active development across feature work, version upgrades, and infrastructure migration. AI tooling is now part of the engineering workflow across several areas:

  • Code review: flagging issues, inconsistencies, and potential regressions before human review
  • Documentation: generating inline documentation and technical write-ups from code, reviewed and edited by the team
  • Implementation acceleration: using AI to draft boilerplate, generate test cases, and work through repetitive implementation tasks, with all outputs reviewed before merge
  • Drupal upgrade support: Drupal 10 to 11 migrations involve large amounts of API deprecation work; AI tooling helps identify affected patterns across the codebase at scale

All outputs are reviewed by the project team before use. AI accelerates the workflow. It does not replace the engineer reviewing the output.

AI-powered CMS module vetting

The Drupal ecosystem has seen a wave of AI-powered modules: content generation, summarization, search, recommendation. Integrating any of these into a federal platform requires evaluation that goes beyond “does it work.”

We evaluate third-party AI modules against several criteria before any integration decision:

  • Data handling: where does the module send data, under what conditions, and can that be controlled or restricted
  • Output reliability: does the module produce consistent, accurate outputs, or does it hallucinate in ways that could affect platform content
  • Federal security requirements: FISMA, FedRAMP, and agency-specific ATO requirements constrain what external services can be in the data path
  • Maintainability: AI modules that depend on external services introduce a new failure mode; we evaluate what happens when the upstream service degrades or changes pricing

Most modules that look promising at the demo stage do not clear this bar. The ones that do tend to be those that can be configured to use a self-hosted or FedRAMP-authorized model endpoint.

Communications and dissemination

The platform produces a substantial volume of communications: reports, summaries, training materials, dissemination content for federal health audiences. AI tooling is in use across drafting and refinement, with project team review and approval on all final outputs.

For a team that produces content at this volume, AI assistance reduces the time from first draft to review-ready. The final product reflects the team’s expertise and editorial judgment. AI handles the blank page problem and the mechanical refinement work.

On-premises inference

For workstreams where content sensitivity requires it, inference runs on our on-premises hardware, NVIDIA DGX Spark units running large open-weights models at full scale. This lets us apply capable AI to federal health content without routing data through commercial APIs.

For workstreams where content is not sensitive and a commercial model performs better on the task, we use the appropriate tool. The goal is the right capability at the right sensitivity level, not a single model for everything.