Skip to content

Proving it worked

The pipeline ran without errors. That tells you the code executed. It tells you nothing about whether the data is right.

Four levels of checking

Each catches things the others miss, and skipping any one leaves a characteristic class of defect.

One: structural conformance

Does the instance match the OMOP CDM specification? Correct tables, correct columns, correct data types, primary keys unique, foreign keys resolving, required fields populated.

Cheap, automatable, and catches only the most mechanical problems. A structurally perfect instance can be completely wrong.

Two: systematic quality checks

The Data Quality Dashboard runs a large library of checks against a loaded instance, organized by the Kahn framework categories of conformance, completeness, and plausibility, and evaluated at table, field, and concept level.

Examples of what it catches: concept IDs that do not exist in the vocabulary, concept IDs in the wrong domain for their table, dates outside plausible ranges, events before birth or after death, values outside plausible ranges for their measurement concept, required fields with high null rates.

Run it. Read the failures individually. Some are real defects, some are expected properties of your source, and the work is deciding which is which and writing down the reasoning for the expected ones so the next person does not redo it.

Three: characterization against expectation

Achilles produces descriptive statistics across the instance: counts by table, distributions by concept, temporal patterns, demographic breakdowns.

The check is not whether the numbers are right in the abstract. It is whether they match what you expect from a source you know. A pediatric hospital's instance with a median age of fifty-four is a problem no structural check will find.

Questions to ask of the characterization:

  • Does the age and sex distribution match the source population?
  • Do visit counts per person look clinically plausible?
  • Are the top twenty conditions the ones you would expect for this population?
  • Does the temporal pattern show expected seasonality and unexpected cliffs?
  • Are there impossible combinations, such as prostate procedures in female patients?
  • Does the concept distribution show a suspiciously large share for one general concept, which usually means over-broad mapping?

Four: clinical review

Someone who knows the clinical domain looks at a sample of records end to end and says whether they make sense.

This is the level most often skipped and the one that catches the errors that matter most. Pull twenty people, print their full OMOP CDM record, and hand it to a clinician who knows the population. They will find things no automated check can: a diagnosis that does not fit the medication list, a visit pattern that does not match how care is actually delivered, a laboratory value that is clinically impossible in a way that is within the plausible range.

Budget for this. It is a half-day of a clinician's time and it is the highest-yield quality activity available.

Checks specific to FHIR to OMOP

The generic tooling does not know about your source. Build these yourself.

Resource counts to row counts, by type. For each FHIR resource type, how many resources went in and how many OMOP CDM rows came out, by destination table. Every discrepancy should have a named explanation: routed to a different domain, filtered by status, unmapped and excluded, or split into multiple rows.

This single report answers most questions a reviewer will ask, and building it is a few hours of work.

Domain distribution for each resource type. How many Condition resources went to CONDITION_OCCURRENCE, OBSERVATION, MEASUREMENT, elsewhere. If any resource type shows one hundred percent to its obvious destination, you probably routed on resource type.

Unmapped rate, broken out. By source vocabulary, by domain, by resource type. A single overall percentage hides where the problem is.

Top unmapped codes by volume. The most actionable single artifact in the whole quality suite.

Reference resolution rate. What proportion of subject, encounter, and context references resolved to something in the export. Low rates mean either an incomplete export or a linking defect, and the two have different remedies.

Date precision report. How many dates were year-only, month-only, absent, or imputed, by field. This is the report that tells an analyst whether age-based analyses are safe.

Visit construction report. Source encounters in, visits out, details out, the collapse ratio, and the parameters used.

Type concept distribution. For DRUG_EXPOSURE and CONDITION_OCCURRENCE particularly. A single type concept dominating at ninety-nine percent means provenance was probably assigned mechanically.

Reading a failure back to a decision

The skill that separates someone who runs quality checks from someone who uses them.

A worked example. The Data Quality Dashboard reports an implausibly high rate of a general diabetes concept relative to specific ones.

Trace it: which source codes map to that concept? If a local code was mapped to a general concept because no specific equivalent existed, that is a mapping decision showing up as a quality signal. The check is working correctly and the finding is a documentation item rather than a bug. If instead the specific codes are present in source and being mapped up, that is a real defect in the mapping step.

Same signal, two causes, opposite responses. The lineage from the staging chapter is what lets you tell them apart in minutes rather than days.

A second example. Achilles shows a sharp drop in condition counts starting in a particular month.

Candidate causes: the source system changed coding practice, a resource type stopped being exported, a status filter began excluding more records, or the export was incomplete for that period. Check the raw layer counts first. If raw dropped too, it is a source or export issue. If raw held steady and OMOP CDM dropped, it is your pipeline.

What to publish about your instance

Whether the consumer is a colleague, a collaborating site, or a reviewer, these travel with the data.

  • CDM_SOURCE fully populated, with a real description
  • OMOP CDM version and vocabulary version
  • Population definition and how the Group was constructed
  • Date range covered
  • Observation period derivation rule
  • Resource-to-row reconciliation
  • Unmapped rate by domain and top unmapped codes
  • Known gaps and empty tables, with reasons
  • Every imputation and default, with affected counts
  • Data Quality Dashboard results with a note on each expected failure

That list looks long and most of it is generated rather than written. Assembled once as a template, it costs little per subsequent load and it is the difference between a data set a researcher can use responsibly and one they have to take on faith.

A note on expectations

A first FHIR to OMOP load will have problems. Every one does. The measure of a good pipeline is not that quality checks pass on the first run; it is that failures are findable, explainable, and traceable to a decision that can be revisited.

Teams that expect a clean first run tend to respond to failures by suppressing checks. Teams that expect failures respond by investigating them, and they end up with better instances and much better documentation.

Next