Source and standard¶
Every clinical table in the OMOP CDM carries a pair of columns for the same clinical idea: one standardized, one preserved as it arrived. Understanding why both exist, and what belongs in each, prevents a category of error that is very hard to detect after the fact.
The pattern¶
For any clinical table, substituting the domain noun:
| Column | Contains | Standard? |
|---|---|---|
<domain>_concept_id |
The standard concept, or 0 | Always standard, or 0 |
<domain>_source_concept_id |
The concept representing the original code | Usually non-standard, or 0 |
<domain>_source_value |
The original code as a text string | Not a concept at all |
So for R. Alvarez's diabetes:
condition_concept_id = 201826 SNOMED, standard
condition_source_concept_id = 35208414 ICD-10-CM, non-standard
condition_source_value = 'E11.9' the literal string
Why three¶
Each answers a different question.
_concept_id answers "what does this mean, in shared terms". It is what every cohort definition, every characterization, and every network study queries. It is the analytical surface of your instance.
_source_concept_id answers "what did the source say, in structured form". It makes your translation auditable. A reviewer can query for every row where the source concept was ICD-10-CM E11.9 and see what it became. It also lets you re-run a mapping without going back to the original files, which becomes important when a vocabulary release changes a mapping and you need to know what is affected.
_source_value answers "what did the source literally say". It exists because concepts sometimes fail to capture local reality. A local code with no concept at all still has a string. A code that was mistyped in the source is preserved as it was. When someone asks why a particular row looks strange, this column is where the investigation starts.
What goes wrong when one is skipped¶
Skipping _source_concept_id. Extremely common, because it takes an extra lookup and the pipeline runs fine without it. The cost appears months later, when a vocabulary update changes a mapping and nobody can determine which rows were affected without reprocessing the raw source. If the raw source has been rotated out of storage, that answer is gone permanently.
Skipping _source_value. Less common but worse for local codes. If a code has no concept, _source_concept_id is 0 and _source_value is the only remaining trace. Skipping it means the row is a person, a date, and two zeros, which is indistinguishable from a bug.
Putting the source concept in _concept_id. This one is silently destructive. The row looks populated, the counts look right, and every analysis misses it, because analyses query on standard concepts and this one is not standard. It presents as "our diabetes cohort is smaller than expected" with no obvious cause.
The FHIR side of the pair¶
FHIR gives you the raw material for all three columns, and often for more than one coding.
A CodeableConcept can carry several coding entries plus a text. A well-behaved source might send:
"code": {
"coding": [
{ "system": "http://hl7.org/fhir/sid/icd-10-cm", "code": "E11.9",
"display": "Type 2 diabetes mellitus without complications" },
{ "system": "http://snomed.info/sct", "code": "44054006",
"display": "Diabetes mellitus type 2" }
],
"text": "Type 2 DM"
}
Now you have a choice, and it deserves a written policy rather than whatever your parser does first.
A reasonable policy: prefer a coding that is already standard in OMOP, because it removes a translation step and its associated risk. Here that is the SNOMED coding. Record the ICD-10-CM coding as the source concept if it corresponds to how the source system actually stores the data, and put the text into _source_value if no code string is more informative.
A different reasonable policy: always take the coding from the system your source system considers primary, translate it, and record it as the source. This preserves a truer picture of the source's own practice, which helps if you are characterizing coding behavior rather than clinical facts.
Both are defensible. What is not defensible is having no policy, because then the choice varies by resource depending on array ordering, and your instance contains two different conventions with no marker distinguishing them.
Write the coding-selection policy down before you write the code
Include it in your ETL documentation with an example. It is one of the first questions a careful reviewer asks and one of the last things anyone remembers to document.
SOURCE_TO_CONCEPT_MAP¶
When a source code has no standard concept and you decide to map it yourself, SOURCE_TO_CONCEPT_MAP is where the mapping lives.
It is a simple table: source vocabulary, source code, target concept, and validity dates. Its value is that the mapping is data rather than code. Someone can inspect it, review it, version it, and disagree with it without reading your transformation logic.
Local mappings should be treated as a documented interpretation. For each entry, a reviewer should be able to find out who made the decision, what alternatives were considered, and what confidence was assigned. That is not bureaucracy; it is what makes the mapping defensible when a study result depends on it.
For R. Alvarez's local frailty score, a SOURCE_TO_CONCEPT_MAP entry might point FRLTY-7 at a general frailty assessment concept. That is a real interpretive act: the seven-point local scale and the standard concept are not the same instrument, and an analyst pooling this with frailty data from another site would be combining different measurements. Whether that is acceptable depends entirely on the research question, which is exactly why the decision belongs in reviewable data rather than buried in code.
Where this quietly breaks¶
Mapping to a concept that is standard but in the wrong domain. Check the domain of the target, not just its standard status. A standard concept in the Observation domain does not belong in CONDITION_OCCURRENCE no matter how much the source resource looked like a condition.
Truncating _source_value. These columns have length limits, and long local codes or concatenated code-plus-description strings get cut. Truncation at the wrong point can make two distinct source codes look identical.
Reusing _source_value for something else. It is tempting to stuff a composite key or a debugging string in there. Do not. Downstream tools and reviewers read this column as "the source code" and will be misled.
Letting local mappings drift out of sync with vocabulary releases. A local mapping pointing at a concept that has since been deprecated will not error. It will just point somewhere current tools no longer look. Re-validate local maps when you upgrade vocabularies.