Skip to content

OSTI to/from xBRC Conversion

This page describes how this project converts metadata between OSTI E-Link records and the xBRC schema used for BRC data on bioenergy.org.

In this repository, xBRC refers to the cross-BRC metadata model represented by brc_schema.yaml. It captures the fields needed by the Bioenergy Research Centers and the bioenergy.org data portal, including BRC affiliation, dataset landing pages, BRC-specific topics and themes, organism metadata, repository metadata, and portal-oriented status fields.

The conversion logic lives in:

  • src/brc_schema/transform/osti_to_brc.yaml
  • src/brc_schema/transform/brc_to_osti.yaml
  • src/brc_schema/transform.py

When to Use Each Direction

Use osti_to_brc when records have been retrieved from OSTI E-Link and need to be loaded, validated, or reviewed as xBRC DatasetCollection data:

uv run brcschema transform -T osti_to_brc -o brc_datasets.yaml osti_records.json

Use brc_to_osti when xBRC metadata needs to be prepared for OSTI E-Link submission or update workflows:

uv run brcschema transform -T brc_to_osti -o osti_records.yaml brc_datasets.yaml

Input files may be JSON or YAML. Output format is selected from the output file extension.

OSTI to xBRC

The osti_to_brc transform expects an OSTI-style wrapper object with a records array and produces an xBRC DatasetCollection with datasets.

Key mappings include:

OSTI source xBRC target Notes
records datasets Each OSTI Record becomes one xBRC Dataset.
title title Direct mapping.
description description, abstract OSTI description is copied into both xBRC fields.
publication_date or entry_date date Publication date is preferred.
osti_id identifier Rendered as https://www.osti.gov/biblio/<osti_id>.
doi bibliographicCitation Rendered as https://doi.org/<doi>.
keywords or subjects keywords Comma-separated strings are split into individual keywords.
persons, authors creator, contributors Structured persons are preferred; legacy author strings are fallback input. Every person affiliation is kept, in order, in affiliations (name → organizationName, ror_id → ror: CURIE); the deprecated affiliation field receives the first affiliation name. Creator order follows OSTI author order; OSTI has no author-role metadata, so roles are positional: the first author gets individual_roles: [FirstAuthor], all others [ContributingAuthor]. primaryContact goes to the author matching an OSTI CONTACT person (by ORCID, email, or name), else the first author.
organizations with type: AUTHOR organizational_creators Consortium, collaboration, and institutional authors, in order. Kept alongside individual creator entries rather than replacing them. A dataset needs creator, organizational_creators, or both.
sponsor organizations or sponsor_orgs funding Award identifiers are copied when available.
related_identifiers relatedItem OSTI related identifiers become xBRC related items.
identifiers, related_identifiers, and legacy identifier fields has_related_ids Contract numbers and taxon identifiers are excluded; BioProject URLs and IDs are normalized where possible.
taxon URLs in related_identifiers, and organism names in keywords species NCBI Taxonomy IDs go to NCBITaxID, JGI GOLD and IMG IDs to taxon_ids. See Species and Taxonomy Identifiers.
media media OSTI media package metadata is preserved.
geolocations geolocations Points, boxes, and polygons are kept with their labels; OSTI type → geolocation_type. A missing type is inferred from the point count (1 → POINT, 2 → BOX, 3 or more → POLYGON), and an open polygon is closed on its first point. A geolocation with an out-of-range coordinate or a point count that does not fit its type is dropped.
site_url or first links entry dataset_url site_url is preferred.
workflow_status active R maps to true; missing status maps to false.

xBRC Affiliation

The xBRC brc field is derived from OSTI metadata in this order:

  1. Use site_ownership_code directly when present.
  2. Otherwise inspect contract fields and CN_DOE identifiers.
  3. Map known BRC contract numbers:
Contract number xBRC brc
SC0018420 CABBI
SC0018409 GLBRC
AC36-08GO28308 CBI
AC02-05CH11231 JBEI

If neither site_ownership_code nor a known contract number is present, the transform cannot infer the BRC affiliation.

Current xBRC-Specific Gaps

Some xBRC fields are more specific than the current OSTI E-Link record model or are intended for bioenergy.org display and filtering. These fields are not currently populated from OSTI records unless the transform explicitly maps them:

  • plasmid_features
  • analysisType
  • datasetType
  • topic
  • theme
  • category
  • ontology_annotations

OSTI has no dedicated species field either, but species is populated from taxon identifiers in related identifiers and from organism names in keywords. Species and Taxonomy Identifiers describes what to include in OSTI records so that organisms come through.

xBRC to OSTI

The brc_to_osti transform expects an xBRC DatasetCollection and produces an OSTI-style object with a records array.

Key mappings include:

xBRC source OSTI target Notes
datasets records Each xBRC Dataset becomes one OSTI Record.
title title Direct mapping.
description description Direct mapping.
date publication_date, released_to_osti_date The same xBRC date is used for both OSTI fields.
identifier osti_id Only extracted when the identifier contains osti.gov/biblio/.
bibliographicCitation doi Supports https://doi.org/... and doi:... forms.
brc site_ownership_code Direct BRC code passthrough.
keywords keywords Direct mapping.
dataset_url or non-OSTI identifier site_url, links The dataset landing page is preferred.
active workflow_status true maps to R; falsey values map to SR.
creator, contributors authors, persons Names are split into OSTI first/middle/last fields when possible. Author order is preserved; individual_roles has no E-Link 2.0 equivalent and is not exported. All affiliations are exported with bare ROR IDs; the deprecated affiliation name is used only when affiliations is absent.
organizational_creators organizations (type: AUTHOR) ROR IDs are exported in bare form. Entries without an organizationName are skipped, because OSTI requires a name.
funding, brc organizations, sponsor_orgs, research_orgs xBRC funding and BRC affiliation are converted into OSTI organization structures.
has_related_ids identifiers, related_identifiers BioProject IDs and DOI references get special handling.
species related_identifiers Each NCBITaxID, GOLD ID, and IMG taxon OID becomes a URL related identifier with relation References. Species with a name only are not written.
media media Media package metadata is preserved.
geolocations geolocations Shape, label, and coordinates are exported with the same checks as the import. OSTI has no fields for altitude, horizontal_accuracy, or vertical_accuracy, so they are not exported.

The transform also sets these OSTI defaults:

  • access_limitations: ["UNL"]
  • product_type: DA

BRC-Derived OSTI Fields

When xBRC brc is present, the transform emits BRC-specific OSTI metadata:

  • site_ownership_code
  • DOE contract identifiers
  • default sponsor organization information when no xBRC funding is provided
  • BRC research organization names for OSTI research_orgs

This behavior is specific to xBRC. Generic metadata conversion tools should not assume that all BRC-derived OSTI fields can be recovered from arbitrary OSTI records.

Round-Trip Expectations

The transforms are designed for operational metadata exchange, not a lossless round trip. Several fields are normalized, inferred, defaulted, or omitted.

Expected non-lossless behavior includes:

  • OSTI description maps to both xBRC description and abstract; converting back uses only xBRC description.
  • xBRC active is represented as OSTI workflow_status, not as a native xBRC boolean in OSTI.
  • xBRC-specific display and filtering fields, such as topic, theme, category, and ontology_annotations, are not currently emitted to OSTI.
  • OSTI fields without xBRC equivalents may be omitted unless they have an explicit mapping.
  • xBRC species entries that have a name but no identifier are not written to OSTI; entries with identifiers round-trip through related_identifiers.
  • xBRC geolocation altitude, horizontal_accuracy, and vertical_accuracy are not written to OSTI, which has no equivalent fields.
  • BRC affiliation may be inferred from contract numbers in osti_to_brc, while brc_to_osti emits contract numbers from the xBRC brc value.

For conversion validation, use the repository test suite:

make test