Back to Home
Data Processing Pipeline

Data
Processing

The technical pipeline of how our engine ingests, cleans, and validates metadata for streaming platform submission — from raw payload to certified delivery, in under 0.3 seconds.

Six Stages

The Pipeline

Stage 01

Ingestion

Metadata is received via API (JSON), file upload (DDEX XML, CSV, XLSX), or direct integration with a distributor's pipeline. Every payload is immediately hashed (SHA-256) and stored in immutable, encrypted storage. The original file is never modified — all subsequent processing operates on an in-memory copy.

Supported formats: JSON, DDEX XML, CSV, XLSX
TLS 1.3 encryption in transit
Payload hash computed on receipt for integrity tracking
Regional routing: EU, US, or APAC processing infrastructure
Stage 02

Parsing & Normalization

The ingestion parser converts the incoming format into our internal canonical schema — a standardized JSON structure that represents metadata in a format-agnostic way. This normalization step ensures that regardless of how metadata arrives, it's processed identically downstream. Character encoding is normalized to UTF-8, whitespace is trimmed, and empty fields are flagged.

Format-agnostic canonical schema
Automatic UTF-8 encoding normalization
Whitespace trimming and field cleanup
Empty and null field detection
Stage 03

Field-Level Validation

The core engine validates every field against 2,400+ rules organized by target store specification. Each field is checked for type, format, length, character encoding, and store-specific requirements. Validations run in parallel across all target stores — a release destined for 6 stores is validated against all 6 simultaneously, not sequentially.

2,400+ validation rules across 150+ store specifications
Parallel multi-store validation
ISRC (ISO 3901) and UPC (GS1) checksum verification
DDEX contributor role enforcement
Genre taxonomy mapping with confidence scores
Stage 04

Cross-Reference & Enrichment

The engine cross-references identifiers against external registries — MusicBrainz for recording data, ISRC registries for code validity, and store-specific taxonomies for genre mapping. This enrichment step catches mismatches that field-level validation alone cannot detect: an ISRC that's structurally valid but assigned to a different recording, or a genre tag that maps to the wrong browse category.

MusicBrainz cross-referencing for recording data
ISRC registry validation
Per-store genre taxonomy mapping
Contributor name disambiguation via ISNI
Artwork pixel-dimension verification
Stage 05

Result Compilation

Validation results are compiled into a field-by-field compliance report. Each issue includes the field name, the store that flagged it, the severity (error or warning), the rule that was violated, and a suggested fix. The report is structured as JSON for API consumers and rendered as a PDF for human review. Every result is timestamped and bound to the original payload hash.

Field-by-field compliance report
Severity classification: error, warning, pass
AI-driven fix suggestions for every issue
JSON and PDF output formats
Cryptographic binding to original payload
Stage 06

Certification & Delivery

If validation passes (or passes with warnings), a compliance certificate is generated with a SHA-256 hash binding the metadata payload to its validation result. The certificate is stored immutably for 7 years. The validated metadata and certificate are then ready for delivery to the target stores via the distributor's standard delivery pipeline — or via our API for automated delivery.

Cryptographic compliance certificate (SHA-256)
7-year immutable certificate retention
API-ready delivery package
Webhook notification on validation completion
Tamper-evident audit trail
< 0.3s
Avg. Validation Time
2,400+
Validation Rules
150+
Store Specifications
10,000
Max Batch Size

See it in action

Run your metadata through the full pipeline and get a field-by-field compliance report in under a second.