omx validation

Semantic validation tests — inferred from compiled IR, executed against live warehouse data.

Tests are derived automatically from the metric SQL. No YAML. No configuration files. The compiler infers what to test from the aggregation type, time columns, and join paths.

Commands

omx validation list

List all inferred tests without running them.

omx validation list
omx validation list --confidence all       # include low-confidence tests
omx validation list --confidence medium     # medium + high confidence
omx validation list --json                  # machine-readable

Output:

INFERRED VALIDATION TESTS

  HIGH confidence (22 tests):
    o total_amount — Non-negative values
      Sum(total_amount) >= 0 · Inferred from Sum aggregation
    o total_amount — Freshness check
      MAX(invoice_date) within 24h · Inferred from time_column "invoice_date"
    o total_amount — Table not empty
      COUNT(*) > 0 on invoices · Base table must contain data
    o top_products — Referential integrity
      invoice_items.product_id -> products.id · Inferred from join path

  22 tests across 7 metrics

omx validation run

Execute inferred tests against live warehouse data.

omx validation run                         # HIGH confidence only (default)
omx validation run --confidence all        # all confidence tiers
omx validation run --confidence medium     # medium + high
omx validation run --quiet                 # CI mode: exit 0 = pass, 1 = fail
omx validation run --json                  # machine-readable output

Output:

VALIDATION RESULTS — 22 tests

  11 passing    3 failing    0 warning

  FAILURES:

  FAIL  total_revenue · Non-negative values
  |-- Severity:       HIGH
  |-- Rows affected:  7,901 (17.6% of 44,876)
  |-- Confidence:     HIGH
  `-- Inspect:        SELECT * FROM invoices WHERE total_amount < 0 LIMIT 10

  PASSING: 11 tests
    v total_amount — Table not empty
    v total_revenue — Table not empty
    v top_products — Table not empty, Referential integrity

Inferred Test Rules

Rule	Trigger	Confidence	What it checks
Non-negative	SUM aggregation in IR	HIGH	MIN(column) >= 0
Positive integer	COUNT aggregation in IR	HIGH	COUNT(*) >= 0
Freshness	time_column declared	HIGH	MAX(time_col) within 24h
Table not empty	Any structured metric	HIGH	COUNT(*) > 0
Referential integrity	Join path in IR	HIGH	Orphan foreign keys = 0
Bounded ratio	COUNT/COUNT pattern	HIGH	Result between 0 and 1

CI Integration

Use --quiet for CI pipelines:

omx validation run --quiet
# Exit code: 0 = all tests pass, 1 = failures exist

In GitHub Actions:

- name: Validate metrics
  run: |
    pip install onlymetrix
    omx validation run --quiet
  env:
    OMX_API_KEY: ${{ secrets.OMX_API_KEY }}