On this page

Lookup matches (all_lookups_found)

Asserts that every source row resolves against a lookup table.

Minimal example

Add this entry to the actions list in your flowgroup.

- name: orders_product_lookup
  type: test
  test_type: all_lookups_found
  source: v_orders_clean
  lookup_table: main.sales.products
  lookup_columns: [product_id]
  lookup_result_columns: [id]

Options

Field

Type

Required

Default

Description

source

string

Yes

—

Table or view holding the lookup keys.

lookup_table

string

Yes

—

Three-part name (catalog.schema.table) of the lookup table.

lookup_columns

list

Yes

—

Key columns in source; equal length to lookup_result_columns.

lookup_result_columns

list

Yes

—

Matching columns in lookup_table; equal length to lookup_columns.

on_violation

string

No

fail

fail, warn, or drop.

Base fields

Common to every test action, regardless of test_type.

Field

Type

Required

Default

Description

name

string

Yes

—

Action name; unique within the flowgroup.

type

string

Yes

—

Must be test.

test_type

string

Yes

—

One of the nine supported test types. Any other value is rejected at validation.

target

string

No

tmp_test_<name>

Name of the generated temporary table.

description

string

No

Test: <test_type>

Table comment and function docstring.

depends_on

list[string]

No

—

Extra upstream table or view references added to the dependency graph. See Dependency analysis reference.

on_violation

string

No

fail

fail, warn, or drop; invalid values coerced to fail. Selects the decorator for the seven built-in types; for custom_sql / custom_expectations the decorator comes from each entry in expectations.

test_id

string

No

—

Correlates this test with a test-reporting hook.

Run lhp validate --env dev --include-tests and lhp generate --env dev --include-tests to include test actions. For result delivery, see Test reporting configuration.

A test action always reads its source with batch SQL. Point it at a table or a batch view: a streaming view fails on Databricks, and lhp validate reports it as LHP-VAL-070.

Generated output

Every test action generates a function decorated with @dp.table(name="<target>", comment="<description>", temporary=True) that returns the violation query, plus one expectation decorator per violation bucket. The on_violation value maps to the decorator as follows:

on_violation

Decorator

Behavior

fail

@dp.expect_all_or_fail

Pipeline update fails when any row violates.

warn

@dp.expect_all

Violations recorded as metrics; rows retained.

drop

@dp.expect_all_or_drop

Violating rows dropped from the output.

For the seven built-in types the action-level on_violation drives the decorator. For custom_sql and custom_expectations the decorators are built from each entry in expectations using that entry’s own on_violation.