On this page

Row counts (row_count)

Compares record counts between exactly two sources within a tolerance.

Minimal example

Add this entry to the actions list in your flowgroup.

- name: orders_row_count
  type: test
  test_type: row_count
  source: [v_orders_raw, v_orders_clean]
  tolerance: 0

Options

Field

Type

Required

Default

Description

source

list

Yes

—

Exactly two tables/views whose counts are compared.

tolerance

integer

No

0

Allowed absolute difference in row counts.

on_violation

string

No

fail

fail, warn, or drop.

Base fields

Common to every test action, regardless of test_type.

Field

Type

Required

Default

Description

name

string

Yes

—

Action name; unique within the flowgroup.

type

string

Yes

—

Must be test.

test_type

string

Yes

—

One of the nine supported test types. Any other value is rejected at validation.

target

string

No

tmp_test_<name>

Name of the generated temporary table.

description

string

No

Test: <test_type>

Table comment and function docstring.

depends_on

list[string]

No

—

Extra upstream table or view references added to the dependency graph. See Dependency analysis reference.

on_violation

string

No

fail

fail, warn, or drop; invalid values coerced to fail. Selects the decorator for the seven built-in types; for custom_sql / custom_expectations the decorator comes from each entry in expectations.

test_id

string

No

—

Correlates this test with a test-reporting hook.

Run lhp validate --env dev --include-tests and lhp generate --env dev --include-tests to include test actions. For result delivery, see Test reporting configuration.

A test action always reads its source with batch SQL. Point it at a table or a batch view: a streaming view fails on Databricks, and lhp validate reports it as LHP-VAL-070.

Generated output

Every test action generates a function decorated with @dp.table(name="<target>", comment="<description>", temporary=True) that returns the violation query, plus one expectation decorator per violation bucket. The on_violation value maps to the decorator as follows:

on_violation

Decorator

Behavior

fail

@dp.expect_all_or_fail

Pipeline update fails when any row violates.

warn

@dp.expect_all

Violations recorded as metrics; rows retained.

drop

@dp.expect_all_or_drop

Violating rows dropped from the output.

For the seven built-in types the action-level on_violation drives the decorator. For custom_sql and custom_expectations the decorators are built from each entry in expectations using that entry’s own on_violation.