On this page

Dependency analysis reference

lhp dag builds its graph from the data each action reads and produces. Most edges come from an action’s configured source or from table reads extracted from its SQL and Python bodies. Use the action-level depends_on field when an upstream cannot be discovered statically.

This field declares data dependencies. Its values are table or view references, not action names, flowgroup names, or pipeline names.

Important

Do not confuse the action-level depends_on field documented here with the depends_on keys in dependency-analysis JSON or generated Databricks job YAML. Those contain pipeline or task names derived from the completed graph.

depends_on field

depends_on is optional and is accepted on every action type.

Property

Contract

Location

On an action, alongside name, type, source, and target.

Type

List of non-empty strings.

Default

Unset. An empty list has no effect.

Accepted shapes

catalog.schema.table, schema.table, or a single-part name. See How references match for the different matching rules.

Graph behavior

Additive by default: declared references are added to sources discovered from configuration, SQL, and Python.

Runtime behavior

None. The field is used by dependency analysis only and is never emitted into generated pipeline code.

Validation

Invalid values raise LHP-VAL-063.

Example

This Python transform reads exchange rates through runtime-only logic that static analysis cannot resolve:

pipelines/silver_orders.yaml
actions:
  - name: enrich_orders
    type: transform
    transform_type: python
    source: v_orders
    module_path: transforms/enrich_orders.py
    function_name: run
    depends_on:
      - ${catalog}.${reference_schema}.exchange_rates
    target: v_enriched_orders

If another action in the analyzed project produces the identically authored ${catalog}.${reference_schema}.exchange_rates table, LHP creates an edge from that producer to enrich_orders. If no producer matches, the reference is reported as an external source.

How the DAG is built

LHP constructs the graph in four steps:

  1. Collect producers. A load or transform target registers a temporary view. A streaming-table or materialized-view write registers its table. A Delta sink with options.tableName also registers a table; other sinks do not.

  2. Collect sources. For each action, LHP combines configured source references, table reads extracted from SQL and Python, and the action’s depends_on entries.

  3. Match sources to producers. Each match creates an action edge pointing from producer to consumer. An unmatched source remains external.

  4. Roll up the graph. Cross-action edges become flowgroup edges when the actions belong to different flowgroups, then pipeline edges when those flowgroups belong to different pipelines.

The rolled-up pipeline graph determines execution stages, cycle detection, JSON and DOT output, and the task dependencies in orchestration jobs generated by lhp dag --format job.

Default mode: additive declarations

In the default mode, depends_on adds declarations without replacing what LHP can discover:

action sources = configured sources
               + SQL/Python-extracted sources
               + depends_on declarations

This makes the field suitable as an escape hatch for one dynamic or otherwise opaque read while retaining every statically discoverable edge on the action. Repeated references collapse to the same graph edge.

Warning suppression

A non-empty depends_on suppresses that action’s LHP-DEP-002 and LHP-DEP-003 extraction advisories. Suppression applies to the whole action, not to an individual read: LHP cannot determine which declared reference was intended to cover which unresolved expression.

Important

If one action has several unresolved reads, declaring only one upstream still suppresses the advisories for all of them. Declare every upstream the action requires.

Default mode still reads and parses the action’s SQL and Python bodies. A non-empty depends_on suppresses extraction advisories, but it does not bypass missing-file errors or other failures encountered while loading those bodies.

Authoritative mode

Use --trust-depends-on when actions fully declare their upstreams and body extraction is unnecessarily expensive:

$ lhp dag --trust-depends-on

For each action with a non-empty depends_on list, LHP skips SQL and Python body extraction entirely. Its source set becomes:

action sources = configured sources + depends_on declarations

Actions without depends_on continue to use normal extraction.

Behavior

Default

--trust-depends-on

Parse bodies for an action with declarations

Yes

No

Keep statically extracted reads

Yes

No

Keep configured source references

Yes

Yes

Add declared references

Yes

Yes

Surface extraction advisories for that action

No

No extraction runs

Encounter missing SQL/Python body files for that action

Possible

No

Warning

Trust mode treats declarations as authoritative. An incomplete list silently omits the undeclared body-derived edges, which can produce an incomplete DAG and incorrect orchestration order.

How references match

depends_on entries use the same matcher as automatically discovered reads. Before comparison, LHP lowercases each dotted part, removes backticks, and trims surrounding whitespace.

Declared reference

Matching behavior

catalog.schema.table

Matches the same canonical three-part table. It can also match a two-part Delta-sink producer with an omitted catalog; it never suffix-matches a producer in a different explicit catalog.

schema.table

Matches a three-part producer only when exactly one registered catalog produces that schema.table. Multiple catalogs make the reference ambiguous, so it remains external.

view_name

Matches a load or transform target only within the same pipeline. A single-part name does not match a global table producer.

No matching producer

Remains an external source; LHP does not create a phantom edge.

Views are pipeline-scoped because Lakeflow temporary views exist only within their pipeline. Tables are global and can form edges across pipelines.

Substitution tokens

Dependency analysis does not resolve ${...} substitutions. Tokens remain literal parts of the reference and must be authored consistently on both sides of the edge. For example, a declaration containing ${catalog}.silver.orders does not match a producer named dev_catalog.silver.orders.

Spell depends_on in the same literal form as the producer when the producer and consumer otherwise use different substitution forms.

Validation and edge cases

Each entry must be a non-empty string with no more than three dot-separated parts and no blank part. These are invalid:

depends_on:
  - ""
  - catalog..table
  - workspace.catalog.schema.table

They raise LHP-VAL-063. The following values are structurally valid but can still remain external:

  • a well-formed table that no analyzed action produces;

  • an ambiguous two-part table produced in multiple catalogs;

  • a single-part name with no matching view in the same pipeline; or

  • a tokenized reference whose literal spelling differs from the producer.

Manually declared edges participate in cycle detection exactly like discovered edges. A declaration that points back to the consumer, directly or through other actions, can produce LHP-DEP-001.

What the field does not do

Action-level depends_on does not:

  • add a Spark read to generated code;

  • change generation-time action ordering inside a flowgroup;

  • rename references for sandbox table isolation;

  • require an upstream to exist in the project; or

  • directly name a Databricks job task.

It changes the dependency-analysis graph. Pipeline and job ordering are then derived from that graph.