Documentation

Website ↗ GitHub ↗

On this page

Temporary table (temp_table)

Temporary table (temp_table)¶

Materializes an intermediate temporary table via @dp.table(temporary=True), cleaned up when the pipeline run completes.

Minimal example¶

Add this entry to the actions list in your flowgroup.

- name: stage_orders
  type: transform
  transform_type: temp_table
  source: v_orders
  target: tmp_orders
  sql: "SELECT * FROM v_orders WHERE status = 'open'"

Options¶

Field	Type	Required	Default	Description
`source`	string / dict	Yes	—	Input view name (string), or a dict with a `view` / `source` key.
`sql`	string	No	—	Optional query. Use the input view name in the query. Without `sql` the action is a passthrough materialization of `source`.
`readMode`	string	No	`batch`	`batch` or `stream`.

The normal CLI resolves substitution tokens before this generator runs. A bare {source} in SQL is treated as a substitution token and fails validation if no such token is defined; use the view name or a defined ${token} instead.

Common fields¶

Fields accepted by every transform_type. Type-specific fields are listed in the options above.

Field	Type	Required	Default	Description
`name`	string	Yes	—	Action name, unique within the flowgroup.
`type`	string	Yes	—	Always `transform`.
`transform_type`	string	Yes	—	One of `sql`, `python`, `data_quality`, `temp_table`, `schema`.
`target`	string	Yes	—	Name of the produced view (or temporary table). Required for all transforms.
`source`	string / list / mapping	Yes	—	Upstream view name(s). Accepted shape and requirement vary per type — see this type’s options.
`description`	string	No	auto	Comment on the generated view/table. Defaults to an auto-generated string.
`operational_metadata`	bool / list	No	—	Add operational metadata columns; a list selects column names. Honored by `sql`, `python`, `data_quality`, `temp_table`; ignored by `schema`.
`depends_on`	list	No	—	Extra upstream table or view references added to the dependency graph. See Dependency analysis reference.