On this page

Load actions

Choose a source to see its minimal YAML, source options and common action fields together. Every load sets type: load and a source mapping whose type selects the reader. For step-by-step examples, see Read data.

Action-level fields

These fields apply to every load sub-type. name, type, source and target are required.

Field

Type

Required

Default

Description

name

string

Yes

—

Unique action name within the flowgroup.

type

string

Yes

—

Always load.

source

mapping

Yes

—

Source configuration. source.type selects the sub-type. Use a mapping for every load type, including SQL.

target

string

Yes

—

Name of the temporary view this action creates.

readMode

string

No

sub-type

batch or stream; selects spark.read vs spark.readStream. Default and constraints vary by sub-type (below). Ignored by sql, python, and jdbc. The view this action creates is a stream or a batch view, and later actions in the flowgroup must read it the same way (LHP-VAL-070); see Read mode.

description

string

No

auto

Docstring for the generated view function.

operational_metadata

bool or list[string]

No

—

Explicit column names to add; action-level false disables inherited selection. true alone adds no columns. See Operational metadata.

depends_on

list[string]

No

—

Extra upstream table or view references added to the dependency graph. See Dependency analysis reference.

cloudfiles

Auto Loader (cloudfiles): syntax and all options.

delta

Delta table (delta): syntax and all options.

sql

SQL query (sql): syntax and all options.

python

Python function (python): syntax and all options.

jdbc

JDBC database (jdbc): syntax and all options.

kafka

Kafka stream (kafka): syntax and all options.

custom_datasource

Custom DataSource (custom_datasource): syntax and all options.