On this page

• Python function (python)
  • Minimal example
  • Source fields
  • Action-level fields

Python function (python)

YAML location: an entry in a flowgroup’s actions list. Source selector: source.type: python.

Calls a Python function that returns a DataFrame. Fields live under source:.

Minimal example

Add this entry to the actions list in your flowgroup.

- name: load_external
  type: load
  source:
    type: python
    module_path: "loaders/external.py"
    function_name: get_df
    parameters:
      region: "${region}"
  target: v_external

Source fields

Field	Type	Required	Default	Description
type	string	Yes	—	Must be python.
module_path	string	Yes	—	Path to a .py file (relative to project root); must end in .py.
function_name	string	No	get_df	Entry function, called as fn(spark, parameters) -> DataFrame.
parameters	mapping	No	{}	Passed as the function’s second argument.

Action-level fields

These fields apply to every load sub-type. name, type, source and target are required.

Field	Type	Required	Default	Description
name	string	Yes	—	Unique action name within the flowgroup.
type	string	Yes	—	Always load.
source	mapping	Yes	—	Source configuration. source.type selects the sub-type. Use a mapping for every load type, including SQL.
target	string	Yes	—	Name of the temporary view this action creates.
readMode	string	No	sub-type	batch or stream; selects spark.read vs spark.readStream. Default and constraints vary by sub-type (below). Ignored by sql, python, and jdbc. The view this action creates is a stream or a batch view, and later actions in the flowgroup must read it the same way (LHP-VAL-070); see Read mode.
description	string	No	auto	Docstring for the generated view function.
operational_metadata	bool or list[string]	No	—	Explicit column names to add; action-level false disables inherited selection. true alone adds no columns. See Operational metadata.
depends_on	list[string]	No	—	Extra upstream table or view references added to the dependency graph. See Dependency analysis reference.

See the worked guide or choose another load source.