Monitoring job configuration
============================

**File:** the path selected by ``monitoring.job_config_path`` in ``lhp.yaml``.
For orchestration jobs generated by ``lhp dag``, use :doc:`jobs`.


``monitoring.job_config_path`` points to a flat single-document YAML mapping.
LHP deep-merges it over its defaults (``max_concurrent_runs: 1``,
``queue.enabled: true``, ``performance_target: STANDARD``) and then
token-substitutes it per environment. The job name is fixed at
``<pipeline_name>_job`` and the ``pipeline_task`` is generated automatically.
All fields are optional.

.. list-table::
   :header-rows: 1
   :widths: 30 70

   * - Field
     - Description
   * - ``max_concurrent_runs``
     - Maximum concurrent job runs. Default ``1``.
   * - ``performance_target``
     - ``STANDARD`` (default) or ``PERFORMANCE_OPTIMIZED``.
   * - ``queue``
     - ``queue.enabled`` toggles job-run queueing.
   * - ``notebook_cluster.new_cluster``
     - New-cluster spec for the notebook task. Mutually exclusive with ``existing_cluster_id``; serverless when neither is set.
   * - ``notebook_cluster.existing_cluster_id``
     - Attach the notebook task to an existing cluster.
   * - ``timeout_seconds``
     - Job-level timeout in seconds.
   * - ``tags``
     - Free-form ``{key: value}`` mapping.
   * - ``email_notifications``
     - ``on_start`` / ``on_success`` / ``on_failure`` recipient lists.
   * - ``webhook_notifications``
     - ``on_start`` / ``on_success`` / ``on_failure`` webhook-id lists.
   * - ``permissions``
     - User/group permission entries (``level`` plus ``user_name`` or ``group_name``).
   * - ``schedule``
     - Quartz cron schedule: ``quartz_cron_expression``, ``timezone_id``, ``pause_status``.
   * - ``tasks``
     - Extra tasks, added after the generated ``union_event_logs`` notebook task and, when the monitoring pipeline is generated, the ``<pipeline_name>_pipeline`` task. Must be a list; each task needs a non-empty string ``task_key`` that no other task uses. List a generated task key under ``depends_on`` to run after it.

Any key not listed above is passed through verbatim into the job resource, so
new Databricks Jobs fields work without an LHP change. The exception is
``name``: LHP sets it and rejects a value here. A ``name`` or an invalid
``tasks`` value fails ``lhp generate`` with ``LHP-CFG-072``. Use the flat shape shown below. LHP does not interpret a ``project_defaults``
wrapper or ``job_name`` selector here; unsupported resource keys can pass through
and fail Databricks validation. LHP generates the job name and its own tasks.

.. code-block:: yaml

   # config/monitoring_job_config.yaml
   max_concurrent_runs: 1
   performance_target: STANDARD
   queue:
     enabled: true
   tags:
     environment: "${bundle.target}"


Use the `Databricks job resource reference
<https://docs.databricks.com/aws/en/dev-tools/bundles/resources#job>`_ for
external fields. See :doc:`/operate/adjust-monitoring` for change instructions.