Skip to content

[v3-3-test] Clarify logging_config_class contract and document REMOTE_TASK_LOG (#67104) - #70592

Merged
jason810496 merged 1 commit into
v3-3-testfrom
backport-d8b8620-v3-3-test
Jul 29, 2026
Merged

[v3-3-test] Clarify logging_config_class contract and document REMOTE_TASK_LOG (#67104)#70592
jason810496 merged 1 commit into
v3-3-testfrom
backport-d8b8620-v3-3-test

Conversation

@github-actions

Copy link
Copy Markdown
Contributor
  • Clarify logging_config_class contract and document REMOTE_TASK_LOG

[logging] logging_config_class is documented as a "Logging class" but
actually resolves to a logging.config.dictConfig dict, and the
REMOTE_TASK_LOG / DEFAULT_REMOTE_CONN_ID side channel that powers
remote log read-back was undocumented. Custom configs silently lost UI log
read-back as a result.

  • Document the real contract for logging_config_class (dict, not class)
    and the REMOTE_TASK_LOG / DEFAULT_REMOTE_CONN_ID module-level
    attributes in the config option help, advanced-logging-configuration.rst,
    and the discover_remote_log_handler docstring.
  • Add a startup WARNING when remote_logging is on but the user's
    logging module is missing REMOTE_TASK_LOG, emitted from
    configure_logging after dictConfig runs so it sees the final state.
  • Fix CI error

  • CI: Fix pyproject.toml

  • Fix pyproject.toml

  • Drop LOGGING_CONFIG dict from remote logging docs section

Document only REMOTE_TASK_LOG / DEFAULT_REMOTE_CONN_ID in the new remote logging section; do not show users a LOGGING_CONFIG dict to build.

  • Clarify user-defined logging config detection and simplify warning tests

An empty logging_config_class falls back to the default, so treating
it as user-defined under the old user_defined name was ambiguous per
review feedback. Rename it to state what it actually checks and document
why the empty-path case is excluded. Also collapse the near-duplicate
TestWarnIfMissingRemoteTaskLog tests into one parametrized test using
the project's conf_vars helper for the config override, per review
suggestions on #67104.

  • Default remote_task_log to None and clarify empty logging_config_class handling

_ActiveLoggingConfig.remote_task_log had no default, so
_warn_if_missing_remote_task_log() raised AttributeError if it ran
before _load_logging_config() ever populated the class. A short
comment also clarifies that the or DEFAULT_LOGGING_CONFIG_PATH
fallback intentionally covers an explicitly empty
logging_config_class = "".

  • Resolve missing-REMOTE_TASK_LOG warning via get_remote_task_log()

The check read _ActiveLoggingConfig.remote_task_log directly, which is
only populated once something has triggered resolution (previously the
deprecated Elasticsearch/OpenSearch handler self-registration during
dictConfig). A user with a custom logging_config_class whose remote
logging actually resolves through ProvidersManager dispatch -- with no
ES/OS handler in the mix -- got a false-positive warning because the
cache was still cold at check time. Going through get_remote_task_log()
triggers the real resolution lazily, so the check reflects whether
remote logging is actually available.
(cherry picked from commit d8b8620)

Co-authored-by: Jason(Zhe-You) Liu 68415893+jason810496@users.noreply.github.com

…REMOTE_TASK_LOG`` (#67104)

* Clarify ``logging_config_class`` contract and document REMOTE_TASK_LOG

``[logging] logging_config_class`` is documented as a "Logging class" but
actually resolves to a ``logging.config.dictConfig`` dict, and the
``REMOTE_TASK_LOG`` / ``DEFAULT_REMOTE_CONN_ID`` side channel that powers
remote log read-back was undocumented. Custom configs silently lost UI log
read-back as a result.

- Document the real contract for ``logging_config_class`` (dict, not class)
  and the ``REMOTE_TASK_LOG`` / ``DEFAULT_REMOTE_CONN_ID`` module-level
  attributes in the config option help, ``advanced-logging-configuration.rst``,
  and the ``discover_remote_log_handler`` docstring.
- Add a startup ``WARNING`` when ``remote_logging`` is on but the user's
  logging module is missing ``REMOTE_TASK_LOG``, emitted from
  ``configure_logging`` after ``dictConfig`` runs so it sees the final state.

* Fix CI error

* CI: Fix pyproject.toml

* Fix pyproject.toml

* Drop LOGGING_CONFIG dict from remote logging docs section

Document only REMOTE_TASK_LOG / DEFAULT_REMOTE_CONN_ID in the new remote logging section; do not show users a LOGGING_CONFIG dict to build.

* Clarify user-defined logging config detection and simplify warning tests

An empty ``logging_config_class`` falls back to the default, so treating
it as user-defined under the old ``user_defined`` name was ambiguous per
review feedback. Rename it to state what it actually checks and document
why the empty-path case is excluded. Also collapse the near-duplicate
``TestWarnIfMissingRemoteTaskLog`` tests into one parametrized test using
the project's ``conf_vars`` helper for the config override, per review
suggestions on #67104.

* Default remote_task_log to None and clarify empty logging_config_class handling

_ActiveLoggingConfig.remote_task_log had no default, so
_warn_if_missing_remote_task_log() raised AttributeError if it ran
before _load_logging_config() ever populated the class. A short
comment also clarifies that the `or DEFAULT_LOGGING_CONFIG_PATH`
fallback intentionally covers an explicitly empty
`logging_config_class = ""`.

* Resolve missing-REMOTE_TASK_LOG warning via get_remote_task_log()

The check read _ActiveLoggingConfig.remote_task_log directly, which is
only populated once something has triggered resolution (previously the
deprecated Elasticsearch/OpenSearch handler self-registration during
dictConfig). A user with a custom logging_config_class whose remote
logging actually resolves through ProvidersManager dispatch -- with no
ES/OS handler in the mix -- got a false-positive warning because the
cache was still cold at check time. Going through get_remote_task_log()
triggers the real resolution lazily, so the check reflects whether
remote logging is actually available.
(cherry picked from commit d8b8620)

Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
@jason810496
jason810496 merged commit 4bc20e6 into v3-3-test Jul 29, 2026
245 of 254 checks passed
@jason810496
jason810496 deleted the backport-d8b8620-v3-3-test branch July 29, 2026 03:40
@vatsrahul1001 vatsrahul1001 added the type:doc-only Changelog: Doc Only label Aug 3, 2026
@vatsrahul1001 vatsrahul1001 added this to the Airflow 3.3.1 milestone Aug 3, 2026
vatsrahul1001 pushed a commit that referenced this pull request Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants