Skip to content

[Bug][python] YAML workflow loader cannot resolve task_type "K8S" (Kubernetes.yaml example fails to parse) #176

Description

@initiallyqq

The shipped example examples/yaml_define/Kubernetes.yaml declares task_type: K8S but fails to load:

pydolphinscheduler.exceptions.PyDSTaskNoFoundException: cant not find task K8S

The YAML loader resolves a task type to its task class by class name only. In
src/pydolphinscheduler/core/yaml_workflow.py, get_task_cls() builds
{type_.capitalize(): type_ for type_ in tasks.__all__} and looks up
task_type.capitalize(). So it expects "Kubernetes" (the class name), but the
documented/constant task type is K8S:

  • "K8S".capitalize()"K8s", which is not in the class-name map.
  • The internal constant is TaskType.KUBERNETES = "K8S" (constants.py), i.e. the
    rest of the SDK and the backend use "K8S", but the YAML loader never consults
    TaskType — only class names. The loader and the rest of the SDK disagree on the
    K8S identifier.

There is a second bug in the same example: it uses camelCase field names
minCpuCores / minMemorySpace, but the Kubernetes task constructor expects
snake_case min_cpu_cores / min_memory_space. After getting past the loader you
then hit:

TypeError: __init__() missing 2 required positional arguments: 'min_cpu_cores' and 'min_memory_space'

Why CI never caught it

Kubernetes.yaml is the only one of the 22 examples omitted from the parametrize list in
tests/core/test_yaml_workflow.py::test_get_create_workflow (all others are covered).
So the broken example ships untested and CI stays green.

To Reproduce

from pydolphinscheduler.core.yaml_workflow import create_workflow
create_workflow("examples/yaml_define/Kubernetes.yaml")
# -> PyDSTaskNoFoundException: cant not find task K8S

Or just the resolver:

from pydolphinscheduler.core.yaml_workflow import get_task_cls
get_task_cls("K8S")   # -> PyDSTaskNoFoundException: cant not find task K8S
get_task_cls("Kubernetes")  # OK (only the class name works today)

Which version are you using?

Reproduced on:

  • PyPI release apache-dolphinscheduler 4.1.0 (latest release).
  • main branch HEADget_task_cls, Kubernetes.yaml, and the parametrize
    list are byte-for-byte the same as 4.1.0, confirming this is a current defect
    rather than a released-only regression.

Expected behavior

task_type: K8S (the documented constant value) should resolve to the Kubernetes
task class, and the shipped Kubernetes.yaml example should parse and be covered by CI.

Additional context

I'll open a PR that (1) makes the loader fall back to TaskType constant values,
(2) fixes the camelCase fields in the example, and (3) adds the example +
K8S/k8s/Kubernetes cases to the tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions