Skip to content

Semantic layer: build_model push-loop transactional rollback #295

Description

@ottomansky

Background

SemanticLayerService.build_model (services/semantic_layer_service.py) generates a heuristic model from a list of storage tables and pushes it to the metastore in dependency order via _push_built_model (_semantic_layer_internals.py):

POST semantic-model -> get model UUID
for each plural in PUSH_ORDER:
    for each item in generated[plural]:
        POST semantic-X (with modelUUID = model UUID)

There is no rollback if a child POST fails after the model has been created. The model + already-POSTed children remain in the metastore.

Problem

If build_model fails halfway through (e.g. constraint regex rejected by the server, network drop mid-push), the operator is left with a partially-populated model that:

  • Cannot be cleanly retried (duplicate-name POSTs return ALREADY_EXISTS).
  • Cannot be kbagent semantic-layer model delete-d in one shot (refuses while children exist).
  • Has to be torn down by hand: kbagent semantic-layer remove ... per child, then model delete.

Acceptance criteria

  • Either: implement a rollback path that on failure tracks every successfully-POSTed item (model + children) and DELETEs them in reverse order, restoring the project to its pre-build state.
  • Or: document the current non-transactional behavior in gotchas.md with the manual teardown recipe, and surface a --keep-on-failure flag (mirrors kbagent data-app create --keep-on-failure) so CI scripts can opt into preserve-state-on-failure for debugging.
  • Update the build_model docstring + keboola-expert.md to reflect the chosen semantics.

Filed from PR #293 review (pre-merge follow-up tracking)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions