fix(0.30.8): close issue #274 -- per-element sql_resplit closes ODBC statement-count crash - #275
Conversation
…statement-count crash Closes the remaining gap on the v0.28.0 (#245) `script[]` normalization. v0.28.0 covered `str -> array` for the lax-Storage-API / strict-runtime shape validator mismatch. v0.30.8 covers the case where `script` is already a list but an element packs multiple `;`-separated statements -- the Snowflake/BigQuery/etc. runtime crashes at `odbc_prepare` with `Actual statement count 2 did not match the desired statement count 1` (SQL state 0A000), the same write-side trap the v0.28.0 fix closed for string inputs. Live-reproduced against project 901 (`padak`) config `01km0sd189fdrcnjwk89cd1fkc`: push CREATE+ALTER as single list element -> Storage API 200 OK, version 58->59, normalizations empty (pre-fix); job 1307622107 crashed with the exact ODBC error. Fix runs each SQL-transformation list element back through `split_statements()` (same state machine #245 wired up) and replaces multi-statement entries inline. Each replacement emits a `sql_resplit` entry in the `normalizations` envelope with `path` pointing at the **original** element index on input. Non-SQL components (Python `;` is a valid intra-statement separator) skip this pass. 11 new unit tests + 1 new integration test cover: CREATE+ALTER, CREATE+SELECT, well-formed list no-op, mixed bad+good (original-index reporting), multiple bad elements, `;` inside `/* ... */` not a separator, `;` inside `'...'` not a separator, Python list passthrough, BigQuery list resplit, defensive None-element passthrough, and the end-to-end `ConfigService.update_config` write path. Suite now passes 2884 (+ 12 deselected as `not e2e`) tests, 0 regressions.
padak
left a comment
There was a problem hiding this comment.
Review of #275 — fix(0.30.8): per-element sql_resplit closes ODBC statement-count crash
Generated by
kbagent-pr-reviewersubagent. Verdict and findings below
are advisory; the human author retains every veto. CI-coverable issues
(lint, format, tests) are confirmed viamake check, not duplicated here.
Summary
This PR closes issue #274 by extending normalize_blocks_codes_script in src/keboola_agent_cli/sync/code_extraction.py with a per-element re-split branch for SQL transformations: when script is already a list but an element packs multiple ;-separated statements, each element is re-run through split_statements() and replaced inline. The fix is well-scoped (smallest possible delta on top of the existing #245 machinery), the live repro is documented, test coverage is thorough (11 unit tests + 1 service-layer integration test), and all plugin/agent surfaces listed in CONTRIBUTING.md's Plugin synchronization map have been updated. One non-blocking gap exists in the CLI-layer (CliRunner) test suite, and the existing human-mode warning message text is mildly misleading for the new action — both noted below.
Verdict: APPROVE
Verdict
- Verdict: APPROVE
- Blocking findings: 0
- Non-blocking findings: 2
- Nits: 1
Blocking findings
(none)
Non-blocking findings
[NB-1] tests/test_normalize_script.py:776 — no CliRunner test exercises the sql_resplit path end-to-end
TestConfigUpdateCliNormalization has two existing tests that cover sql_split (string input) via CliRunner — one for JSON-mode envelope, one for human-mode warning text. Neither was extended for the new sql_resplit path (list-with-multi-statement input). The service-layer integration test at line 733 (test_sql_list_element_resplit_before_push) confirms the post-resplit payload reaches the HTTP client, but it bypasses the Typer command layer entirely. A CliRunner test that feeds --configuration '{"parameters":{"blocks":[...script: ["SELECT 1; SELECT 2;"]]...}}' and asserts envelope["data"]["normalizations"][0]["action"] == "sql_resplit" would pin the full path from CLI argument through service through output formatter, consistent with the existing sql_split coverage. Per CONTRIBUTING.md, every CLI command change requires a CLI-layer test.
Fix: add a test_json_envelope_carries_sql_resplit method to TestConfigUpdateCliNormalization that supplies a list-with-multi-statement element and asserts action == "sql_resplit" in the JSON envelope.
[NB-2] src/keboola_agent_cli/commands/config.py:751 — human-mode warning text says "string -> list" when sql_resplit fires
The _emit_normalizations_warning function (not modified in this PR) emits the hardcoded message "Auto-normalized N script field(s) to array (string -> list)." at line 751. When sql_resplit fires the transformation is list -> list (element expansion), not string -> list. This PR intentionally reuses the existing action-agnostic warning, which is a reasonable trade-off, but the literal message text is now factually wrong for the new case. The PR's own changelog entry acknowledges this ("the line is action-agnostic so it fires for sql_split, wrap_array, AND sql_resplit") but the message was not updated to match. Users who see the warning in human mode and parse it literally will misunderstand what was normalized.
Fix: update the message to "Auto-normalized N script field(s). See --json for details." (dropping to array (string -> list) which is now inaccurate for one of the three action types). This is a one-line change to commands/config.py:751-752. Alternatively, emit per-action counts with accurate text.
Nits
[NIT-1]tests/test_normalize_script.py:489-503—test_well_formed_list_is_nooppassescfg(the original dict) tonormalize_blocks_codes_scriptwithoutcopy.deepcopy, then assertsout == before. Since the function mutatesconfigin place,cfgandoutare the same object and the equality assertion is trivially true even if mutation occurred. The assertion is sound only because no mutation happens for this input, but the test pattern is fragile: a future regression that mutates a well-formed list would not be caught. All other tests in the class correctly usecopy.deepcopy(cfg).
Verification log
gh auth status→ authenticated to github.com aspadak(active account) ✓gh pr view 275 --json state,...→state: OPEN,+489/-68, 10 files, conventionalfix(0.30.8):prefix ✓git rev-parse --abbrev-ref HEADin worktree →fix/issue-274-sql-resplit✓ (matches<branch>input)- 3-layer compliance:
grep typer/click/httpxin new+lines across layers → (none) ✓ - New
@*.command()decorators in diff → (none); no new CLI command, sopermissions.py,context.py,hints/definitions/,CLAUDE.md All CLI Commandsupdates are not required ✓ - Plugin synchronization map (all "NO" rows):
keboola-expert.md§1 Rule 6 VERSION GATE updated with0.30.8+list-element re-split entry ✓; §2 Tool Selection Matrix row for "Update SQL transformation body" updated with0.30.8+note ✓; §3 Inline Gotchas new entry added ✓;gotchas.mdextended with(since v0.30.8; #274)tag ✓;commands-reference.mdconfig updatenote updated ✓ grep 'commands/config.py' /tmp/kbagent-pr-275.files→ NOT in PR diff (warning text not updated — leads to NB-2) ✓make check(worktree) → ruff clean, 211 files formatted, SKILL.md up-to-date, plugin.json in sync, changelog complete, error-code check OK, 2884 passed, 7 skipped exit 0 ✓CliRunnertest forsql_resplitpath → absent inTestConfigUpdateCliNormalization(leads to NB-1)test_well_formed_list_is_noopmutation-safety → usescfgwithoutdeepcopy, trivially true equality (leads to NIT-1)- Behavior verification: live repro documented in PR description (project 901, job 1307622107, exact ODBC error); cannot re-run without E2E credentials. Unit test
test_create_plus_alter_session_resplit_in_placereproduces the canonical input shape from the bug report and confirms correct 2-element split output ✓ - Security: no token exposure, no httpx outside client files, no bare
except:, no magic numbers, noprint()in production code ✓ - Backward compat:
before_lengthis a new field insql_resplitrecords only; existing consumers ofnormalizations(e.g.commands/config.py:706,test_e2e.py:6772) use.get()/or []patterns that tolerate extra fields ✓
Open questions for the author
(none)
…276) Both PR #266 (feat: project edit --new-alias + --dry-run) and PR #275 (fix: per-element sql_resplit closes ODBC statement-count crash on #274) shipped as patch bumps (0.30.7 / 0.30.8) but never carried an external release. Consolidate them into a single minor release 0.31.0 -- a feature warrants the minor bump and the bug-fix rides along. No code changes; this is purely the version-label rename across the silent-drift sync surfaces: - pyproject.toml + plugin.json + marketplace.json + uv.lock -> 0.31.0 (via make version-sync) - changelog.py: 0.30.8 + 0.30.7 keys merged into one 0.31.0 block, features first (highlight ordering), then the SQL fix, then tests - gotchas.md: 5 references to (since v0.30.7) / (since v0.30.8) -> (since v0.31.0) - commands-reference.md: project edit gotcha pointer + config update auto-normalize version-gate annotation -> 0.31.0 - keboola-expert.md: 6 references in Rule 6 VERSION GATE, Tool Selection Matrix rows, and inline gotchas -> 0.31.0 make check (lint + format + skill freshness + version sync + changelog completeness + 2895 tests) clean. No regressions.
Summary
Closes #274.
Closes the remaining gap on the v0.28.0 (#245)
script[]normalization. v0.28.0 coveredstr -> arrayfor the lax-Storage-API / strict-runtime shape validator mismatch. v0.30.8 covers the case wherescriptis already a list but an element packs multiple;-separated statements — the Snowflake / BigQuery / etc. runtime crashes atodbc_preparewithActual statement count 2 did not match the desired statement count 1(SQL state 0A000), the same write-side trap the v0.28.0 fix closed for string inputs.Repro before fix (verified)
Live-reproduced against project 901 (
padak) config01km0sd189fdrcnjwk89cd1fkc:kbagent config update --configuration-file repro.jsonwithscript: ['CREATE TABLE ...; alter session unset week_start;'](list of 1, packs 2 stmts)normalizations: [](no warning)kbagent job run --wait: job 1307622107 crashed with:Fix
Extend
normalize_blocks_codes_scriptinsrc/keboola_agent_cli/sync/code_extraction.py. After the existingisinstance(script, str)branch, add anelif is_sql and isinstance(script, list)branch that runs each element through the samesplit_statements()state machine #245 already wired up, and replaces multi-statement elements inline. Path: smallest possible delta — sameis_sql_transformation_component()gate, same splitter helper.Each replacement emits a
sql_resplitentry in thenormalizationsenvelope:{ "path": "parameters.blocks[0].codes[0].script[2]", "action": "sql_resplit", "before_type": "str", "after_type": "list", "before_length": 1, "after_length": 2 }pathpoints at the original element index on input (not the post-split position) so users can map the warning back to their source payload even when later elements shift due to upstream splits in the same list.Non-SQL components (Python
;is a valid intra-statement separator:print('a'); print('b')) skip this pass — sameis_sqlgate as #245.Tests
tests/test_normalize_script.py::TestSqlResplitListElementscovering: CREATE+ALTER (canonical config update: re-split script[] elements that contain multiple statements (gap beyond #245) #274), CREATE+SELECT (second failure mode from config update: re-split script[] elements that contain multiple statements (gap beyond #245) #274 follow-up), well-formed list no-op, mixed bad+good (only bad element recorded, original-index reporting), multiple bad elements in the same code,;inside/* ... */not a separator,;inside'...'string literal not a separator, Python list passthrough, BigQuery list resplit, defensiveNone-element passthrough.TestConfigServiceUpdateNormalizes::test_sql_list_element_resplit_before_pushintegration test pinning thatConfigService.update_configsends the post-resplit payload to the HTTP client.Plugin / agent surfaces updated (per CLAUDE.md convention #17)
plugins/kbagent/agents/keboola-expert.md— Rule 6 VERSION GATE entry, matrix row for SQL transformation body updates, inline gotcha block.plugins/kbagent/skills/kbagent/references/gotchas.md— extended the existing update_sql_transformation: validate / preserve script field as array (string causes runtime parse error) #245 gotcha with the v0.30.8 list-element re-split section.plugins/kbagent/skills/kbagent/references/commands-reference.md—config updateauto-normalize note.Test plan
make check— lint, format-check, skill-check, version-check, changelog-check, all 2884 unit/integration testskbagent-e2eafter merge (not run in CI; can be added totest_e2e.py::TestE2EConfigUpdateNormalizationfollow-up if desired)