get_configs: provide structured JSON output (text format breaks programmatic parsing)

[beckavac]

Problem

get_configs returns configurations as a single flattened text blob (a YAML/pretty-printed format inside the content[0] string), rather than the structured JSON returned by the Storage API.

For interactive use this is fine, but programmatic consumers have to:

1. Parse the text with regex
2. Deal with escape-sequence handling for multi-line values (notably SQL query fields with \n literals)
3. Reconstruct nested fields like parameters.primaryKey from indented sub-keys

In our case this directly broke a workflow: detecting a top-level WHERE clause in MySQL extractor queries was unreliable because newlines were rendered as \n literals in the text output, not as actual \n characters. We had to fall back to direct curl calls against /v2/storage/components/{component}/configs/{config}/rows to get usable JSON.

Repro

kbagent --json tool call get_configs --project <p> --input '{"configs":[{"component_id":"keboola.ex-db-mysql","configuration_id":"<id>"}]}'

→ The data.results[0].content[0] is a string. Multi-line parameters.query values come back with escape sequences, not real newlines.

Proposal

Either:

• (preferred) Add a format: "json" flag (default "text" for back-compat) that returns the full Storage API response as structured JSON.
• Or ship a dedicated get_config_raw tool that returns the row payload as-is.

Impact

Without this, any agent doing structured edits on configurations (esp. SQL queries, nested params, primary keys) has to bypass MCP for reads.

[padak]

The proposed format=json flag would belong upstream in keboola-mcp-server -- the TOON serialization is set there explicitly (serializer=toon_serializer_compact in tools/components/tools.py), and kbagent is a passthrough that can't safely reverse-engineer it.

But for the specific use case you describe (programmatic inspection of a config), kbagent already has a native command that bypasses MCP entirely:

kbagent --json config detail --project P --component-id keboola.ex-db-mysql --config-id <id>

This calls GET /v2/storage/components/{component_id}/configs/{config_id} directly and returns the full JSON exactly as the Storage API serves it. No TOON, no double-escaping; multi-line parameters.query arrives with real newlines after standard json.loads. There's also a bulk variant (omit --config-id, repeat --project for multiple projects) and --with-state for inline runtime state.

Concrete comparison on the same Snowflake transformation config:

	config detail	tool call get_configs
content[0] type	structured JSON	string (TOON, ~15 KB)
Real newlines in SQL script	71	0 (escaped as \n literals -- 350 occurrences)
Detect WHERE clause	'WHERE' in script	needs TOON parser + double-unescape

Your curl /v2/storage/.../rows workaround is essentially what config detail does internally -- one CLI call, token + URL handled.

Design pattern in kbagent: every MCP read-tool has a native CLI equivalent (get_configs <-> config detail, get_buckets <-> storage buckets, get_tables <-> storage tables, get_jobs <-> job list, ...). MCP is optimized for token-efficient AI agent consumption; the native commands are for shell scripts and structured/programmatic work.

If config detail covers your scenario, we can close this. I'll also add a more explicit hint in kbagent context and the expert agent prompt so future readers don't fall into the same trap.

[beckavac]

Closing the loop:

$ kbagent --json config detail --project <p> --component-id keboola.snowflake-transformation --config-id 746769004

Verified directly:

• Top-level keys: configuration, rows, state, ... — full Storage API shape.
• script field type: list (proper array, no string flattening).
• Real newlines: 1826 | escaped \n literals: 0.
• WHERE detection works with plain 'WHERE' in script after '\n'.join(script).

So this is exactly the structured JSON path I was looking for — same as the raw curl /v2/storage/.../rows I ended up using, just wrapped properly. Same pattern as #243 (CLI has the right tool, I went straight to MCP and bypassed).

Happy to close this issue. The hint in kbagent context + expert agent prompt sounds like the right fix — the MCP <-> CLI mapping table you sketched (get_configs ↔ config detail, get_buckets ↔ storage buckets, get_tables ↔ storage tables, get_jobs ↔ job list) is gold, would be useful to surface explicitly. I'll keep that mapping in my own notes for future reference.

Two issues, same lesson learned: MCP tool ≠ CLI surface; CLI is broader, default to it for programmatic / structured work. Thanks for the patient handholding.

[Dave-London]

+1 on this. The dual-output pattern — emit structured JSON in structuredContent, keep human-readable text in content for clients that need it — is what we landed on across pare-tools' 28 MCP servers. Same MCP envelope, no flag needed; agents read structuredContent, shell-style consumers read content.

The Storage-API → MCP shape-mapping friction you're describing is exactly what falls out when everything is flattened into content[0]. Anything with nested structure (or escape-sensitive fields like SQL with literal \n) is unrecoverable downstream once it's gone through pretty-printing.

[padak]

MCP server supports switching output from TOON to JSON -- you can pass format: json in the tool input or configure it at server startup. So the original TOON-vs-JSON concern is addressable on the MCP side.

That said, the deeper takeaway from this discussion is that CLI is the better interface for AI agents working locally: 33% more token-efficient, billions of training examples vs. near-zero for custom MCP protocols, composes natively with shell pipelines, and doesn't require a running subprocess.

As a result, we've filled the remaining CLI gaps in PR #255 (feat(0.29.0)):

• kbagent search QUERY -- cross-project search (textual + config-based), covers the search_tables / search_configurations MCP tools
• kbagent project info -- full project details, features, limits
• kbagent config row-create / row-update -- config row lifecycle
• kbagent config oauth-url -- OAuth authorization URL

After #255 merges, kbagent has native CLI coverage for all MCP tools that matter. Recommend CLI as primary path; kbagent tool call remains available for edge cases.