feat(0.28.0): data-app secrets + validate-repo + --auth public fix (logs deferred)

[ottomansky]

Why these three changes travel together

v0.28.0 rounds out the v0.27.0 data-app surface with three changes that
travelled together because the third was discovered while live-validating
the first two. Running the new secrets-set against a real --auth public
data app surfaced that v0.27.0's --auth public writes no authorization
key into the Storage config — the Keboola app-proxy returns HTTP 503 and
the UI's auth-type selector shows blank. Root-cause took 30 minutes
against the public AppProxyDefinition.php validator; fixing it in this
PR keeps v0.27.0 → v0.28.0 from shipping a known silent break. All three
changes touch the same parameters.dataApp.* write paths and share the
read-modify-write / fail-closed encryption / metadata-only-read patterns;
they are not three independent features.

Logs / auto-log-dump on deploy failure was the originally-planned fourth
surface. It is deferred because the Data Science API does not document
a programmatic logs endpoint (the Terminal Log
tab help canon
documents only an in-UI "Download Logs" button). Tracked as
#240 — needs
platform-side API exposure first.

What changed

Surface 1 — kbagent data-app secrets-set / -list / -get / -remove

• Manages #-prefixed app-runtime secrets in parameters.dataApp.secrets
  on a keboola.data-apps Storage config.
• Per-project KMS encryption, fail-closed. Same semantics as
  v0.27.0's --git-pat-encrypted: if the Encryption API does not return
  a KBC::Project* ciphertext, the command aborts with
  ENCRYPTION_FAILED before Storage is touched.
  --allow-plaintext-on-encrypt-failure is bootstrap/debug only.
• Read-modify-write at the SERVICE layer, NOT Storage merge=True
  (which is shallow at the top level only and would clobber sibling keys
  nested inside parameters.dataApp.secrets). Every untouched key in
  the config body is preserved bit-identical — asserted by
  tests/test_data_app_secrets_service.py::TestSetSecretsHappyPath::test_sibling_keys_preserved_bit_identical.
• secrets-get is metadata-only. NEVER echoes the decrypted plaintext
  to stdout / stderr / logs / change descriptions. The Encryption API has
  no decrypt endpoint; the CLI does not attempt to decrypt under any
  branch. NOT_FOUND on an absent key never enumerates sibling keys
  (avoids leaking neighbour presence).
• secrets-remove is idempotent. Removing a key that isn't set is
  exit 0 with removed: 0 and no Storage write.
• Reserved-name shadowing detection. Setting a secret whose derived
  env-var name collides with RESERVED_RUNTIME_ENV_VARS (canon-documented
  floor: KBC_TOKEN, KBC_URL per
  https://help.keboola.com/data-apps/storage-access/) emits a stderr WARN
  and surfaces shadowed_by_runtime in the JSON envelope. The write
  still happens — the platform shadows the value at runtime, not at
  write time.
• Runtime env-var translation rule (per https://help.keboola.com/data-apps/python-js/):
  strip #, replace - with _, uppercase. #my-api-key → MY_API_KEY.

Surface 2 — kbagent data-app validate-repo

• Pre-flight Golden-Rule check via the GitHub Trees+Contents API.
  ≤5 calls regardless of repo size (1 trees-recursive + up to 4
  contents fetches), so the 60/hour unauth GitHub rate limit is no
  longer the common-case failure mode.
• BLOCKING / WARN / OK per check, each carrying a citation back to the
  help-doc anchor that defines the rule
  (https://help.keboola.com/data-apps/python-js/).
• --type python-js only in 0.28.0; streamlit / pure-Python / R /
  Node-only follow-up.
• GitHub-only; GitLab / Bitbucket follow-up.

Surface 3 — --auth public fix

• v0.27.0 wrote NO authorization key when --auth public, leaving the
  Keboola app-proxy unable to route (HTTP 503 / "Service Unavailable")
  and the UI's "Authentication Type" selector blank.
• v0.28.0 writes the canonical noneProxyAuthorization shape:

  { "app_proxy": {
      "auth_providers": [],
      "auth_rules": [
        { "type": "pathPrefix", "value": "/", "auth_required": false }
      ]
  } }

• Authoritative source (public, sufficient on its own):
  keboola/job-queue-job-configuration AppProxyDefinition.php.
  The validator's invariants (auth_providers may be empty;
  auth_rules requires ≥1 element; auth_required: false requires
  auth to NOT be set) make the shape this PR writes the unique
  solution.
• Corroborating source (private — Keboola org members only):
  keboola/ui apps/kbc-ui/src/scripts/modules/data-apps/constants.ts
  exports this exact shape as noneProxyAuthorization for the "None"
  UI option. Mentioned for internal verifiability — the public validator
  above is sufficient for external readers; no clickable URL is provided
  for the UI repo because it is not externally accessible.
• Existing --auth password behaviour unchanged.

Other auth providers (OIDC / GitHub OAuth / GitLab OAuth / JumpCloud /

Auth0) are NOT yet exposed by the CLI's --auth flag — tracked as
#241.

Plumbing

• New ErrorCode entries: DATA_APP_INVALID_SECRET, DATA_APP_INVALID_REPO,
  DATA_APP_REPO_VALIDATION_BLOCKING.
• Permission registry: data-app.secrets-set (write), data-app.secrets-list
  / data-app.secrets-get (read), data-app.secrets-remove (destructive),
  data-app.validate-repo (read).
• 5 new --hint client/service definitions; all 9 hint snippets ast.parse
  cleanly (parameterized test in tests/test_data_app_secrets_cli.py).
• All 8 KeboolaApiError catch sites in commands/data_app.py now propagate
  details=exc.details to formatter.error (was: dropped silently). The
  service populates details={"project_alias": ..., "failed_keys": [...]}
  on ENCRYPTION_FAILED so the operator can recover the failed-key list
  from the JSON envelope.
• _auth_block_for(auth) dispatch helper raises ValueError on unknown
  values rather than returning None — structurally prevents another
  v0.27.0-style silent-503 regression if a future auth mode is added to
  _validate_create_inputs but not to the dispatch.
• _redact_secrets_block extends the existing _redact_storage_config
  helper so data-app detail JSON output never echoes encrypted
  ciphertext from the secrets sub-dict.
• New service module services/repo_validate_service.py carries the
  RepoValidateService + GitHubContentsClient + pure
  validate_keboola_repo function (no I/O); the rest of kbagent stays
  Keboola-API-only.

Sync-map walk

Every silent-drift surface from CONTRIBUTING.md "Plugin synchronization
map" updated:

• pyproject.toml 0.27.0 → 0.28.0; plugin.json + marketplace.json synced via make version-sync.
• changelog.py — one entry-list for 0.28.0 covering all three surfaces.
• commands/context.py AGENT_CONTEXT — 5 new command sigs.
• CLAUDE.md ## All CLI Commands — 5 new lines.
• keboola-expert.md matrix — 5 new rows; §1 Rule 6 VERSION GATE notes
  secrets / validate-repo need 0.28.0+; §3 Inline Gotchas — new
  --auth public + secrets-* metadata-only + validate-repo GitHub-only entries.
• commands-reference.md — 5 new bullets.
• gotchas.md — three new (since v0.28.0) entries (auth fix; secrets;
  validate-repo).
• data-app-workflow.md — "Managing app-runtime secrets" + "Pre-flight
  repo validation" recipe sections; "When to use what" table extended;
  the existing "deliberately does NOT cover the build/runtime log" note
  links to issue data-app logs + auto-log-dump on deploy failure #240.
• SKILL.md regenerated via make skill-gen.

Verifying the auth fix end-to-end

# 1. Reproduce the v0.27.0 bug:
pip install keboola-agent-cli==0.27.0
kbagent data-app create --project P --name pre-fix --slug pre-fix \
  --git-repo https://github.com/keboola/example-python-js-hello-world \
  --git-public --auth public --type python-js --wait
# Note the URL in the output, then:
curl -s -o /dev/null -w '%{http_code}\n' <URL>     # -> 503  (broken)
kbagent --json data-app detail --project P --app-id <ID> | \
  jq '.data.raw.storage_config.configuration.authorization'
# -> null  (the v0.27.0 silent-break: no authorization key written)

# 2. Verify the v0.28.0 fix from this PR:
git fetch origin feat/data-app-secrets-and-validate-repo
git checkout feat/data-app-secrets-and-validate-repo
uv pip install -e .
kbagent data-app create --project P --name post-fix --slug post-fix \
  --git-repo https://github.com/keboola/example-python-js-hello-world \
  --git-public --auth public --type python-js --wait
curl -s -o /dev/null -w '%{http_code}\n' <URL>     # -> 200  (fixed)
kbagent --json data-app detail --project P --app-id <ID> | \
  jq '.data.raw.storage_config.configuration.authorization'
# -> exact bit-identical noneProxyAuthorization shape:
# { "app_proxy": { "auth_providers": [],
#                  "auth_rules": [{ "type": "pathPrefix", "value": "/", "auth_required": false }] } }

# 3. UI eyeball: the Authentication tab now shows "None" pre-selected.

Live validation (project 1143, on stack europe-west3.gcp)

All scenarios PASS. Verbatim CLI output captured in
~/.claude/plans/validation-output.md (442 lines); summary table:

#	Scenario	Mode	Result
1	data-app create --wait	LIVE	URL minted; state == running on first poll
2	secrets-set normal + reserved-name (#KBC_TOKEN=foo)	LIVE (human + JSON)	exit 0, stderr WARN in human mode, shadowed_by_runtime: ["KBC_TOKEN"] in JSON; both secrets written
3	secrets-list ± --show-fingerprint	LIVE	default mode: NO ciphertext at all; --show-fingerprint: 8-char fingerprint + KBC::ProjectSecureGKMS prefix only
4	secrets-get plaintext-absence canary	LIVE	grep -F -c 'hello' across stdout AND stderr in BOTH JSON and human modes: 0 hits
5	data-app deploy --wait (§9 redeploy contract)	LIVE	Storage v5, deployed v3 → both v5
6	secrets-remove + redeploy	LIVE	#TEST_KEY gone, version 5 → 6, deployed=6
7	idempotent secrets-remove (--key '#NONEXISTENT')	LIVE	exit 0, removed: [], no version bump
8	validate-repo against keboola/example-python-js-hello-world	LIVE	6 OK + 1 WARN (offline requires-python skip), 0 BLOCKING, exit 0
9	validate-repo against torvalds/linux	LIVE	3 BLOCKING + 2 WARN incl. meta.tree-truncated (>100k entries), is_failure: true
10	§5.5 replay 1, 2, 5	LIVE	malformed-key / KBC::-detection / NOT_FOUND-no-enumeration all exit-2-or-1 with the right error_code
11	--auth public post-fix HTTP 200	LIVE	written authorization block bit-identical to canon; curl → HTTP 200 first attempt

Test plan

• make check — green: lint + format + skill freshness + version
  consistency + changelog + error-codes + 2505 tests pass
  (5 e2e skipped, 64 deselected as not e2e)
• §5.5 failure-mode replay live — see table above + validation-output.md
• --auth public re-validation — HTTP 200, bit-identical to canon
• Sibling-preservation regression test passes
  (TestSetSecretsHappyPath::test_sibling_keys_preserved_bit_identical)
• Plaintext-absence canary passes (CLI + service tests)
• All 9 hint snippets ast.parse cleanly
• Reviewer-side: UI eyeball — Authentication tab shows "None" pre-selected on a freshly-created --auth public app
• Reviewer-side: UI eyeball — Configuration → Secrets tab shows #KBC_TOKEN listed (encrypted ciphertext) after a secrets-set write
• Reviewer-side: optional E2E run with reviewer's own E2E_API_TOKEN + E2E_URL

Out of scope (orphan-prevention issues filed before merge)

• data-app logs + auto-log-dump on deploy failure → tracked as
  data-app logs + auto-log-dump on deploy failure #240 (OPEN; needs platform-side API exposure first — the Data
  Science API does not document a programmatic logs endpoint per the
  Terminal Log tab help canon).
• --auth oidc / --auth github / --auth gitlab / --auth jumpcloud
  / --auth auth0 → tracked as data-app: extend --auth flag to OIDC / GitHub OAuth / GitLab OAuth / JumpCloud / Auth0 #241 (OPEN; per-provider config
  schemas need a separate design pass for client_secret encryption,
  per-provider flags, and live-validation against each external service).
• data-app validate-repo --type for non-python-js repos and
  non-GitHub hosts → noted in data-app: extend --auth flag to OIDC / GitHub OAuth / GitLab OAuth / JumpCloud / Auth0 #241's scope adjacent.

Suggested labels (maintainer to apply)

enhancement is the most relevant existing label on this repo. The PR
also touches area:data-app if/when that label exists. gh issue edit --add-label failed with permission errors when filing #240/#241 from
the fork; padak (or any collaborator) can apply labels post-open.

Files changed

23 files, +4093 / -24:

src/keboola_agent_cli/
  commands/data_app.py             — 5 new commands + Reference: epilogs
  commands/context.py              — AGENT_CONTEXT (+44 lines for 5 commands)
  hints/definitions/data_app.py    — 5 new HintRegistry blocks (+262 lines)
  services/data_app_service.py     — secrets methods, _auth_block_for,
                                      _build_public_auth_block,
                                      _redact_secrets_block (+682 lines)
  services/repo_validate_service.py — NEW (759 lines)
  cli.py                           — wire RepoValidateService
  errors.py                        — 3 new ErrorCode entries
  permissions.py                   — 5 new registry entries
  changelog.py                     — 0.28.0 block (7 entries)

tests/
  test_data_app_secrets_service.py     — NEW (456 lines, 27 tests)
  test_data_app_secrets_cli.py         — NEW (596 lines, 22 tests)
  test_data_app_validate_repo_service.py — NEW (283 lines, 20 tests)
  test_data_app_service.py             — TestDataAppCreateAuthBlock added (+107)

plugins/kbagent/
  agents/keboola-expert.md         — matrix (+5 rows), version gate, gotchas
  skills/kbagent/SKILL.md          — auto-regenerated
  skills/kbagent/references/commands-reference.md — 5 new bullets
  skills/kbagent/references/gotchas.md            — 3 new (since v0.28.0)
  skills/kbagent/references/data-app-workflow.md  — recipes + table

CLAUDE.md                          — All CLI Commands +5 lines
pyproject.toml                     — version 0.27.0 → 0.28.0
plugins/kbagent/.claude-plugin/plugin.json — auto-synced
.claude-plugin/marketplace.json    — auto-synced
uv.lock                            — version metadata only

make check clean, single commit, force-pushed-with-lease.

[padak]

Closing in favor of new PR from padak/keboola_agent_cli — branch has been rebased onto feat-0.29.0 and will be merged from there.