Problem
Current sync pull and sync push operate on the entire project -- all configurations at once. For many real-world use cases, users need to work with a single specific configuration:
- CI/CD deployment of data apps -- push just one Streamlit/data-app config after build
- Quick iteration on a single transformation -- pull one config, edit, push back
- Selective config management -- update just a specific extractor config without touching anything else
- Lightweight usage -- no need for full
sync init + project workspace when you just need one config
Proposed Solution
New CLI options for existing commands
# Pull a single configuration
kbagent sync pull --project my-proj --config keboola.python-transformation-v2/12345
# Push a single configuration
kbagent sync push --project my-proj --config keboola.python-transformation-v2/12345
# Shorthand: component-id and config-id as separate flags
kbagent sync pull --project my-proj --component-id keboola.python-transformation-v2 --config-id 12345
Standalone single-config mode (no sync init required)
For CI/CD and quick one-off operations, a lightweight mode that doesn't need a full .keboola/manifest.json:
# Pull single config to a directory (creates _config.yml + code files)
kbagent config pull --project my-proj --component-id keboola.python-transformation-v2 --config-id 12345 --output-dir ./my-config/
# Push single config from a directory
kbagent config push --project my-proj --component-id keboola.python-transformation-v2 --config-id 12345 --input-dir ./my-config/
# Push with auto-create (if config doesn't exist yet, create it)
kbagent config push --project my-proj --component-id keboola.python-transformation-v2 --input-dir ./my-config/ --create
CI/CD example (data app deployment)
# GitHub Actions example
- name: Deploy data app config to Keboola
run: |
kbagent config push \
--project production \
--component-id keboola.data-apps \
--config-id ${{ vars.DATA_APP_CONFIG_ID }} \
--input-dir ./data-app-config/
Technical Analysis
Complexity: Medium
The building blocks already exist:
- API client methods --
get_config_detail(), create_config(), update_config() + row equivalents are all implemented in client.py
- Config format conversion --
api_config_to_local() / local_config_to_api() in sync/config_format.py handle bidirectional YAML <-> API JSON
- Code extraction --
extract_code_files() / merge_code_files() in sync/code_extraction.py handle SQL/Python file splitting
- Row support --
list_config_rows(), create_config_row(), update_config_row() are implemented
What needs to be built
Option A: Filter mode (within existing sync)
- Add
--config filter to pull() and push() in SyncService
- For pull: fetch only
get_config_detail() + list_config_rows() instead of list_components_with_configs()
- For push: filter diff changeset to only the specified config
- Manifest tracking still applies (subset update)
- Effort: ~2-3 days
Option B: Standalone mode (no sync init)
- New
config pull / config push commands in commands/config.py
- Lightweight: reads/writes a single directory with
_config.yml + code files
- No manifest needed -- uses component_id + config_id directly
- Reuses existing format conversion and code extraction
- Needs: detect create vs update, handle rows, encrypted values
- Effort: ~3-5 days
Recommended: Both options
- Option A is quick and useful for developers already using sync
- Option B unlocks the CI/CD use case which is the primary driver
Edge cases to handle
- Encrypted values (
#encrypted# markers) -- on push, skip fields that haven't changed to avoid overwriting secrets
- Config rows -- need to handle row-level CRUD (add/update/delete individual rows)
- Dev branch support -- both modes should respect
--branch flag
- Conflict detection in filter mode -- what if remote changed since last pull?
- Config creation --
config push --create needs to handle first-time deployment
Open Questions
- Should standalone mode support rows, or just top-level config for v1?
- Should
config push auto-encrypt values marked with # prefix? (Current sync push has this logic)
- Naming:
config pull/push vs sync pull --config vs new deploy command group?
Problem
Current
sync pullandsync pushoperate on the entire project -- all configurations at once. For many real-world use cases, users need to work with a single specific configuration:sync init+ project workspace when you just need one configProposed Solution
New CLI options for existing commands
Standalone single-config mode (no sync init required)
For CI/CD and quick one-off operations, a lightweight mode that doesn't need a full
.keboola/manifest.json:CI/CD example (data app deployment)
Technical Analysis
Complexity: Medium
The building blocks already exist:
get_config_detail(),create_config(),update_config()+ row equivalents are all implemented inclient.pyapi_config_to_local()/local_config_to_api()insync/config_format.pyhandle bidirectional YAML <-> API JSONextract_code_files()/merge_code_files()insync/code_extraction.pyhandle SQL/Python file splittinglist_config_rows(),create_config_row(),update_config_row()are implementedWhat needs to be built
Option A: Filter mode (within existing sync)
--configfilter topull()andpush()inSyncServiceget_config_detail()+list_config_rows()instead oflist_components_with_configs()Option B: Standalone mode (no sync init)
config pull/config pushcommands incommands/config.py_config.yml+ code filesRecommended: Both options
Edge cases to handle
#encrypted#markers) -- on push, skip fields that haven't changed to avoid overwriting secrets--branchflagconfig push --createneeds to handle first-time deploymentOpen Questions
config pushauto-encrypt values marked with#prefix? (Currentsync pushhas this logic)config pull/pushvssync pull --configvs newdeploycommand group?