S3 credential check - #281
Closed
MathewsTjale wants to merge 17 commits into
Closed
MathewsTjale wants to merge 17 commits into
MathewsTjale wants to merge 17 commits into
Conversation
Python (both install paths): pypdf, pdfplumber, defusedxml, lxml, validators. JavaScript: docx@9.7.1. packages_ready() now also checks pdfplumber so stale /pkgs volumes reinstall.
Control plane only - sandbox-runner is profile-gated off and the worker's dependency on it is removed; executions launch Firecracker MicroVMs via CODEAPI_SANDBOX_BACKEND=lambda-microvm. Host bindings reduced to 172.17.0.1:3112 (api, for host.docker.internal) and 127.0.0.1:3190 (egress gateway behind the Caddy vhost). Dev-default redis/minio secrets replaced by required env interpolation.
Modern Docker (buildx >= 0.11 with the containerd store) pushes an OCI image INDEX whose top digest includes an attestation manifest. The zip stage then renders FROM <repo>@<index-digest>, and CreateMicrovmImage fails its boot phase with 'An unknown error occurred.' (empty stateReason at image/version level) without ever starting the container. --provenance=false --sbom=false makes the pushed digest a plain arm64 image manifest again, which is what the artifact contract assumes.
The current Lambda MicroVMs image builder cannot exec a shebang script as the container entrypoint: the boot phase dies before the script's first line with CREATE_FAILED / 'An unknown error occurred.' and no stdout. Identical image content with an ELF entrypoint (bash -c wrapper or bun) builds fine, and the script itself runs cleanly inside a live MicroVM guest. Exec-ing bash explicitly is semantically identical and works on both runc and the AWS builder.
The image build's boot/snapshot executor calls ecr-public:GetAuthorizationToken (CloudTrail: AccessDenied for the Lambda-microvmsExecutor session). The module only granted private ecr:GetAuthorizationToken. Grant ecr-public auth plus sts:GetServiceBearerToken so the executor can authenticate to public.ecr.aws.
…e condition RunMicrovm evaluates iam:PassedToService as lambda-microvms.amazonaws.com; with only lambda.amazonaws.com the worker's launch is denied iam:PassRole on the execution role despite the resource match.
…worker PassRole Neither lambda.amazonaws.com nor lambda-microvms.amazonaws.com matches the context RunMicrovm evaluates; the resource pin to the execution role keeps the statement least-privilege.
Non-hardened mode builds manifest claims in the API router only when the manifest key env is present there; base compose sets it worker-only, so MicroVM execs failed with 'X-CodeAPI-Execution-Manifest is required'.
Non-hardened deployments mint upload grants only when the grant secret is present (sandbox-egress.ts); without it the MicroVM runner has the gateway URL but no grant, and generated files are silently dropped (job.ts refuses ungranted uploads). The gateway already verifies with the same secret.
On push to main and manual dispatch, build the lambda-microvm-runner target (arm64, native builder) and push to 323463077991.dkr.ecr.af-south-1.amazonaws.com/synapseai-codeapi:<short-sha>. Delegates to scripts/build-lambda-microvm-artifact.sh build push so CI behaviour matches local runs (buildx flags, provenance/SBOM off, digest captured to .build-lambda-microvm/). zip/upload stages remain manual.
Adds pandoc, libreoffice-writer/calc/impress, tesseract-ocr (+eng) and debianutils to both the lambda-microvm-runner rootfs (api/Dockerfile) and the worker-sandbox image (docker/Dockerfile.worker-sandbox), and mounts a 64MiB noexec tmpfs at /dev/shm so LibreOffice's UNO bootstrap can shm_open during document conversion. Supports the document-processing skills (docx/pptx/xlsx/pdf) running inside the sandbox.
Brings in 113 upstream commits (Hosted App Runner, BYOM workspaces, bug fixes including 'Allow Lambda MicroVM metadata in hardened mode' and memory buffering on streamed uploads). Conflict in scripts/build-lambda-microvm-artifact.sh resolved by taking upstream's parametrized MICROVM_IMAGE_TARGET while keeping our --provenance=false --sbom=false buildx flags. All fork customizations preserved: document-processing packages (pandoc/libreoffice/tesseract), /dev/shm mount, pypdf/pdfplumber/docx, AWS dev overlay, Lambda MicroVM fixes, ECR push workflow.
Adds a build-control-plane matrix job that builds and pushes the six images the Momentum depman spec references, all tagged with the same short SHA so a rollout pins a coherent set: synapseai-codeapi <- service/Dockerfile:api synapseai-codeapi-worker <- service/Dockerfile:worker synapseai-codeapi-files <- service/Dockerfile:production synapseai-codeapi-toolcall <- service/Dockerfile.tool-call-server:production synapseai-codeapi-egress <- service/Dockerfile.egress-gateway:production synapseai-codeapi-sandbox <- api/Dockerfile:sandbox-runner-baked Built for linux/amd64 (Momentum EKS x86 nodes). The sandbox-runner uses the baked target so /pkgs (our fork's package tree) is baked in. The synapseai-codeapi-sandbox ECR repo must exist before that row can push.
The build-and-push job was pointing at synapseai-codeapi (the API repo). With immutable tags and the new control-plane matrix sharing the same short SHA, the API image won the tag race and the runner push failed. Points the Lambda runner at the dedicated synapseai-codeapi-microvm-runner repo (Path B). No change to the control-plane matrix (Path A).
Momentum platform-services nodes are m5 Xen instances with no /dev/kvm (nested virtualization needs m7i/m8i Nitro or bare metal; Lambda MicroVMs isn't in af-south-1). Switch the sandbox image from sandbox-runner-baked (KVM, baked /pkgs) to sandbox-runner (directory root), and add the codeapi-package-init image that populates the shared packages PVC. --target is now optional per matrix row (package-init is single-stage).
…runner repo (#1) The Enviro ECR placeholder for the NsJail sandbox-runner is named synapseai-codeapi-sandbox-runner, but this matrix row pushed to synapseai-codeapi-sandbox, which does not exist in account 323463077991. The image builds fine; only the push failed: ERROR: failed to push .../synapseai-codeapi-sandbox:ff5c770: The repository with name 'synapseai-codeapi-sandbox' does not exist in the registry with id '323463077991' Align the push target with the repo that exists. The deploy spec (hydra/synapseai specs/dev/code-intepreter.yaml) is updated to pull the same name. Co-authored-by: ammaganyane <amos.maganyane@momentum.co.za>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.