Run the Docker build checks with python -I - #4
Merged
Conversation
The image carries two copies of the package: the source tree that
COPY . . places at /app, and the copy pip builds into site-packages.
The source tree has every data file, because they are checked into git.
The installed copy has only what package_data selected, which is what
was broken in 0.1.2.
The build checks were reading the wrong copy. Both ran under WORKDIR
/app, and python -c puts the working directory at the head of sys.path,
so `import BLIMMP_Scripts` found /app/BLIMMP_Scripts and never reached
site-packages. Nothing outside the image was involved; both copies are
baked in.
Two consequences, both visible in the published 0.1.3 image:
- The packaged-data assertion inspected the source tree, where the
five required files exist by construction. It could not fail. That
is why the broken 0.1.2 image built cleanly, and it means the guard
added against a packaging regression did nothing.
- ensure_module_graphs() resolved its target directory relative to the
source copy, so the KEGG module graphs extracted into /app, which
nothing reads at run time. site-packages received none of the 340
module_*_nodes.json files, so every run re-extracts from the zip
into a cache dir instead of using graphs baked into the image.
Run time was never affected. The BLIMMP console script lives in
/usr/local/bin, and for a script Python puts the script's own directory
on sys.path rather than the working directory, so real runs have always
imported from site-packages.
-I (isolated mode) keeps the working directory off sys.path, so both
commands resolve to the installed copy. The assertion now fails the
build if package_data drops a file again.
Verified against the published 0.1.3 image by pulling its layers from
ghcr: site-packages holds the data files but no extracted graphs, and
the extraction sits under /app instead.
Once merged, bump the version to 0.1.4 and tag v0.1.4 to publish an
image with the graphs pre-extracted.
Implemented with assistance from Claude (Opus 5)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The build-time checks added in #3 were inspecting the wrong copy of the package, so the
assertion meant to catch a packaging regression could never fail.
The image holds two copies of BLIMMP_Scripts: the source tree that
COPY . .puts at/app, and the copy pip builds into site-packages. The source tree has every data file,since they are checked into git. The installed copy has only what
package_dataselected,which is what broke in 0.1.2. Both build checks ran under
WORKDIR /app, andpython -cputs the working directory first on
sys.path, soimport BLIMMP_Scriptsresolved to/appand never reached site-packages.Two effects, both present in the published 0.1.3 image:
regardless of what pip installed. The broken 0.1.2 image would have built cleanly
under it.
ensure_module_graphs()extracted the KEGG module graphs into/app, where nothingreads them. site-packages got none of the 340
module_*_nodes.jsonfiles, so each runre-extracts from the zip into a cache directory.
Adding
-Ito both commands keeps the working directory offsys.path. The graphs nowextract into the installed package and get baked into the image, and the assertion fails
the build if
package_datadrops a file again.Runs were never affected. The
BLIMMPconsole script lives in/usr/local/bin, and for ascript Python puts the script's own directory on
sys.pathrather than the workingdirectory, so real runs have always imported from site-packages. Results from 0.1.3 are
correct.
Verification: CI passes on this branch, including the read-only filesystem case, and the
build log now prints
where it previously printed an
/apppath. I also confirmed the 0.1.3 behavior directlyby pulling that image's layers from ghcr: site-packages holds the data files but no
extracted graphs, and the extraction sits under
/app.After merge, bump to 0.1.4 and tag
v0.1.4to publish an image with the graphspre-extracted.