test(ci): add CLI smoke and image format round-trip tests; run ctest in CI - #64
Merged
Merged
Conversation
This was referenced Sep 30, 2026
…structed m_taskThread was declared before m_taskCond, m_taskMutex, m_taskStop and m_taskQueue. Members are constructed in declaration order, so the worker thread started and locked m_taskMutex before the mutex existed. This intermittently aborted every tool that creates a media library (utvio, utv) with: boost::lock_error: mutex lock failed in pthread_mutex_lock: Invalid argument Reproduced in ~1 of 40 serial utvio runs and within ~80 parallel runs; 0 failures in 900 parallel runs after the fix. - Declare m_taskThread last so it starts after the members it uses. - Don't reset m_taskStop in threadMain(): if the destructor ran before the thread started, the reset discarded the stop request and join() hung forever. - Only join() a joinable thread (the protected default constructor never starts one). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
Previously any argument, including --help, ran a live update check that could open a GUI dialog. Matches the --help added to openutv-diagnostics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
…in CI Refs #60. The existing 118 CTest tests were never run on macOS or Linux CI, and the Windows step had continue-on-error: true. New tests (src/test/CliSmokeTest, Python stdlib only): - cli-smoke: every bundled CLI (utvio, utvls, utvpkg, utvpush, utvshell, utvprof, py-interp, openutv-diagnostics, openutv-check-updates) must start, resolve its runtime dependencies and exit with the expected output within a timeout. Uses the Windows .cmd wrappers when present, suppresses Windows loader error dialogs, and runs Qt tools offscreen off macOS. Catches #58-style dependency failures. - image-format-roundtrip: for every image format `utvio -formats` advertises as writable, convert a generated test image to that format and back, then check orientation, channels and values at known sample points. Catches #59 (advertised formats that cannot be written), flipped/flopped output, truncated files and codec regressions. CI: - Run `ctest --output-on-failure` after the macOS and Linux builds in branch-build, dev-build and release (Linux with QT_QPA_PLATFORM=offscreen). - Make the Windows ctest step blocking and raise its timeout for the round-trip test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
…lable FBOVideoDevice required an accelerated CGL pixel format and exit(-1)'d without one, so utvio could not run at all (even -version / -formats) on headless macOS VMs, CI runners or render nodes without a GPU: ERROR: choosing pixel format: invalid pixel format Retry with the existing Apple software renderer attributes and warn, instead of exiting. UTV_SOFTWARE_GL=1 forces the software renderer. Verified: all CLI checks and all 34 image format round-trips pass with UTV_SOFTWARE_GL=1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
- Linux: staged binaries are built without RPATHs, so ctest could not load any of them (including the pre-existing py-interp, QFontTest and FastMemcpyTest tests). Use the same LD_LIBRARY_PATH build.sh sets up (stage lib + linuxbrew lib). - Windows: stage Mesa's opengl32sw.dll as opengl32.dll before ctest, matching the installed layout. Without it utvio got the GDI OpenGL 1.1 stub on the GPU-less runner and crashed (0xC0000005) when rendering. - cli_common: exit code -1 (0xFFFFFFFF) on Windows is a plain exit(-1), not an NTSTATUS crash (utvprof -help exits -1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
- Linux: Qt, OpenColorIO and OpenEXR live in linuxbrew keg dirs (opt/<formula>/lib), not the linked prefix lib, and the staged binaries reference unversioned sonames. Add every keg lib dir to LD_LIBRARY_PATH for ctest (workaround until #66 fixes Linux launch). - format_roundtrip: support per-platform known failures reported as xfail/XPASS. Windows heic/heif/hif (no HEVC encoder) and fits (read back fails) are tracked in #67. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
FBOVideoDevice passed the result of XOpenDisplay() straight to DefaultRootWindow()/DefaultScreen() and dereferenced glXChooseVisual()'s result, so utvio segfaulted (even for -version/-formats) whenever no X display was available, e.g. headless render nodes and CI runners. Exit with a clear error naming DISPLAY and suggesting xvfb-run, and likewise when no suitable GLX visual exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
- Existing tests (FastMemcpyTest, QFontTest, ApplicationTest, AudioRendererTest, LoadingSharedLibrariesTest) replaced LD_LIBRARY_PATH via `cmake -E env`, discarding the caller's library path. Use `cmake -E env --modify LD_LIBRARY_PATH=path_list_prepend:...` instead. - Linux CI: run ctest under xvfb-run so utvio's offscreen GLX context has a display. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
TwkGLF's NV fence sync object called glGenFencesNV & co. directly (via GL_GLEXT_PROTOTYPES). libGL implementations without GL_NV_fence -- Mesa, i.e. AMD/Intel drivers and software rendering -- don't export those symbols, so binaries linking TwkGLF failed to load: symbol lookup error: ...: undefined symbol: glGenFencesNV Resolve the five entry points with glXGetProcAddressARB and only use the NV fence implementation when GL_NV_fence is advertised and all entry points resolve; otherwise fall back to the stub. GL_ARB_sync remains the preferred path, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
…reen in CI - FBOVideoDevice: report a clear error when glXCreateContext or glXMakeCurrent fails, instead of failing later with "frame buffer incomplete: status = 0". - CI: xvfb-run defaults to an 8-bit screen with no 32-bit GLX visual; run ctest on a 1920x1080x24 screen. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
Linux binaries that link the rendering libraries can't load against Mesa's libGL (direct GL extension symbols), and utvio's offscreen FBO fails under llvmpipe. Tracked in #68. - Linux ctest excludes LoadingSharedLibrariesTest, ApplicationTest, AudioRendererTest and image-format-roundtrip until #68 is fixed; all other Linux tests are blocking. - cli-smoke reports utvio on Linux as xfail (XPASS once fixed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Michael Oliver <mcoliver@gmail.com>
mcoliver
force-pushed
the
test/ci-smoke-suite
branch
from
September 30, 2026 15:34
d5fc4d7 to
6e64dba
Compare
mcoliver
changed the base branch from
fix/windows-deps-and-oiio-write
to
main
September 30, 2026 15:34
This was referenced Sep 30, 2026
mcoliver
marked this pull request as ready for review
September 30, 2026 16:16
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First pass at #60: run the existing tests in CI, and add end-to-end tests of the shipped command-line tools and image formats. Also fixes an intermittent crash the new tests found.
Crash fix:
TwkMediaLibrary::Libraryworker thread raceutvio(and anything else that creates a media library, including the GUI) aborted about 1 in 40 runs, more often under load, with:m_taskThreadwas declared beforem_taskMutex,m_taskCond,m_taskStopandm_taskQueue. Members are constructed in declaration order, so the worker thread started and locked a mutex that didn't exist yet.threadMain()also resetm_taskStop = false. If the destructor ran before the thread started, that erased the stop request andjoin()hung forever.join()a joinable thread.utvioruns. Before the fix it crashed within about 80.New tests (
src/test/CliSmokeTest, standard Python only)cli-smoke: each bundled CLI (utvio,utvls,utvpkg,utvpush,utvshell,utvprof,py-interp,openutv-diagnostics,openutv-check-updates) must start, find its runtime dependencies, and exit with the expected output within a timeout..cmdwrappers, the way users run the tools.image-format-roundtrip: every image formatutvio -formatsadvertises as writable gets a generated test image written to it and read back. Colored blocks in three corners are checked for orientation, channels and values, with a looser tolerance for lossy codecs.Bugs found by running the tests in CI, fixed here
utvioon GPU-less Macs (VMs, CI, render nodes): it required a hardware pixel format and exited, even for-version. It now falls back to Apple's software renderer with a warning.UTV_SOFTWARE_GL=1forces the software renderer.utvioon Linux without an X display: segfaulted on an uncheckedXOpenDisplay()/glXChooseVisual(). It now exits with a clear error. GLX context creation failures are reported too.glGenFencesNV& co. were linked directly, so binaries failed to load without NVIDIA's libGL. They're now looked up at runtime.LD_LIBRARY_PATHviacmake -E env. They now prepend to it instead (--modify ...path_list_prepend).Known issues found (tracked, not fixed here)
LD_LIBRARY_PATHas a workaround.xfailon Windows.glBlitFramebufferEXT, …), andutvio's FBO fails under llvmpipe. Linux CI excludes the 4 affected tests, andutvioisxfailincli-smoke.CLI
openutv-check-updates --helpprints help. Previously any argument ran a live update check that could open a dialog.CI
ctest --output-on-failurenow runs after the macOS and Linux builds inbranch-build,dev-buildandrelease. Linux usesQT_QPA_PLATFORM=offscreen.cteststep is now blocking (it hadcontinue-on-error: true), with a timeout long enough for the round-trip test.ctestpasses 120/120 in about 12 s.CI status
Branch CI passes on macOS, Windows and Linux (run 36659762676). The known issues above are reported as
xfailor excluded with links, so they stay visible without blocking.Not in this PR (rest of #60)
UI automation, testing the installed/signed package on clean machines, and movie/audio format round-trips.
🤖 Generated with Claude Code