馃悰 fix(sqlite): isolate forked connections - #654
Closed
gaborbernat wants to merge 1 commit into
Closed
gaborbernat wants to merge 1 commit into
gaborbernat wants to merge 1 commit into
Conversation
gaborbernat
force-pushed
the
fix/635-sqlite-fork
branch
from
July 14, 2026 09:43
13d0c3e to
3d2a3b2
Compare
gaborbernat
force-pushed
the
fix/635-sqlite-fork
branch
from
July 14, 2026 10:33
3d2a3b2 to
c877a37
Compare
Open SQLite connections for each outer acquisition and abandon inherited handles without calling SQLite in the child. Reset caches and executors after fork, preserve parent ownership, and reject child reuse until exec. Use a reentrant fork-transition gate because connection finalizers can re-enter registration while the current thread owns the gate. Refs tox-dev#635
gaborbernat
force-pushed
the
fix/635-sqlite-fork
branch
from
July 14, 2026 10:40
c877a37 to
acb5754
Compare
This was referenced Jul 14, 2026
gaborbernat
added a commit
that referenced
this pull request
Jul 14, 2026
SQLite forbids a process from using or closing a database connection inherited through `fork()`. Read-write locks kept one connection for the object's lifetime, so a child could close its parent's handle during interpreter shutdown and corrupt the database. Each outer acquisition now owns its connection, and a forked child abandons inherited handles according to [SQLite's fork guidance](https://sqlite.org/howtocorrupt.html#_carrying_an_open_database_connection_across_a_fork_). Closes #635. The child rejects an active database by canonical path or inode until `exec()`. CPython 3.10 and 3.11 keep inherited base connections alive through a raw-reference escrow; later CPython versions use the guarded subclass finalizer. The implementation follows the [CPython 3.10 `_sqlite` lifecycle](https://github.com/python/cpython/blob/v3.10.20/Modules/_sqlite/connection.c) and [current CPython source](https://github.com/python/cpython/blob/main/Modules/_sqlite/connection.c). A child forked from a SQLite callback waits for the call to leave SQLite, or exits with status 70 if an earlier audit hook prevents coordination. Fork registration uses a reentrant gate because connection finalizers can register another lock while the current thread owns that gate. PyPy can retain unsafe process-wide SQLite state after tracked connections close. A PyPy child rejects read-write locks when the parent used SQLite after importing filelock; importing filelock in the child cannot detect earlier parent use. The public acquisition interfaces stay unchanged. Final release closes the per-acquisition connection. A local 2,000-acquisition benchmark measured 5.7 microseconds for the former persistent connection and 45.6 microseconds for the per-acquisition design. This replaces #654, which GitHub closed after merging #653 deleted its base branch `fix/634-fork-ownership`. The branch now sits on `main` and carries the same work. It also carries the `close_failure` fixture change from #656, without which the `pypy3.11` job fails here. Landing #656 first drops that commit from this branch on the next rebase.
gaborbernat
added a commit
that referenced
this pull request
Jul 14, 2026
The `close_failure` fixture patches `filelock._api.os.close`, and `filelock._api.os` is the `os` module, so mocking it swaps `os.close` for the whole process. While a test holds the mock, any `os.close` in the process raises `OSError`, including the ones a garbage collector runs inside a finalizer. CPython hides this. Refcounting reclaims lock objects as soon as a test drops them, outside the window the mock covers. PyPy reclaims them whenever a cycle runs, so a collection can land inside that window. `BaseFileLock.__del__` then force-releases a lock an earlier test left held, its close hits the mock, and the `OSError` escapes the finalizer as an unraisable exception. Pytest reports it against whichever test happens to be running, which is how the `pypy3.11` job on #654 failed inside `test_context_group_handles_deep_body_context`, a test that touches none of this. The mock now raises for the descriptor the test captured through `on_acquired` and hands every other descriptor to the real `os.close`. The lock under test still fails to close, so the `close_error_policy` cases keep their meaning, and a collector running a finalizer mid-test stops tripping over an unrelated fd. This also drops `docs/changelog/coverage-path-remapping.bugfix.rst`. #651 changed how CI maps coverage paths across tox environments, which no user of the library can observe, and the slug filename does not render through the `:pr:` role that `issue_format` applies to every fragment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SQLite forbids a process from using or closing a database connection inherited through
fork(). Read-write locks keptone connection for the object's lifetime, so a child could close its parent's handle during interpreter shutdown and
corrupt the database. Each outer acquisition now owns its connection, and a forked child abandons inherited handles
according to
SQLite's fork guidance.
Closes #635.
The child rejects an active database by canonical path or inode until
exec(). CPython 3.10 and 3.11 keep inheritedbase connections alive through a raw-reference escrow; later CPython versions use the guarded subclass finalizer. The
implementation follows the
CPython 3.10
_sqlitelifecycle andcurrent CPython source. A child forked from
a SQLite callback waits for the call to leave SQLite, or exits with status 70 if an earlier audit hook prevents
coordination. Fork registration uses a reentrant gate because connection finalizers can register another lock while the
current thread owns that gate.
PyPy can retain unsafe process-wide SQLite state after tracked connections close. A PyPy child rejects read-write locks
when the parent used SQLite after importing filelock; importing filelock in the child cannot detect earlier parent use.
The public acquisition interfaces stay unchanged. Final release closes the per-acquisition connection. A local
2,000-acquisition benchmark measured 5.7 microseconds for the former persistent connection and 45.6 microseconds for the
per-acquisition design.