fix: catch OpenAI AuthenticationError before multiprocessing pickle

[DevamShah]

Summary

Catch openai.AuthenticationError and openai.PermissionDeniedError inside _call_model and re-raise as the picklable garak.exception.GarakException, so that runs with --parallel_attempts > 1 abort cleanly on a bad API key instead of crashing the Pool._handle_results background thread.

Problem / motivation

When a user runs garak with --parallel_attempts 4 and an invalid OPENAI_API_KEY, the first generator call in each worker process raises openai.AuthenticationError. That exception is a subclass of openai.APIStatusError, which carries an httpx.Response attribute. httpx.Response is not picklable — its __reduce__ is not defined and its __init__ requires keyword-only arguments (response= and body=) that pickle does not supply on reconstruction.

Python's multiprocessing.Pool runs a background thread (_handle_results) that deserialises worker results from a queue. When pickle.loads() tries to reconstruct the AuthenticationError, it raises:

Exception in thread Thread-6 (_handle_results):
TypeError: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'

This kills _handle_results, the pool becomes unable to deliver any further results, and the main process hangs indefinitely rather than surfacing the root cause to the user. The issue was root-caused in #1357 by a community member who filed no PR.

Change

garak/generators/openai.py — OpenAICompatible._call_model

Added an except (openai.AuthenticationError, openai.PermissionDeniedError) clause in the existing try / except block around generator.create(). Both HTTP 401 and 403 are terminal authentication failures — retrying with the same key serves no purpose and the @backoff decorator must not loop on them. The handler:

1. Logs the full error message at ERROR level, including self.key_env_var so the user knows exactly which environment variable to fix.
2. Raises garak.exception.GarakException (a plain Exception subclass with no un-picklable attributes) from None to suppress the chained traceback.
3. Does not add GarakException to the backoff decorator's exception list — it propagates immediately as a fatal run error.

PermissionDeniedError (HTTP 403) is included because it shares the identical pickling defect and represents the same class of terminal credential failure.

tests/generators/test_openai.py — three new tests

Test	What it asserts
test_call_model_auth_error_raises_garak_exception	HTTP 401 mock → GarakException raised, message contains "401" or the key env var name
test_call_model_auth_exception_is_picklable	The raised exception survives a pickle.dumps / pickle.loads round-trip (the precise property that makes multiprocessing safe)
test_call_model_permission_denied_raises_garak_exception	HTTP 403 mock (via mocker.patch.object) → GarakException raised

The existing auth_fail entry already present in tests/_assets/generators/openai.json is reused for the HTTP-level mocks.

Security rationale

Masking a credential failure behind an unhandled thread exception is an observability gap: the operator has no indication that the run aborted due to an invalid key, so they may misinterpret empty results as "no vulnerabilities found" — a false-negative outcome for a security evaluation tool. Surfacing authentication failures as first-class, logged, named exceptions aligns with OWASP LLM09 (Overreliance) mitigation (auditable failure modes) and the general principle that security tooling must fail loudly and correctly.

Testing / validation

pytest tests/generators/test_openai.py -v

Results on Python 3.14.4, openai 2.41.1, respx 0.23.1, pytest-mock 3.15.1:

tests/generators/test_openai.py::test_openai_version                                      PASSED
tests/generators/test_openai.py::test_openai_invalid_model_names                          PASSED
tests/generators/test_openai.py::test_openai_completion                                   SKIPPED (no live key)
tests/generators/test_openai.py::test_openai_chat                                         SKIPPED (no live key)
tests/generators/test_openai.py::test_reasoning_switch                                    PASSED
tests/generators/test_openai.py::test_call_model_auth_error_raises_garak_exception        PASSED
tests/generators/test_openai.py::test_call_model_auth_exception_is_picklable              PASSED
tests/generators/test_openai.py::test_call_model_permission_denied_raises_garak_exception PASSED

6 passed, 2 skipped

The root cause was also verified empirically: reproducing the broken path (worker raises raw openai.AuthenticationError) produces Exception in thread Thread-6 (_handle_results): TypeError: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'; the fixed path catches cleanly as GarakException through the pool.

[jmartin-tech]

This works, testing shows this will expose the env var value so some additional consideration may be needed as a future improvement to avoid exposing possibly sensitive data in logs/output. That exposure exists today so not reason to expand scope. This will be accepted as is.

% OPENAI_API_KEY="fakekey" python -m garak -t openai -n gpt-4o -p lmrc.Bullying --parallel_attempts 5
garak LLM vulnerability scanner v0.15.2.pre1 ( https://github.com/NVIDIA/garak ) at 2026-06-15T09:28:38.326976
📜 logging to /Users/jemartin/.local/share/garak/garak.log
🦜 loading generator: OpenAI: gpt-4o
📜 reporting to /Users/jemartin/.local/share/garak/garak_runs/garak.1451e1f9-1a21-431d-bc98-887619670cd4.report.jsonl
🕵️  queue of probes: lmrc.Bullying
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 201/201 [00:00<00:00, 64340.62it/s]
probes.lmrc.Bullying:   0%|                                                                                                                         | 0/7 [00:00<?, ?it/s]
OpenAI API authentication failed (HTTP 401); verify OPENAI_API_KEY is valid. Original error: Error code: 401 - {'error': {'message': 'Incorrect API key provided: fakekey. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}