Skip to content

feat(collectors): harmonize the collector project structure and workflow (#561) - #562

Merged
Frédéric BASLER (Megafredo) merged 14 commits into
mainfrom
feat/561-harmonize-collectors
Sep 8, 2026
Merged

Frédéric BASLER (Megafredo) merged 14 commits into
mainfrom
feat/561-harmonize-collectors

Conversation

@Megafredo

@Megafredo Frédéric BASLER (Megafredo) commented Aug 10, 2026

Copy link
Copy Markdown
Member

Proposed changes

Align the Dockerfile, pyproject.toml, and README of the collectors with the current project conventions.

In particular:

  • standardize the Python version and Docker setup
  • use the same Poetry installation workflow
  • standardize the collector entry point
  • document the local client-python development workflow using an editable installation
  • ensure the documented commands match the actual project configuration
  • modernize and align project metadata, dependency groups (remove obsolete dependency extras)
  • improve container security, runtime isolation, and Docker layer caching (align template collector) - add and standardize UBI9 images across collectors

Update CircleCI

  • refactor CircleCI into a dynamic, per-collector test, build, and publication workflow
  • automatically discover collectors and isolate their CI jobs
  • standardize repository and Docker exclusion rules

Update Github Action

  • Aligned with Python version 3.14
  • Aligned with Poetry version 2.3.2
  • Removed argument handling method (prod extras) from run_test.sh
  • Removed the search for "test" (All collector's containing tests must have a folder named "tests")

Testing Instructions

  1. Step-by-step how to test
  2. Environment or config notes

Related issues

Checklist

  • I consider the submitted work as finished
  • I tested the code for its functionality
  • I wrote test cases for the relevant uses case
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality
  • For bug fix -> I implemented a test that covers the bug

Further comments

This PR depends on a change in client-python related to Python 3.14 compatibility.

The behavior of class annotations changed between Python 3.13 and Python 3.14, which affects the way pyoaev discovers annotations used to create managers.

The corresponding change is tracked in the client-python repository and must be available before this PR can be fully validated with Python 3.14.

Copilot AI lite review requested due to automatic review settings August 10, 2026 06:45
@Megafredo Frédéric BASLER (Megafredo) added filigran team Item from the Filigran team. do not merge Do not merge until this label is removed. tech foundation Technical refactor or improvement is needed labels Aug 10, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR harmonizes multiple collectors to follow a consistent project structure and execution workflow: standardized python -m <package> entrypoints, aligned Poetry-based installation instructions, and updated Docker/ignore files (notably moving most collectors toward Python 3.14).

Changes:

  • Standardizes collector entrypoints by introducing main() functions and package-level __main__.py wrappers.
  • Updates Dockerfiles (Alpine + UBI variants) and documentation to converge on a single build/install/run workflow.
  • Adds/updates .gitignore / .dockerignore files across collectors to normalize build artifacts and local config exclusions.

Reviewed changes

Copilot reviewed 211 out of 211 changed files in this pull request and generated 31 comments.

Show a summary per file
File Description
xtm-one/xtm_one/openaev_xtm_one.py Switches CLI execution to a main() function for standardized entrypoints.
xtm-one/xtm_one/main.py Adds package python -m xtm_one entrypoint wrapper.
xtm-one/README.md Updates manual deployment instructions to the standardized Poetry workflow.
xtm-one/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
xtm-one/Dockerfile Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m xtm_one.
xtm-one/.gitignore Adds standardized ignores for local config and build/cache artifacts.
xtm-one/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
template/README.md Updates template documentation for Python 3.14+ and standardized local dev instructions.
template/pyproject.toml Migrates template metadata/scripts and removes obsolete extras to match the new conventions.
template/Dockerfile_ubi9 Adds a UBI9 build/deploy path aligned with the new standard runtime expectations.
template/Dockerfile Bumps base image to Python 3.14 and aligns Poetry install steps with the standardized workflow.
tanium-threat-response/tanium_threat_response/openaev_tanium_threat_response.py Switches CLI execution to a main() function for standardized entrypoints.
tanium-threat-response/tanium_threat_response/main.py Adds package python -m tanium_threat_response entrypoint wrapper.
tanium-threat-response/README.md Updates manual deployment instructions to the standardized Poetry workflow.
tanium-threat-response/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
tanium-threat-response/.gitignore Adds standardized ignores for local config and build/cache artifacts.
tanium-threat-response/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
splunk-es/README.md Updates manual deployment instructions to the standardized Poetry workflow.
splunk-es/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
splunk-es/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
sentinelone/README.md Updates manual deployment instructions to the standardized Poetry workflow.
sentinelone/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
sentinelone/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
qradar/README.md Updates manual deployment instructions to the standardized Poetry workflow.
qradar/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
qradar/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
prompt-security/README.md Updates manual deployment instructions to the standardized Poetry workflow.
prompt-security/prompt_security/openaev_prompt_security.py Switches CLI execution to a main() function for standardized entrypoints.
prompt-security/prompt_security/main.py Adds package entrypoint file (currently empty in this diff).
prompt-security/Dockerfile_ubi9 Adds a UBI9 build/deploy path aligned with the new standard runtime expectations.
prompt-security/Dockerfile Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m prompt_security.
prompt-security/.gitignore Adds standardized ignores for local config and build/cache artifacts.
prompt-security/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
prisma-airs/README.md Updates manual deployment instructions to the standardized Poetry workflow.
prisma-airs/prisma_airs/openaev_prisma_airs.py Switches CLI execution to a main() function for standardized entrypoints.
prisma-airs/prisma_airs/main.py Adds package python -m prisma_airs entrypoint wrapper.
prisma-airs/Dockerfile_ubi9 Adds a UBI9 build/deploy path aligned with the new standard runtime expectations.
prisma-airs/.gitignore Adds standardized ignores for local config and build/cache artifacts.
prisma-airs/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
palo-alto-cortex-xsoar/README.md Updates requirements/install/run docs while preserving both Poetry and uv workflows.
palo-alto-cortex-xsoar/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
palo-alto-cortex-xsoar/.python-version Aligns local tooling to Python 3.14.
palo-alto-cortex-xsoar/.gitignore Standardizes ignores for local config and build/cache artifacts.
palo-alto-cortex-xsoar/.dockerignore Adds standardized Docker ignore rules.
palo-alto-cortex-xdr/README.md Updates manual deployment instructions to the standardized Poetry workflow.
palo-alto-cortex-xdr/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
palo-alto-cortex-xdr/.python-version Aligns local tooling to Python 3.14.
palo-alto-cortex-xdr/.gitignore Adds standardized ignores for local config and build/cache artifacts.
palo-alto-cortex-xdr/.dockerignore Adds standardized Docker ignore rules.
openaev/README.md Updates manual deployment instructions to the standardized Poetry workflow.
openaev/openaev/openaev_openaev.py Switches CLI execution to a main() function for standardized entrypoints.
openaev/openaev/main.py Adds package python -m openaev entrypoint wrapper.
openaev/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
openaev/.gitignore Adds standardized ignores for local config and build/cache artifacts.
openaev/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
nvd-nist-cve/README.md Updates manual deployment instructions to the standardized Poetry workflow.
nvd-nist-cve/nvd_nist_cve/openaev_nvd_nist_cve.py Refactors entrypoint logic (introduces a main() function in the diff).
nvd-nist-cve/nvd_nist_cve/main.py Adds package python -m nvd_nist_cve entrypoint wrapper.
nvd-nist-cve/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
nvd-nist-cve/.gitignore Standardizes ignores for local config and build/cache artifacts.
nvd-nist-cve/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
netwitness/README.md Updates manual deployment instructions to the standardized Poetry workflow.
netwitness/pyproject.toml Aligns metadata/scripts/dev tooling for the standardized collector structure.
netwitness/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
netwitness/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
mitre-attack/README.md Updates manual deployment instructions to the standardized Poetry workflow.
mitre-attack/mitre_attack/openaev_mitre.py Switches CLI execution to a main() function for standardized entrypoints.
mitre-attack/mitre_attack/main.py Adds package python -m mitre_attack entrypoint wrapper.
mitre-attack/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
mitre-attack/.gitignore Adds standardized ignores for local config and build/cache artifacts.
mitre-attack/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
mitre-atlas/README.md Updates manual deployment instructions to the standardized Poetry workflow.
mitre-atlas/mitre_atlas/openaev_atlas.py Switches CLI execution to a main() function for standardized entrypoints.
mitre-atlas/mitre_atlas/main.py Adds package python -m mitre_atlas entrypoint wrapper.
mitre-atlas/Dockerfile_ubi9 Adds a UBI9 build/deploy path aligned with the new standard runtime expectations.
mitre-atlas/.gitignore Adds standardized ignores for local config and build/cache artifacts.
mitre-atlas/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
microsoft-sentinel/README.md Updates manual deployment instructions to the standardized Poetry workflow.
microsoft-sentinel/microsoft_sentinel/openaev_microsoft_sentinel.py Switches CLI execution to a main() function for standardized entrypoints.
microsoft-sentinel/microsoft_sentinel/main.py Adds package python -m microsoft_sentinel entrypoint wrapper.
microsoft-sentinel/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
microsoft-sentinel/.gitignore Adds standardized ignores for local config and build/cache artifacts.
microsoft-sentinel/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
microsoft-intune/README.md Updates manual deployment instructions to the standardized Poetry workflow.
microsoft-intune/microsoft_intune/openaev_microsoft_intune.py Switches CLI execution to a main() function for standardized entrypoints.
microsoft-intune/microsoft_intune/main.py Adds package python -m microsoft_intune entrypoint wrapper.
microsoft-intune/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
microsoft-intune/Dockerfile Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m microsoft_intune.
microsoft-intune/.gitignore Adds standardized ignores for local config and build/cache artifacts.
microsoft-intune/.dockerignore Adds standardized Docker ignore rules.
microsoft-entra/README.md Updates manual deployment instructions to the standardized Poetry workflow.
microsoft-entra/microsoft_entra/openaev_microsoft_entra.py Switches CLI execution to a main() function for standardized entrypoints.
microsoft-entra/microsoft_entra/main.py Adds package python -m microsoft_entra entrypoint wrapper.
microsoft-entra/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
microsoft-entra/.gitignore Adds standardized ignores for local config and build/cache artifacts.
microsoft-entra/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
microsoft-defender/README.md Updates manual deployment instructions to the standardized Poetry workflow.
microsoft-defender/microsoft_defender/openaev_microsoft_defender.py Switches CLI execution to a main() function for standardized entrypoints.
microsoft-defender/microsoft_defender/main.py Adds package python -m microsoft_defender entrypoint wrapper.
microsoft-defender/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
microsoft-defender/.gitignore Adds standardized ignores for local config and build/cache artifacts.
microsoft-defender/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
microsoft-defender-o365/README.md Updates install/run instructions to remove extras and document editable local client-python installs.
microsoft-defender-o365/pyproject.toml Updates metadata/dependency declarations and removes obsolete extras markers.
microsoft-defender-o365/Dockerfile Bumps base image to Python 3.14 and aligns Poetry install behavior (--only main).
microsoft-azure/README.md Updates manual deployment instructions to the standardized Poetry workflow.
microsoft-azure/microsoft_azure/openaev_microsoft_azure.py Switches CLI execution to a main() function for standardized entrypoints.
microsoft-azure/microsoft_azure/main.py Adds package python -m microsoft_azure entrypoint wrapper.
microsoft-azure/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
microsoft-azure/Dockerfile Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m microsoft_azure.
microsoft-azure/.gitignore Adds standardized ignores for local config and build/cache artifacts.
microsoft-azure/.dockerignore Adds standardized Docker ignore rules.
logrhythm/README.md Updates manual deployment instructions to the standardized Poetry workflow.
logrhythm/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
logrhythm/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
lakera-guard/README.md Updates manual deployment instructions to the standardized Poetry workflow.
lakera-guard/lakera_guard/openaev_lakera_guard.py Switches CLI execution to a main() function for standardized entrypoints.
lakera-guard/lakera_guard/main.py Adds package python -m lakera_guard entrypoint wrapper.
lakera-guard/Dockerfile_ubi9 Adds a UBI9 build/deploy path aligned with the new standard runtime expectations.
lakera-guard/.gitignore Adds standardized ignores for local config and build/cache artifacts.
lakera-guard/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
hiddenlayer/README.md Updates manual deployment instructions to the standardized Poetry workflow.
hiddenlayer/hiddenlayer/openaev_hiddenlayer.py Switches CLI execution to a main() function for standardized entrypoints.
hiddenlayer/hiddenlayer/main.py Adds package python -m hiddenlayer entrypoint wrapper.
hiddenlayer/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
hiddenlayer/Dockerfile Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m hiddenlayer.
hiddenlayer/.gitignore Adds standardized ignores for local config and build/cache artifacts.
hiddenlayer/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
google-workspace/README.md Updates manual deployment instructions to the standardized Poetry workflow.
google-workspace/google_workspace/openaev_google_workspace.py Switches CLI execution to a main() function for standardized entrypoints.
google-workspace/google_workspace/main.py Adds package python -m google_workspace entrypoint wrapper.
google-workspace/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
google-workspace/Dockerfile Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m google_workspace.
google-workspace/.gitignore Adds standardized ignores for local config and build/cache artifacts.
google-workspace/.dockerignore Adds standardized Docker ignore rules.
elastic/README.md Updates manual deployment instructions to the standardized Poetry workflow.
elastic/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
elastic/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
crowdstrike/README.md Updates manual deployment instructions to the standardized Poetry workflow.
crowdstrike/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
crowdstrike/crowdstrike/openaev_crowdstrike.py Switches CLI execution to a main() function for standardized entrypoints.
crowdstrike/crowdstrike/main.py Adds package python -m crowdstrike entrypoint wrapper.
crowdstrike/.gitignore Standardizes ignores for local config and build/cache artifacts.
crowdstrike/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
cisco-ai-defense/README.md Updates manual deployment instructions to the standardized Poetry workflow.
cisco-ai-defense/Dockerfile_ubi9 Adds a UBI9 build/deploy path aligned with the new standard runtime expectations.
cisco-ai-defense/cisco_ai_defense/openaev_cisco_ai_defense.py Switches CLI execution to a main() function for standardized entrypoints.
cisco-ai-defense/cisco_ai_defense/main.py Adds package python -m cisco_ai_defense entrypoint wrapper.
cisco-ai-defense/.gitignore Adds standardized ignores for local config and build/cache artifacts.
cisco-ai-defense/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
aws-resources/README.md Updates manual deployment instructions to the standardized Poetry workflow.
aws-resources/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
aws-resources/Dockerfile Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m aws_resources.
aws-resources/aws_resources/openaev_aws_resources.py Switches CLI execution to a main() function for standardized entrypoints.
aws-resources/aws_resources/main.py Adds package python -m aws_resources entrypoint wrapper.
aws-resources/.gitignore Adds standardized ignores for local config and build/cache artifacts.
aws-resources/.dockerignore Adds standardized Docker ignore rules.
atomic-red-team/README.md Updates manual deployment instructions to the standardized Poetry workflow.
atomic-red-team/Dockerfile_ubi9 Updates UBI runtime to Python 3.14 and standardizes module entrypoint.
atomic-red-team/atomic_red_team/openaev_atomic_red_team.py Switches CLI execution to a main() function for standardized entrypoints.
atomic-red-team/atomic_red_team/main.py Adds package python -m atomic_red_team entrypoint wrapper.
atomic-red-team/.gitignore Adds standardized ignores for local config and build/cache artifacts.
atomic-red-team/.dockerignore Reworks ignore rules to a standardized, security-conscious template.
Suppressed comments (1)

nvd-nist-cve/nvd_nist_cve/openaev_nvd_nist_cve.py:26

  • openaev_nvd_nist_cve.py now defines main() twice (the second definition overwrites the first), which changes runtime behavior (drops the try/except + docstring) and can confuse entrypoints. Keep a single main() and move the env-var aliasing logic out of a second main() definition.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread prompt-security/prompt_security/__main__.py
Comment thread template/Dockerfile_ubi9 Outdated
Comment thread netwitness/Dockerfile_ubi9 Outdated
Comment thread netwitness/pyproject.toml
Comment thread xtm-one/Dockerfile_ubi9 Outdated
Comment thread palo-alto-cortex-xsoar/Dockerfile_ubi9 Outdated
Comment thread microsoft-defender/Dockerfile_ubi9 Outdated
Comment thread microsoft-sentinel/Dockerfile_ubi9 Outdated
Comment thread crowdstrike/Dockerfile_ubi9 Outdated
Comment thread splunk-es/Dockerfile_ubi9 Outdated
@mariot

Copy link
Copy Markdown
Member

🧪 PR #562 Test Report — Collector Harmonization (updated with pyoaev patch)

Branch tested: feat/561-harmonize-collectors @ 579dd36
Environment: Docker (colima, arm64), Mimikyu (fake OpenAEV) + ofapi (fake vendor APIs), Poetry 2.4.1

Update: pyoaev blocker resolved

The initial pass found that every real collector crashed after registering with Mimikyu, with:

AttributeError: 'Collector' object has no attribute '__annotations__'
  File ".../pyoaev/base.py", line 160, in _create_managers

This was a Python 3.14 incompatibility in pyoaev (client-python) 3.260805.0, surfaced by this PR's bump to Python 3.14 across Dockerfiles/pyproject files.

The fix from client-python PR #339 was applied (patch pyoaev/base.py::_create_managers to use get_type_hints(type(self)) instead of self.__annotations__, injected directly into each container's installed pyoaev package) and all 29 collectors were re-tested functionally. Result: the patch fully resolves the crash across the board — 26/28 real collectors now complete a full successful end-to-end cycle (register with Mimikyu, call ofapi's vendor fake or push public data, settle into their polling loop) with zero new errors.

🐛 New finding: separate Python 3.14 bug in 2 collectors

Once unblocked from the pyoaev issue, microsoft-defender and microsoft-entra hit a distinct, new crash in their own code (not pyoaev, not vendor-auth related):

RuntimeError: There is no current event loop in thread 'MainThread'

at asyncio.get_event_loop() (openaev_microsoft_defender.py:457, openaev_microsoft_entra.py:233). In Python 3.14, asyncio.get_event_loop() no longer implicitly creates an event loop when called from a thread that has none — this is a real, unresolved incompatibility in these two collectors' own source that should be fixed (e.g. via asyncio.new_event_loop() + set_event_loop(), or asyncio.run()) before/alongside this PR's Python 3.14 bump.

Full results by collector

Collector Docker build UBI9 image Poetry install Functional test (after pyoaev patch)
crowdstrike ✅ full success — registered, queried ofapi CrowdStrike alerts, processed expectation
tanium-threat-response ✅ full success — registered, queried ofapi Tanium alerts endpoint
xtm-one ✅ full success — registered, catalog import + AI-expectation check against ofapi
elastic ✅ full success — registered, entered expectation-polling loop cleanly
logrhythm ✅ full success — registered, entered expectation-polling loop cleanly
netwitness ✅ full success — full setup + processing cycle completed
qradar ✅ full success — full setup + processing cycle completed
splunk-es ✅ full success — full setup + processing cycle completed
palo-alto-cortex-xdr ⚠️ pyoaev crash gone; reaches real HTTPS vendor call, fails only on expected Connection refused (ofapi not TLS-fronted at :8443 — out of scope)
palo-alto-cortex-xsoar ✅ full success — called ofapi endpoint, processed empty result cleanly
sentinelone ✅ full success — full setup + processing cycle completed
hiddenlayer ✅ full success — registered platform + collector, polling AI expectations
lakera-guard ✅ full success — registered platform + collector, polling AI expectations
prisma-airs ✅ full success — registered platform + collector, polling AI expectations
prompt-security ✅ full success — registered platform + collector, polling AI expectations
atomic-red-team ✅ full success — imported real ATT&CK-mapped payloads into Mimikyu, looping
cisco-ai-defense ✅ full success — registered platform + collector, polling AI expectations
mitre-atlas ✅ full success — fetched real MITRE ATLAS STIX bundle from GitHub, looping
mitre-attack ✅ full success — fetched real MITRE ATT&CK STIX bundle from GitHub, looping
nvd-nist-cve ✅ full success — actively fetching real NVD CVE corpus in batches
openaev ✅ full success — imported built-in payloads into Mimikyu, looping
template N/A (scaffold only) — clean config error, no import issues
aws-resources ✅ unblocked — reaches real AWS STS call, fails as expected (InvalidClientTokenId)
google-workspace ✅ unblocked — reaches real Google OAuth2 endpoint, fails as expected (invalid_grant)
microsoft-azure ✅ unblocked — reaches real Azure AD tenant discovery, fails as expected (AADSTS90002)
microsoft-defender 🐛 NEW bug found: RuntimeError: no current event loop in thread 'MainThread' at asyncio.get_event_loop() — collector-side Python 3.14 incompatibility, blocks before reaching vendor auth
microsoft-defender-o365 ✅ unblocked — reaches real Azure AD tenant discovery, fails as expected (AADSTS90002)
microsoft-entra 🐛 NEW bug found: same asyncio.get_event_loop() RuntimeError as microsoft-defender
microsoft-intune ✅ unblocked — reaches real Azure AD tenant discovery, fails as expected (AADSTS90002), then gracefully logs "0 managed devices found"

Totals: 29/29 Docker builds ✅ · 29/29 UBI9 builds ✅ · 29/29 Poetry installs ✅ · 26/28 applicable functional tests fully green after the pyoaev patch · 2 collectors (microsoft-defender, microsoft-entra) blocked by a newly-discovered, separate Python 3.14 asyncio bug in their own code.

Other notes

  • Dockerfiles use RUN --mount=type=cache, requiring BuildKit (DOCKER_BUILDKIT=1) — fails on legacy/classic builders.
  • COLLECTOR_PERIOD must be an ISO-8601 duration (e.g. PT1M); a bare integer of seconds fails Pydantic validation — a test-harness detail, not a PR bug.

Conclusion

The Dockerfile/pyproject/README harmonization changes in this PR are solid — no packaging regressions across all 29 collectors. With the upstream pyoaev fix (client-python #339) applied, 26 of 28 real collectors run cleanly end-to-end. Recommend:

  1. Bumping/patching the pyoaev dependency pin to include the fix from client-python PR fix(microsoft-defender): handle custom detection rules when ProcessId and ProcessCommandLine are not present in AlertEvidence #339 before merging (or merging that PR first).
  2. Fixing the asyncio.get_event_loop() incompatibility in microsoft-defender and microsoft-entra for Python 3.14 compatibility (this PR's version bump exposes it).

@mariot Mariot Tsitoara (mariot) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it works!
just need to rebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Do not merge until this label is removed. filigran team Item from the Filigran team. tech foundation Technical refactor or improvement is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(collectors): harmonize collector dockerfiles, pyproject.toml

4 participants