feat(collectors): harmonize the collector project structure and workflow (#561) - #562
Conversation
There was a problem hiding this comment.
Pull request overview
This PR harmonizes multiple collectors to follow a consistent project structure and execution workflow: standardized python -m <package> entrypoints, aligned Poetry-based installation instructions, and updated Docker/ignore files (notably moving most collectors toward Python 3.14).
Changes:
- Standardizes collector entrypoints by introducing
main()functions and package-level__main__.pywrappers. - Updates Dockerfiles (Alpine + UBI variants) and documentation to converge on a single build/install/run workflow.
- Adds/updates
.gitignore/.dockerignorefiles across collectors to normalize build artifacts and local config exclusions.
Reviewed changes
Copilot reviewed 211 out of 211 changed files in this pull request and generated 31 comments.
Show a summary per file
| File | Description |
|---|---|
| xtm-one/xtm_one/openaev_xtm_one.py | Switches CLI execution to a main() function for standardized entrypoints. |
| xtm-one/xtm_one/main.py | Adds package python -m xtm_one entrypoint wrapper. |
| xtm-one/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| xtm-one/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| xtm-one/Dockerfile | Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m xtm_one. |
| xtm-one/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| xtm-one/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| template/README.md | Updates template documentation for Python 3.14+ and standardized local dev instructions. |
| template/pyproject.toml | Migrates template metadata/scripts and removes obsolete extras to match the new conventions. |
| template/Dockerfile_ubi9 | Adds a UBI9 build/deploy path aligned with the new standard runtime expectations. |
| template/Dockerfile | Bumps base image to Python 3.14 and aligns Poetry install steps with the standardized workflow. |
| tanium-threat-response/tanium_threat_response/openaev_tanium_threat_response.py | Switches CLI execution to a main() function for standardized entrypoints. |
| tanium-threat-response/tanium_threat_response/main.py | Adds package python -m tanium_threat_response entrypoint wrapper. |
| tanium-threat-response/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| tanium-threat-response/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| tanium-threat-response/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| tanium-threat-response/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| splunk-es/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| splunk-es/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| splunk-es/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| sentinelone/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| sentinelone/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| sentinelone/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| qradar/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| qradar/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| qradar/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| prompt-security/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| prompt-security/prompt_security/openaev_prompt_security.py | Switches CLI execution to a main() function for standardized entrypoints. |
| prompt-security/prompt_security/main.py | Adds package entrypoint file (currently empty in this diff). |
| prompt-security/Dockerfile_ubi9 | Adds a UBI9 build/deploy path aligned with the new standard runtime expectations. |
| prompt-security/Dockerfile | Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m prompt_security. |
| prompt-security/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| prompt-security/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| prisma-airs/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| prisma-airs/prisma_airs/openaev_prisma_airs.py | Switches CLI execution to a main() function for standardized entrypoints. |
| prisma-airs/prisma_airs/main.py | Adds package python -m prisma_airs entrypoint wrapper. |
| prisma-airs/Dockerfile_ubi9 | Adds a UBI9 build/deploy path aligned with the new standard runtime expectations. |
| prisma-airs/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| prisma-airs/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| palo-alto-cortex-xsoar/README.md | Updates requirements/install/run docs while preserving both Poetry and uv workflows. |
| palo-alto-cortex-xsoar/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| palo-alto-cortex-xsoar/.python-version | Aligns local tooling to Python 3.14. |
| palo-alto-cortex-xsoar/.gitignore | Standardizes ignores for local config and build/cache artifacts. |
| palo-alto-cortex-xsoar/.dockerignore | Adds standardized Docker ignore rules. |
| palo-alto-cortex-xdr/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| palo-alto-cortex-xdr/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| palo-alto-cortex-xdr/.python-version | Aligns local tooling to Python 3.14. |
| palo-alto-cortex-xdr/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| palo-alto-cortex-xdr/.dockerignore | Adds standardized Docker ignore rules. |
| openaev/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| openaev/openaev/openaev_openaev.py | Switches CLI execution to a main() function for standardized entrypoints. |
| openaev/openaev/main.py | Adds package python -m openaev entrypoint wrapper. |
| openaev/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| openaev/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| openaev/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| nvd-nist-cve/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| nvd-nist-cve/nvd_nist_cve/openaev_nvd_nist_cve.py | Refactors entrypoint logic (introduces a main() function in the diff). |
| nvd-nist-cve/nvd_nist_cve/main.py | Adds package python -m nvd_nist_cve entrypoint wrapper. |
| nvd-nist-cve/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| nvd-nist-cve/.gitignore | Standardizes ignores for local config and build/cache artifacts. |
| nvd-nist-cve/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| netwitness/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| netwitness/pyproject.toml | Aligns metadata/scripts/dev tooling for the standardized collector structure. |
| netwitness/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| netwitness/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| mitre-attack/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| mitre-attack/mitre_attack/openaev_mitre.py | Switches CLI execution to a main() function for standardized entrypoints. |
| mitre-attack/mitre_attack/main.py | Adds package python -m mitre_attack entrypoint wrapper. |
| mitre-attack/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| mitre-attack/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| mitre-attack/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| mitre-atlas/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| mitre-atlas/mitre_atlas/openaev_atlas.py | Switches CLI execution to a main() function for standardized entrypoints. |
| mitre-atlas/mitre_atlas/main.py | Adds package python -m mitre_atlas entrypoint wrapper. |
| mitre-atlas/Dockerfile_ubi9 | Adds a UBI9 build/deploy path aligned with the new standard runtime expectations. |
| mitre-atlas/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| mitre-atlas/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| microsoft-sentinel/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| microsoft-sentinel/microsoft_sentinel/openaev_microsoft_sentinel.py | Switches CLI execution to a main() function for standardized entrypoints. |
| microsoft-sentinel/microsoft_sentinel/main.py | Adds package python -m microsoft_sentinel entrypoint wrapper. |
| microsoft-sentinel/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| microsoft-sentinel/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| microsoft-sentinel/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| microsoft-intune/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| microsoft-intune/microsoft_intune/openaev_microsoft_intune.py | Switches CLI execution to a main() function for standardized entrypoints. |
| microsoft-intune/microsoft_intune/main.py | Adds package python -m microsoft_intune entrypoint wrapper. |
| microsoft-intune/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| microsoft-intune/Dockerfile | Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m microsoft_intune. |
| microsoft-intune/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| microsoft-intune/.dockerignore | Adds standardized Docker ignore rules. |
| microsoft-entra/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| microsoft-entra/microsoft_entra/openaev_microsoft_entra.py | Switches CLI execution to a main() function for standardized entrypoints. |
| microsoft-entra/microsoft_entra/main.py | Adds package python -m microsoft_entra entrypoint wrapper. |
| microsoft-entra/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| microsoft-entra/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| microsoft-entra/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| microsoft-defender/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| microsoft-defender/microsoft_defender/openaev_microsoft_defender.py | Switches CLI execution to a main() function for standardized entrypoints. |
| microsoft-defender/microsoft_defender/main.py | Adds package python -m microsoft_defender entrypoint wrapper. |
| microsoft-defender/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| microsoft-defender/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| microsoft-defender/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| microsoft-defender-o365/README.md | Updates install/run instructions to remove extras and document editable local client-python installs. |
| microsoft-defender-o365/pyproject.toml | Updates metadata/dependency declarations and removes obsolete extras markers. |
| microsoft-defender-o365/Dockerfile | Bumps base image to Python 3.14 and aligns Poetry install behavior (--only main). |
| microsoft-azure/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| microsoft-azure/microsoft_azure/openaev_microsoft_azure.py | Switches CLI execution to a main() function for standardized entrypoints. |
| microsoft-azure/microsoft_azure/main.py | Adds package python -m microsoft_azure entrypoint wrapper. |
| microsoft-azure/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| microsoft-azure/Dockerfile | Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m microsoft_azure. |
| microsoft-azure/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| microsoft-azure/.dockerignore | Adds standardized Docker ignore rules. |
| logrhythm/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| logrhythm/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| logrhythm/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| lakera-guard/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| lakera-guard/lakera_guard/openaev_lakera_guard.py | Switches CLI execution to a main() function for standardized entrypoints. |
| lakera-guard/lakera_guard/main.py | Adds package python -m lakera_guard entrypoint wrapper. |
| lakera-guard/Dockerfile_ubi9 | Adds a UBI9 build/deploy path aligned with the new standard runtime expectations. |
| lakera-guard/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| lakera-guard/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| hiddenlayer/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| hiddenlayer/hiddenlayer/openaev_hiddenlayer.py | Switches CLI execution to a main() function for standardized entrypoints. |
| hiddenlayer/hiddenlayer/main.py | Adds package python -m hiddenlayer entrypoint wrapper. |
| hiddenlayer/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| hiddenlayer/Dockerfile | Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m hiddenlayer. |
| hiddenlayer/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| hiddenlayer/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| google-workspace/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| google-workspace/google_workspace/openaev_google_workspace.py | Switches CLI execution to a main() function for standardized entrypoints. |
| google-workspace/google_workspace/main.py | Adds package python -m google_workspace entrypoint wrapper. |
| google-workspace/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| google-workspace/Dockerfile | Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m google_workspace. |
| google-workspace/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| google-workspace/.dockerignore | Adds standardized Docker ignore rules. |
| elastic/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| elastic/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| elastic/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| crowdstrike/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| crowdstrike/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| crowdstrike/crowdstrike/openaev_crowdstrike.py | Switches CLI execution to a main() function for standardized entrypoints. |
| crowdstrike/crowdstrike/main.py | Adds package python -m crowdstrike entrypoint wrapper. |
| crowdstrike/.gitignore | Standardizes ignores for local config and build/cache artifacts. |
| crowdstrike/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| cisco-ai-defense/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| cisco-ai-defense/Dockerfile_ubi9 | Adds a UBI9 build/deploy path aligned with the new standard runtime expectations. |
| cisco-ai-defense/cisco_ai_defense/openaev_cisco_ai_defense.py | Switches CLI execution to a main() function for standardized entrypoints. |
| cisco-ai-defense/cisco_ai_defense/main.py | Adds package python -m cisco_ai_defense entrypoint wrapper. |
| cisco-ai-defense/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| cisco-ai-defense/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
| aws-resources/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| aws-resources/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| aws-resources/Dockerfile | Reworks Docker build to a multi-stage Poetry venv approach and runs via python -m aws_resources. |
| aws-resources/aws_resources/openaev_aws_resources.py | Switches CLI execution to a main() function for standardized entrypoints. |
| aws-resources/aws_resources/main.py | Adds package python -m aws_resources entrypoint wrapper. |
| aws-resources/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| aws-resources/.dockerignore | Adds standardized Docker ignore rules. |
| atomic-red-team/README.md | Updates manual deployment instructions to the standardized Poetry workflow. |
| atomic-red-team/Dockerfile_ubi9 | Updates UBI runtime to Python 3.14 and standardizes module entrypoint. |
| atomic-red-team/atomic_red_team/openaev_atomic_red_team.py | Switches CLI execution to a main() function for standardized entrypoints. |
| atomic-red-team/atomic_red_team/main.py | Adds package python -m atomic_red_team entrypoint wrapper. |
| atomic-red-team/.gitignore | Adds standardized ignores for local config and build/cache artifacts. |
| atomic-red-team/.dockerignore | Reworks ignore rules to a standardized, security-conscious template. |
Suppressed comments (1)
nvd-nist-cve/nvd_nist_cve/openaev_nvd_nist_cve.py:26
openaev_nvd_nist_cve.pynow definesmain()twice (the second definition overwrites the first), which changes runtime behavior (drops the try/except + docstring) and can confuse entrypoints. Keep a singlemain()and move the env-var aliasing logic out of a secondmain()definition.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
🧪 PR #562 Test Report — Collector Harmonization (updated with pyoaev patch)Branch tested: Update: pyoaev blocker resolvedThe initial pass found that every real collector crashed after registering with Mimikyu, with: This was a Python 3.14 incompatibility in The fix from client-python PR #339 was applied (patch 🐛 New finding: separate Python 3.14 bug in 2 collectorsOnce unblocked from the pyoaev issue, at Full results by collector
Totals: 29/29 Docker builds ✅ · 29/29 UBI9 builds ✅ · 29/29 Poetry installs ✅ · 26/28 applicable functional tests fully green after the pyoaev patch · 2 collectors (microsoft-defender, microsoft-entra) blocked by a newly-discovered, separate Python 3.14 Other notes
ConclusionThe Dockerfile/pyproject/README harmonization changes in this PR are solid — no packaging regressions across all 29 collectors. With the upstream
|
Mariot Tsitoara (mariot)
left a comment
There was a problem hiding this comment.
it works!
just need to rebase
579dd36 to
ab9deb8
Compare
Proposed changes
Align the Dockerfile, pyproject.toml, and README of the collectors with the current project conventions.
In particular:
Update CircleCI
Update Github Action
Testing Instructions
Related issues
Checklist
Further comments
This PR depends on a change in client-python related to Python 3.14 compatibility.
The behavior of class annotations changed between Python 3.13 and Python 3.14, which affects the way pyoaev discovers annotations used to create managers.
The corresponding change is tracked in the client-python repository and must be available before this PR can be fully validated with Python 3.14.