Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,16 @@ PyRIT (Python Risk Identification Tool for generative AI) is an open-source fram

## Architecture

PyRIT uses a modular pluggable-brick design. The main extensibility points are:
PyRIT uses a modular pluggable-brick design.

- **Prompt Converters** (`pyrit/prompt_converter/`) — Transform prompts (70+ implementations). Base: `PromptConverter`.
- **Scorers** (`pyrit/score/`) — Evaluate responses. Base: `Scorer`.
- **Prompt Targets** (`pyrit/prompt_target/`) — Send prompts to LLMs/APIs. Base: `PromptTarget`.
- **Executors / Scenarios** (`pyrit/executor/`, `pyrit/scenario/`) — Orchestrate multi-turn attacks.
- **Memory** (`pyrit/memory/`) — `CentralMemory` for prompt/response persistence.
**[`doc/code/framework.md`](../doc/code/framework.md) is the canonical reference for how these pieces fit together.** It defines each component's responsibilities — what it owns and, critically, what it *does not* own — and how scenarios, attack techniques, executors, and the core/shared layers relate. Read it before adding or reviewing components so new code lands in the right place.

## Code Review Guidelines

When performing a code review, be selective. Only leave comments for issues that genuinely matter:

- Bugs, logic errors, or security concerns
- Bugs, correctness, logic errors, or security concerns
- **Component responsibilities** — Each component should do its job and *only* its job, per [`doc/code/framework.md`](../doc/code/framework.md). Flag responsibility bleed: e.g. an executor assembling prepended/system prompts or role-play framing (that's an attack technique), a converter or target making branching decisions (that's an attack/scorer), a scorer acting on its own result (the attack branches), or business logic living in memory/output. If logic belongs in a different brick, say so.
- Unclear code that would benefit from refactoring for readability
- Violations of the critical coding conventions above (async suffix, keyword-only args, type annotations)

Expand Down
2 changes: 2 additions & 0 deletions .github/instructions/attacks.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ applyTo: "pyrit/executor/attack/**"

`AttackStrategy` subclasses (single-turn attacks like `PromptSendingAttack`, multi-turn attacks like `RedTeamingAttack`, etc.) are pluggable bricks orchestrated by `AttackExecutor` and the `Scenario` framework. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here.

**Does not own** (see [framework.md](../../doc/code/framework.md)): packaging the attack. Prepended/system prompts, role-play framing, the converter stack, and dataset selection are passed in as configuration by the **attack technique** — an attack must accept them as parameters, not assemble them itself (e.g. `RolePlayAttack` building its own prompt scaffolding is attack-technique work bleeding into the executor). It also must not branch on raw responses (use a scorer), construct its own components (use the registry), or format/persist results itself (output/memory). Flag such bleed in review.

## Constructor contract

`AttackStrategy` subclasses MUST follow the keyword-only constructor shape:
Expand Down
4 changes: 4 additions & 0 deletions .github/instructions/converters.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ applyTo: "pyrit/prompt_converter/**"

# Prompt Converter Development Guidelines

**Responsibility**: A converter transforms a prompt into something else (rephrasing, encoding, translating to a Word document, overlaying text on an image, ...). Converters can be stacked and combined, and any converter may also be a NoOp.

**Does not own** (see [framework.md](../../doc/code/framework.md)): conversation state or attack decisions. A converter transforms input into output (and may call a target to do so); it must not branch on results, score, persist to memory itself, or decide when it runs — the attack/technique configures the stack. Flag such bleed in review.

## Base Class Contract

All converters MUST inherit from `PromptConverter` and implement:
Expand Down
4 changes: 4 additions & 0 deletions .github/instructions/datasets.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ applyTo: "pyrit/datasets/seed_datasets/**"

# Seed Dataset Loader Guidelines

**Responsibility**: Seed dataset loaders (`SeedDatasetProvider` subclasses) are the single place to manage the prompts/objectives for a source. They load seeds into `CentralMemory`; components then retrieve seeds from memory — components never read from a loader directly.

**Does not own** (see [framework.md](../../doc/code/framework.md)): a loader defines and holds seeds; it must not select or combine which seeds an attack uses (that's a scenario/attack technique) or render/parameterize prompts at send time (converters/normalizers). Flag such bleed in review.

These rules apply when adding or modifying loaders under `pyrit/datasets/seed_datasets/`.
Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here.

Expand Down
2 changes: 2 additions & 0 deletions .github/instructions/models.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ applyTo: "pyrit/models/**"

# `pyrit.models` Guidelines

**Responsibility**: `pyrit.models` is the lightweight, canonical data layer — the core types shared across components (and preferred in REST) so representations don't drift. It depends only on lightweight Python (the standard library and pydantic) and `pyrit.common`.

## Import Boundary

PyRIT enforces a two-layer rule for its foundational packages. `pyrit.common`
Expand Down
2 changes: 2 additions & 0 deletions .github/instructions/output.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ For full architecture documentation, usage examples, and extension guides, see [

This file covers the rules for **writing and reviewing** code in `pyrit/output/`.

**Does not own** (see [framework.md](../../doc/code/framework.md)): deciding *what* to render or *when*. Components hand results to output; format classes only turn data into strings and must never fetch data, touch `CentralMemory`, or call `print()` directly (that's isolated to leaf printer classes). Flag such bleed in review.

## Critical Rules

### Output goes through the sink — never call `print()` directly
Expand Down
2 changes: 2 additions & 0 deletions .github/instructions/scenarios.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ applyTo: "pyrit/scenario/**"

Scenarios orchestrate multi-attack security testing campaigns. Each scenario groups `AtomicAttack` instances and executes them sequentially against a target.

**Does not own** (see [framework.md](../../doc/code/framework.md)): the per-objective conversation logic. Branching, turn-by-turn adaptation, and scoring-based decisions belong to the attack — a scenario selects and packages existing attack techniques and owns parallelism/resiliency, not new attack algorithms or datasets. Flag such bleed in review.

## Base Class Contract

All scenarios inherit from `Scenario` (ABC) and must:
Expand Down
2 changes: 2 additions & 0 deletions .github/instructions/scorers.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ applyTo: "pyrit/score/**"

Scorers evaluate model responses against an objective and live under `pyrit/score/`. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here.

**Does not own** (see [framework.md](../../doc/code/framework.md)): acting on its own result. A scorer evaluates a response and returns a score; branching on that score is the attack's job and aggregating scores across runs is analytics'. It may call a target to evaluate, but must not send the attack's objective prompt or manage the conversation. Flag such bleed in review.

## Constructor contract

`Scorer` subclasses MUST use the keyword-only constructor shape:
Expand Down
4 changes: 4 additions & 0 deletions .github/instructions/targets.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ applyTo: "pyrit/prompt_target/**"

# Prompt Target Development Guidelines

**Responsibility**: A prompt target is "the thing we're sending the prompt to" — often an LLM, but it can be any endpoint (e.g. a storage account for cross-domain prompt injection). Targets use `message_normalizer` together with `TargetConfiguration` to transform `Message`s into the format the target supports.

**Does not own** (see [framework.md](../../doc/code/framework.md)): what to send or what to do with the response. A target sends a prepared `Message` and returns a response; it must not convert prompts (converters), score (scorers), or manage the conversation / decide the next turn (attacks). Flag such bleed in review.

## Base Class Contract

All targets MUST inherit from ``PromptTarget`` (or one of its public
Expand Down
Loading
Loading