diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 57b8eb0e1f..6ea8ddea76 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -4,19 +4,16 @@ PyRIT (Python Risk Identification Tool for generative AI) is an open-source fram ## Architecture -PyRIT uses a modular pluggable-brick design. The main extensibility points are: +PyRIT uses a modular pluggable-brick design. -- **Prompt Converters** (`pyrit/prompt_converter/`) — Transform prompts (70+ implementations). Base: `PromptConverter`. -- **Scorers** (`pyrit/score/`) — Evaluate responses. Base: `Scorer`. -- **Prompt Targets** (`pyrit/prompt_target/`) — Send prompts to LLMs/APIs. Base: `PromptTarget`. -- **Executors / Scenarios** (`pyrit/executor/`, `pyrit/scenario/`) — Orchestrate multi-turn attacks. -- **Memory** (`pyrit/memory/`) — `CentralMemory` for prompt/response persistence. +**[`doc/code/framework.md`](../doc/code/framework.md) is the canonical reference for how these pieces fit together.** It defines each component's responsibilities — what it owns and, critically, what it *does not* own — and how scenarios, attack techniques, executors, and the core/shared layers relate. Read it before adding or reviewing components so new code lands in the right place. ## Code Review Guidelines When performing a code review, be selective. Only leave comments for issues that genuinely matter: -- Bugs, logic errors, or security concerns +- Bugs, correctness, logic errors, or security concerns +- **Component responsibilities** — Each component should do its job and *only* its job, per [`doc/code/framework.md`](../doc/code/framework.md). Flag responsibility bleed: e.g. an executor assembling prepended/system prompts or role-play framing (that's an attack technique), a converter or target making branching decisions (that's an attack/scorer), a scorer acting on its own result (the attack branches), or business logic living in memory/output. If logic belongs in a different brick, say so. - Unclear code that would benefit from refactoring for readability - Violations of the critical coding conventions above (async suffix, keyword-only args, type annotations) diff --git a/.github/instructions/attacks.instructions.md b/.github/instructions/attacks.instructions.md index 2d5c4d7c96..0a5830fadc 100644 --- a/.github/instructions/attacks.instructions.md +++ b/.github/instructions/attacks.instructions.md @@ -6,6 +6,8 @@ applyTo: "pyrit/executor/attack/**" `AttackStrategy` subclasses (single-turn attacks like `PromptSendingAttack`, multi-turn attacks like `RedTeamingAttack`, etc.) are pluggable bricks orchestrated by `AttackExecutor` and the `Scenario` framework. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here. +**Does not own** (see [framework.md](../../doc/code/framework.md)): packaging the attack. Prepended/system prompts, role-play framing, the converter stack, and dataset selection are passed in as configuration by the **attack technique** — an attack must accept them as parameters, not assemble them itself (e.g. `RolePlayAttack` building its own prompt scaffolding is attack-technique work bleeding into the executor). It also must not branch on raw responses (use a scorer), construct its own components (use the registry), or format/persist results itself (output/memory). Flag such bleed in review. + ## Constructor contract `AttackStrategy` subclasses MUST follow the keyword-only constructor shape: diff --git a/.github/instructions/converters.instructions.md b/.github/instructions/converters.instructions.md index f409395e01..f16a67df13 100644 --- a/.github/instructions/converters.instructions.md +++ b/.github/instructions/converters.instructions.md @@ -4,6 +4,10 @@ applyTo: "pyrit/prompt_converter/**" # Prompt Converter Development Guidelines +**Responsibility**: A converter transforms a prompt into something else (rephrasing, encoding, translating to a Word document, overlaying text on an image, ...). Converters can be stacked and combined, and any converter may also be a NoOp. + +**Does not own** (see [framework.md](../../doc/code/framework.md)): conversation state or attack decisions. A converter transforms input into output (and may call a target to do so); it must not branch on results, score, persist to memory itself, or decide when it runs — the attack/technique configures the stack. Flag such bleed in review. + ## Base Class Contract All converters MUST inherit from `PromptConverter` and implement: diff --git a/.github/instructions/datasets.instructions.md b/.github/instructions/datasets.instructions.md index 4e2bbf8002..bf986b72b4 100644 --- a/.github/instructions/datasets.instructions.md +++ b/.github/instructions/datasets.instructions.md @@ -4,6 +4,10 @@ applyTo: "pyrit/datasets/seed_datasets/**" # Seed Dataset Loader Guidelines +**Responsibility**: Seed dataset loaders (`SeedDatasetProvider` subclasses) are the single place to manage the prompts/objectives for a source. They load seeds into `CentralMemory`; components then retrieve seeds from memory — components never read from a loader directly. + +**Does not own** (see [framework.md](../../doc/code/framework.md)): a loader defines and holds seeds; it must not select or combine which seeds an attack uses (that's a scenario/attack technique) or render/parameterize prompts at send time (converters/normalizers). Flag such bleed in review. + These rules apply when adding or modifying loaders under `pyrit/datasets/seed_datasets/`. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here. diff --git a/.github/instructions/models.instructions.md b/.github/instructions/models.instructions.md index 4a9e32baa0..795e0a1bbe 100644 --- a/.github/instructions/models.instructions.md +++ b/.github/instructions/models.instructions.md @@ -4,6 +4,8 @@ applyTo: "pyrit/models/**" # `pyrit.models` Guidelines +**Responsibility**: `pyrit.models` is the lightweight, canonical data layer — the core types shared across components (and preferred in REST) so representations don't drift. It depends only on lightweight Python (the standard library and pydantic) and `pyrit.common`. + ## Import Boundary PyRIT enforces a two-layer rule for its foundational packages. `pyrit.common` diff --git a/.github/instructions/output.instructions.md b/.github/instructions/output.instructions.md index d099c65a44..05f88fddec 100644 --- a/.github/instructions/output.instructions.md +++ b/.github/instructions/output.instructions.md @@ -8,6 +8,8 @@ For full architecture documentation, usage examples, and extension guides, see [ This file covers the rules for **writing and reviewing** code in `pyrit/output/`. +**Does not own** (see [framework.md](../../doc/code/framework.md)): deciding *what* to render or *when*. Components hand results to output; format classes only turn data into strings and must never fetch data, touch `CentralMemory`, or call `print()` directly (that's isolated to leaf printer classes). Flag such bleed in review. + ## Critical Rules ### Output goes through the sink — never call `print()` directly diff --git a/.github/instructions/scenarios.instructions.md b/.github/instructions/scenarios.instructions.md index 40fb4150b8..799bf6b508 100644 --- a/.github/instructions/scenarios.instructions.md +++ b/.github/instructions/scenarios.instructions.md @@ -6,6 +6,8 @@ applyTo: "pyrit/scenario/**" Scenarios orchestrate multi-attack security testing campaigns. Each scenario groups `AtomicAttack` instances and executes them sequentially against a target. +**Does not own** (see [framework.md](../../doc/code/framework.md)): the per-objective conversation logic. Branching, turn-by-turn adaptation, and scoring-based decisions belong to the attack — a scenario selects and packages existing attack techniques and owns parallelism/resiliency, not new attack algorithms or datasets. Flag such bleed in review. + ## Base Class Contract All scenarios inherit from `Scenario` (ABC) and must: diff --git a/.github/instructions/scorers.instructions.md b/.github/instructions/scorers.instructions.md index b4200704e0..20ce895f8a 100644 --- a/.github/instructions/scorers.instructions.md +++ b/.github/instructions/scorers.instructions.md @@ -6,6 +6,8 @@ applyTo: "pyrit/score/**" Scorers evaluate model responses against an objective and live under `pyrit/score/`. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here. +**Does not own** (see [framework.md](../../doc/code/framework.md)): acting on its own result. A scorer evaluates a response and returns a score; branching on that score is the attack's job and aggregating scores across runs is analytics'. It may call a target to evaluate, but must not send the attack's objective prompt or manage the conversation. Flag such bleed in review. + ## Constructor contract `Scorer` subclasses MUST use the keyword-only constructor shape: diff --git a/.github/instructions/targets.instructions.md b/.github/instructions/targets.instructions.md index 19040be72b..5439f616a4 100644 --- a/.github/instructions/targets.instructions.md +++ b/.github/instructions/targets.instructions.md @@ -4,6 +4,10 @@ applyTo: "pyrit/prompt_target/**" # Prompt Target Development Guidelines +**Responsibility**: A prompt target is "the thing we're sending the prompt to" — often an LLM, but it can be any endpoint (e.g. a storage account for cross-domain prompt injection). Targets use `message_normalizer` together with `TargetConfiguration` to transform `Message`s into the format the target supports. + +**Does not own** (see [framework.md](../../doc/code/framework.md)): what to send or what to do with the response. A target sends a prepared `Message` and returns a response; it must not convert prompts (converters), score (scorers), or manage the conversation / decide the next turn (attacks). Flag such bleed in review. + ## Base Class Contract All targets MUST inherit from ``PromptTarget`` (or one of its public diff --git a/doc/code/framework.md b/doc/code/framework.md index 6425557ffa..3f646e4138 100644 --- a/doc/code/framework.md +++ b/doc/code/framework.md @@ -10,14 +10,19 @@ Learn how to use PyRIT's components to build red teaming workflows. Load, create, and manage seed datasets for red teaming campaigns. :::: -::::{card} ⚔️ Attacks & Executors -:link: ./executor/0_executor -Run single-turn and multi-turn attacks — Crescendo, TAP, Skeleton Key, and more. +::::{card} 📋 Scenarios +:link: ./scenarios/0_scenarios +Run standardized evaluation scenarios at scale across harm categories. :::: -::::{card} 🔌 Targets -:link: ./targets/0_prompt_targets -Connect to OpenAI, Azure, Anthropic, HuggingFace, HTTP endpoints, and custom targets. +::::{card} 🧩 Attack Techniques +:link: ./scenarios/0_attack_techniques +Package an executor, converters, datasets, and strategy into a single named attack. +:::: + +::::{card} ⚔️ Executors and Attacks +:link: ./executor/0_executor +Run single-turn and multi-turn attacks — Crescendo, TAP, Skeleton Key, and more. :::: ::::{card} 🔄 Converters @@ -25,107 +30,344 @@ Connect to OpenAI, Azure, Anthropic, HuggingFace, HTTP endpoints, and custom tar Transform prompts with text, audio, image, and video converters. :::: -::::{card} 📊 Scoring +::::{card} 🔌 Targets +:link: ./targets/0_prompt_targets +Connect to OpenAI, Azure, Anthropic, HuggingFace, HTTP endpoints, and custom targets. +:::: + +::::{card} 📊 Scorers :link: ./scoring/0_scoring Evaluate AI responses with true/false, Likert, classification, and custom scorers. :::: -::::{card} 💾 Memory -:link: ./memory/0_memory -Track conversations, scores, and attack results with SQLite or Azure SQL. -:::: +::::: -::::{card} ⚙️ Setup & Configuration -:link: ./setup/0_setup -Initialize PyRIT, configure defaults, and manage resiliency settings. -:::: +--- -::::{card} 📋 Scenarios -:link: ./scenarios/0_scenarios -Run standardized evaluation scenarios at scale across harm categories. -:::: +The sections above link to detailed guides for each component. The architecture below explains how the pieces fit together — it's primarily aimed at contributors. -::::{card} 🗂️ Registry -:link: ./registry/0_registry -Register and discover targets, scorers, and converters via class and instance registries. -:::: +# Architecture -::::{card} 🖨️ Output -:link: ./output/0_output -Render attack results, scenario results, conversations, and scores to terminal, files, or Jupyter. -:::: +The main components of PyRIT are seeds, scenarios, attack techniques, executors and attacks, converters, targets, and scorers. The best way to contribute to PyRIT is by contributing to one of these components. -::::: +The diagram below shows how the pieces fit together: entry points run **scenarios**, which package **datasets** with **attack techniques**; each technique drives an **attack/executor** that orchestrates **converters**, **targets**, and **scorers**; and a shared library layer (**memory**, **registry**, **models**, **output**, and more) supports all of them. + +```mermaid +flowchart TB + subgraph entry [Entry points] + direction LR + CLI[Scanner / CLI] + GUI[GUI / Backend] + FW[Framework / Notebooks] + end ---- + SCEN["Scenario
packages datasets + attack techniques;
orchestrates parallelism & resiliency"] + TECH["Attack Technique
executor + converters + datasets + strategy"] + ATK["Attack / Executor
manages the conversation to reach an objective"] -The sections above link to detailed guides for each component. The architecture below explains how the pieces fit together — it's primarily aimed at contributors. + subgraph core [Core Components] + direction LR + CONV[Converters] + SCORE[Scorers] + TGT[Targets] + DATA[(Datasets / Seeds)] + end -# Architecture + subgraph lib [Shared Library] + direction LR + MEM[(Memory)] + REG[Registry] + MODEL[Models] + OUT[Output] + end + + entry --> SCEN + DATA --> SCEN + SCEN --> TECH --> ATK + + ATK --> CONV + ATK == objective ==> TGT + ATK -. adversarial .-> TGT + ATK -- decisions based on --> SCORE + CONV -. may call .-> TGT + SCORE -. may call .-> TGT + + REG -. builds .-> core + REG -. builds .-> ATK + ATK <-- reads / writes --> MEM + core <--> MEM + entry --> OUT + MEM -- reads --> OUT + MODEL -. shared types .-> core + + classDef flow fill:#e8f0fe,stroke:#4285f4,color:#15233a; + classDef libnode fill:#f1f3f4,stroke:#9aa0a6,color:#202124; + class SCEN,TECH,ATK flow; + class MEM,REG,MODEL,OUT libnode; +``` + +The orchestration layers **nest from broadest to narrowest** — each owns less than the layer above it: + +- **Scenario** packages many attack techniques and owns parallelism and resiliency. +- **Attack Technique** configures one executor with its converters, seeds, scorers, and strategy. +- **Executor / Attack** runs the algorithm: sends to targets, applies converters, and branches on scorers. + + +```mermaid +flowchart TB + subgraph SCEN["Scenario — owns parallelism & resiliency"] + subgraph TECH["Attack Technique — executor + converters + seeds + scorers + strategy"] + subgraph ATK["Executor / Attack — drives the conversation"] + LEAF["sends to targets · applies converters · branches on scorers"] + end + end + end +``` + + + +# Core Components + +As much as possible, each core component is a pluggable brick of functionality. Prompts from one attack can be used in another. An attack for one scenario can use multiple targets. And sometimes you completely skip components (e.g. almost every component can be a NoOp also, you can have a NoOp converter that doesn't convert, or a NoOp target that just prints the prompts). + +If you are contributing to PyRIT, that work will most likely land in one of the core components buckets and be as self-contained as possible. It isn't always this clean, but when an attack scenario doesn't quite fit (and that's okay!) it's good to brainstorm with the maintainers about how we can modify our architecture. Also, please open issues if you see anything under Framework Plans you do/don't want. + + +## [Datasets](./datasets/0_dataset) + +**Responsibility**: Create a single place to manage seeds + +- New Datasets can be added in the dataset module. +- Datasets should never be retrieved from SeedDatasetProviders; SeedDatasetProviders should load into memory, and then components retrieve from memory +- Most components should always work with seeds passed directly in (except scenarios which may package them from memory). Never use SeedDatasetProviders, file paths, etc. Either pass the seed as an argument or retrieve from memory. +- There is a Seed hierarchy and the right types should be used (SeedObjective, SeedPrompt, SimulatedSeedPrompt, SeedAttackGroup, ...) +- **Does not own**: a dataset defines and holds seeds; it doesn't package them for an attack. Specifically not: + - selecting or combining which seeds an attack uses (that's a scenario / attack technique) + - rendering or parameterizing prompts at send time (converters / normalizers) + - runtime retrieval from providers or filepaths (load into memory first) + +**Framework Plans**: + +- There is some churn here. We haven't managed these much at scale, and we may have to redefine how it works. +- We want more investment in managing datasets and loading them more intelligently +- We need to more consistently pass seeds/use memory (e.g. not using filepaths) +- We need to create seed types for different executors (e.g. SeedExpectedResponse, SeedBenchmarkGroup) + +**Contributing (difficulty easy)**: Are there more prompts and jailbreak templates you can add that include scenarios you're testing for? It is easy to add new dataset providers. + +## [Scenarios](./scenarios/0_scenarios) + +**Responsibility**: This is the avenue to "run PyRIT against something". What does that look like? + +- A scenario takes user input and uses it to package datasets with attack techniques +- A scenario orchestrates resiliency and parallelism from a high level +- No result should depend on previous results (that is an attack's job) +- **Does not own**: the per-objective conversation logic. Branching, turn-by-turn adaptation, and scoring-based decisions belong to the attack; a scenario selects and packages existing attack techniques rather than defining new attack algorithms or datasets. + +**Framework Plans**: + +- Scenarios are new enough that we are still discovering patterns and limitations. So they will regularly be refactored + +**Contributing (difficulty medium)**: Is there a scanner that does something PyRIT doesn't? Add it as a scenario. But because we're changing how things are done rapidly, it is not as well-defined as other areas. + +## [Attack Techniques](./scenarios/0_attack_techniques) + +**Responsibility**: An attack technique packages an executor, converters, datasets, and strategies into a single attack. The goal is that any attack (something trying to achieve an objective) can be defined as an attack technique. + +- **Does not own**: the conversation algorithm itself. Branching, turn management, and scoring decisions live in the executor it wraps — a technique only selects and configures existing components, and shouldn't implement new sending, scoring, or branching logic. + +**Framework Plans**: + +- Managing these better, so scenarios can more easily select or build the attack techniques to use + +**Contributing (difficulty easy)**: Simply add the attack technique to one of the initializers. + +## [Executors and Attacks](./executor/0_executor) + +**Executor Responsibility**: Manage conversations between objective targets and adversarial targets; using datasets, scorers, and converters. + +**Attack Responsibility**: An attack is a type of executor, which manages conversations to achieve an objective. + +- Any branching decision (e.g. the next thing(s) to do is based on a previous result) should be an attack/executor. +- Executors should always make use of other component's responsibilities. An executor should always branch based on a scorer and NOT a direct response. (e.g. was this prompt blocked? is a scorer responsibility, not an executor responsibility) +- Executors should use scoring and target capabilities implicitly. Executors should support multi-modal. +- Compound attacks are possible, combining different attacks in different ways. +- **Does not own**: packaging the attack. Those are passed in as configuration by the **attack technique**, not assembled here: + - prepended / system prompts, role-play framing, the converter stack, or dataset selection (e.g. `RolePlayAttack` building its own prompt scaffolding is attack-technique work bleeding into the executor) + - branching on raw responses (use a scorer), constructing its own components (use the registry), or formatting / persisting results (output / memory) + +**Framework Plans**: + +- We need to move some older attacks that don't belong here. Many (FlipAttack) should just be attack techniques +- There are potential ways we could combine different algorithms. Are Crescendo and TAP ultimately the same? +- We need to support target capabilities more implicitly +- Other executors, like benchmarks, need better end-to-end support; potentially including an `ExpectedResult` seed and associated scorers. +- More flexible compound attacks should continue to be added + +**Contributing (difficulty high)**: The best way to contribute is likely opening issues if you run into limitations. + +## [Converters](./converters/0_converters) + +**Responsibility**: Converters are a component that converts prompts to something else. They can be stacked and combined. They can be as varied as translating a text prompt into a Word document, rephrasing a prompt, or adding a text overlay to an image. + +- **Does not own**: conversation state or attack decisions. A converter transforms input into output (and may call a target to do so), but it doesn't branch on results, score, persist to memory itself (the normalizer handles persistence), or decide when it runs — the attack/technique configures the stack. + +**Framework Plans**: + +- We want to refactor our converter pipeline, so there are currently some things that should be converters that we may want to postpone (e.g. partial converting). This is supported but could be much more dynamic. + +**Contributing (difficulty low)**: The existing pattern is well-defined. Are there ways prompts can be converted that would be useful for an attack? + +## [Targets](./targets/0_prompt_targets) + +**Responsibility**: A target can be thought of as "the thing we're sending the prompt to". Many other components use it, including scorers, attacks, and converters. + +- This is often an LLM, but it doesn't have to be. For Cross-Domain Prompt Injection Attacks, the target might be a storage account that a later target has a reference to. Message and conversation should be generic enough to handle this extra data. +- Target capabilities should be used to see if a target is compatible with the capabilities that the other components want to use. +- Targets should use message_normalizer along with TargetConfiguration to transform `Messages` into formats that target supports. +- Because targets are so varied, it is reasonable to return multiple tool calls, or none at all. +- One attack can have many targets (and in fact, converters and scorers can also use targets to convert/score the prompt). +- **Does not own**: what to send or what to do with the response. A target sends a prepared `Message` and returns a response — it doesn't convert prompts (converters), score (scorers), manage the conversation or decide the next turn (attacks), or apply attack logic. Its retries stay at the target layer (e.g. `RateLimitException`). + +**Framework Plans**: + +- Better agent support may require extra pieces attached to a Message +- Better surface support may require expanding the return types + +**Contributing (difficulty low)**: + +- The pattern is well-defined. +- Are there models you want to use at any stage or for different attacks? But also, can your model just be one of the existing targets? + +## [Scorers](./scoring/0_scoring) + +**Responsibility**: Scorers give feedback to the attack on what happened with the prompt. This could be as simple as "Was this prompt blocked?" or "Was our objective achieved?" + +- Any decision an attack makes should be based on a scorer result +- A scorer is not limited to a prompt, it could be anything (e.g. was this tool called or was this file written). +- **Does not own**: acting on its own result. A scorer evaluates a response and returns a score; branching on that score is the attack's job, and aggregating scores across runs is analytics'. It may call a target to evaluate, but it doesn't send the attack's objective prompt or manage the conversation. + +**Framework Plans**: + +- Scorers will be refactored to be more generic, so they can determine more general results (does a file exist? Was a tool called?) + +**Contributing (difficulty low)**: + +- The pattern is well-defined. +- You can evaluate how accurate probabilistic scorers are and likely make them more accurate. +- Is there data you want to use to make decisions or analyze? + +# Shared Library + +The below talks about responsibilities of most modules in the PyRIT library + +## Analytics + +**Responsibility**: Make sense of results — aggregating across conversations and attacks to answer questions PyRIT itself acts on or reports. + +- This is where cross-run analysis belongs: e.g. "which attack performed best for this objective?", "how often did a technique succeed?", or "which responses match known content?". +- **Does not own**: live, in-attack decisions — any decision made *during* an attack is a scorer's job. Analytics only operates on stored results, after the fact. +- Today it includes `ConversationAnalytics` (inspecting conversation history), `analyze_results` / `AttackStats` (aggregating outcomes across techniques), and text-matching strategies (`ExactTextMatching`, `ApproximateTextMatching`). + +## Auth + +**Responsibility**: Provide authentication helpers for the external services PyRIT talks to, behind a common `Authenticator` abstraction. + +- Components that need credentials should go through these helpers rather than handling tokens themselves. + +## [Exceptions](../contributing/9_exception) + +**Responsibility**: Define PyRIT's exception hierarchy and the retry behavior built around it. + +- Retries should use PyRIT exception types (such as `PyritException`, `BadRequestException`, `RateLimitException`, `EmptyResponseException`, and `InvalidJsonException`) and retry decorators (such as `pyrit_target_retry`, `pyrit_json_retry`, `pyrit_placeholder_retry`) and execution-context utilities (`ExecutionContext`, `ComponentRole`, `RetryCollector`). +- Retries should _only_ be attempted on known exceptions. +- The applicable layer should retry exceptions (e.g. only targets should retry `RateLimitException`, only scorers/attacks/converters should retry `InvalidJsonException`, and only scenarios should retry general exceptions). +- When raising, attach context: every `PyritException` carries a `status_code` and a human-readable `message`, and the active `ExecutionContext` / `ComponentRole` records which component raised it — so failures point back to where they happened. -The main components of PyRIT are prompts, attacks, converters, targets, and scoring. The best way to contribute to PyRIT is by contributing to one of these components. +## [Memory](./memory/0_memory) -![alt text](../../assets/architecture_components.png) +**Responsibility**: Memory persists and retrieves the data that flows between components — prompts, responses, conversations, scores, and attack results — so components stay swappable while still sharing the context they need. -As much as possible, each component is a pluggable brick of functionality. Prompts from one attack can be used in another. An attack for one scenario can use multiple targets. And sometimes you completely skip components (e.g. almost every component can be a NoOp also, you can have a NoOp converter that doesn't convert, or a NoOp target that just prints the prompts). +- One important thing to remember about this architecture is its swappable nature. Seeds, targets, converters, attacks, and scorers should all be swappable. But sometimes one of these components needs additional information. If the target is an LLM, we need a way to look up previous messages sent to that session so we can properly construct the new message. If the target is a blob store, we need to know the URL to use for a future attack. +- Components should access memory through `CentralMemory` rather than passing state directly between each other. +- Memory backends are swappable too (e.g. SQLite or Azure SQL) without changing the components that use them. +- **Does not own**: business logic or decisions. Memory stores and retrieves state; it doesn't decide what to send, how to score, or when to branch — components do that and persist results here. -If you are contributing to PyRIT, that work will most likely land in one of these buckets and be as self-contained as possible. It isn't always this clean, but when an attack scenario doesn't quite fit (and that's okay!) it's good to brainstorm with the maintainers about how we can modify our architecture. +## [Models](../contributing/11_memory_models) -The remainder of this document talks about the different components, how they work, what their responsibilities are, and ways to contribute. +**Responsibility**: pyrit.models is a lightweight module where core types are defined. These should always be used where possible to prevent drift. +- If you are creating a class that has a lot of overlap with another class, or using a dict to serialize across boundaries, consider if you can use/move pyrit.models +- Models includes `identifiers` which are descriptions of the core components. And along with the registry, can often recreate those components. +- Models includes types passed around between components, and should be prefered in REST +- models should never depend on anything except lightweight Python (the standard library and pydantic) and pyrit.common -## Datasets: Prompts, Jailbreak Templates, Source Images, Attack Strategies, etc. +## [Normalizers](./targets/11_message_normalizer) -The first piece of an attack is often a dataset piece, like a prompt. "Tell me how to create a Molotov cocktail" is an example of a prompt. PyRIT is a good place to have a library of things to check for. +**Responsibility**: Reshape prompts and conversations so components and targets can interoperate. There are two distinct modules: -Ways to contribute: Check out our documentation on [seed datasets](./datasets/0_dataset.md); are there more prompts and jailbreak templates you can add that include scenarios you're testing for? +- **`prompt_normalizer`** applies converters and dispatches individual prompts to a `PromptTarget` (handling batching and memory persistence). `NormalizerRequest` and `PromptConverterConfiguration` describe what to send and which converters to apply. +- **`message_normalizer`** reshapes multi-message conversation payloads into the structure a given model expects — for example, handling system-message behavior (keep / squash / ignore), history squashing, and tokenizer chat templates. -## Attacks +## [Output](./output/0_output) -Attacks are responsible for putting all the other pieces together. They make use of all other components in PyRIT to execute an attack technique end-to-end. -PyRIT supports single-turn (e.g. Many Shot Jailbreaks [@anthropic2024manyshot], Role Play, Skeleton Key [@microsoft2024skeletonkey]) and multi-turn attack strategies (e.g. Tree of Attacks [@mehrotra2023tap], Crescendo [@russinovich2024crescendo]), and compound strategies (e.g. `SequentialAttack`) for chaining several techniques against a single objective. +**Responsibility**: The Output module is responsible for writing different components in different formats to different places. -Ways to contribute: Check out our [attack docs](./executor/0_executor.md). There are hundreds of attacks outlined in research papers. A lot of these can be captured within PyRIT. If you find an attack that doesn't fit the attack model please notify the team. Are there scenarios you can write attack modules for? +- It renders the core result types — attack results, scenario results, conversations, and scores — without those components needing to know how they are displayed. +- Format and destination are decoupled: a **format** (e.g. pretty ANSI, Markdown, JSON) is separate from a **sink** (stdout, file, Jupyter), so any result can be rendered any way to anywhere. +- **Does not own**: deciding *what* to render or *when*. Components hand results to output; format classes only turn data into strings and never fetch data, touch `CentralMemory`, or call `print()` directly (that's isolated to leaf printer classes). -## Converters +**Contributing (difficulty low)**: Adding a new format or sink is well-defined. Every new domain printer should come with a matching convenience function in `helpers.py`. -Converters are a powerful component that converts prompts to something else. They can be stacked and combined. They can be as varied as translating a text prompt into a Word document, rephrasing a prompt in 100 different ways, or adding a text overlay to an image. +## [Registry](./registry/0_registry) -Ways to contribute: Check out our [converter docs](./converters/0_converters.ipynb). Are there ways prompts can be converted that would be useful for an attack? +**Responsibility**: The registry is used to build and store the core components. -## Target +- If you are creating a component with user input (e.g. via config, REST, or automatically) it should always use the registry +- If you are storing an instance of a component, it should always use the registry -A Prompt Target can be thought of as "the thing we're sending the prompt to". +## [Setup](./setup/0_setup) -This is often an LLM, but it doesn't have to be. For Cross-Domain Prompt Injection Attacks, the Prompt Target might be a Storage Account that a later Prompt Target has a reference to. +**Responsibility**: Bootstrap a PyRIT session — getting memory, defaults, and components configured so the rest of the framework can run. -One attack can have many Prompt Targets (and in fact, converters and Scoring Engine can also use Prompt Targets to convert/score the prompt). +- `initialize_pyrit_async` is the entry point: it sets up the environment and a memory backend (`IN_MEMORY` / `SQLITE` / `AZURE_SQL` via `MemoryDatabaseType`) and runs any initializers to configure global defaults and components. +- Configuration files are the core way to drive setup. `ConfigurationLoader` / `initialize_from_config_async` read a config that declares the memory backend and a list of initializers to run, so a session can be reproduced without code. +- By default these files live under the PyRIT home directory `~/.pyrit/`: the config file at `~/.pyrit/.pyrit_conf`, and environment variables from `~/.pyrit/.env` and `~/.pyrit/.env.local` (loaded if present). +- A `PyRITInitializer` is a class-based unit of configuration: each one configures part of PyRIT (e.g. registering targets, scorers, scenario techniques, or loading default datasets) and runs in the order provided. Built-in initializers live in the `initializers/` package. +- Users can bring their own: subclass `PyRITInitializer`, implement `initialize_async`, and reference it from config or pass it in — letting teams package their own defaults and components. -Ways to contribute: Check out our [target docs](./targets/0_prompt_targets.md). Are there models you want to use at any stage or for different attacks? +# Application surfaces +The below describes the user-facing surfaces built on top of the framework. -## Scoring Engine +## Backend -The scoring engine is a component that gives feedback to the attack on what happened with the prompt. This could be as simple as "Was this prompt blocked?" or "Was our objective achieved?" +**Responsibility**: Expose PyRIT functionality as a FastAPI REST API consumed by the CLI and frontend. -Ways to contribute: Check out our [scoring docs](./scoring/0_scoring.ipynb). Is there data you want to use to make decisions or analyze? +- Surfaces targets, scenarios, and health/version endpoints; served via `uvicorn` with Swagger/ReDoc docs. +- Wherever possible it should reuse other components rather than reimplementing them (e.g. the registry to build components, `pyrit.models` for its model layer), while adding presentation-specific information on top as needed. +- Organized into `routes/`, `services/`, `models/`, `mappers/`, and `middleware/`, and launched through the `pyrit_backend` command (configurable via `PYRIT_API_HOST` / `PYRIT_API_PORT` / `PYRIT_API_RELOAD`). -## Memory +## [CLI](../scanner/0_scanner) -One important thing to remember about this architecture is its swappable nature. Prompts and targets and converters and attacks and scorers should all be swappable. But sometimes one of these components needs additional information. If the target is an LLM, we need a way to look up previous messages sent to that session so we can properly construct the new message. If the target is a blob store, we need to know the URL to use for a future attack. +**Responsibility**: Offer command-line entry points into PyRIT as a thin REST client over the backend, deliberately avoiding heavy `pyrit` imports. -For more details about memory configuration, please follow the guide in [memory](./memory/0_memory.md). +- Because it talks to the backend over HTTP, the CLI stays lightweight and starts quickly. +- It should not rely on pyrit other than pyrit.models, pyrit.common, and pyrit.output. -Memory modifications and contributions should usually be designed with the maintainers. +## [Documentation](../contributing/7_notebooks) -## The Flow +**Responsibility**: Show how PyRIT is used, concisely and runnably, across all the ways someone might pick it up. -To some extent, the ordering in this diagram matters. In the simplest cases, you have a prompt, an attack takes the prompt, uses prompt normalizer to run it through converters and send to a target, and the result is scored. +PyRIT can be used in three modes ([Scanner](../scanner/0_scanner), [GUI](../gui/0_gui), and [Framework](#core-components)), and the documentation is organized to match: -But this simple view is complicated by the fact that an attack can have multiple targets, converters can be stacked, scorers can use targets to score, etc. +- Notebooks that contain code should be notebooks that can execute. +- Notebooks should execute quickly (within a couple minutes). +- The percent-format `.py` files and their paired `.ipynb` notebooks must be kept in sync. -Sometimes, if a scenario requires specific data, we may need to modify the architecture. This happened recently when we thought a single target may take multiple prompts separately in a single request. Any time we need to modify the architecture like this, that's something that needs to be designed with the maintainers so we can consolidate our other supported scenarios and future plans. +## [Frontend](../gui/0_gui) -## Notebooks +**Responsibility**: Provide CoPyRIT, the graphical interface for human-led red teaming, by talking to the backend REST API. -For all their power, attacks should still be generic. A lot of our front-end code and operators use Notebooks to interact with PyRIT. This is fantastic, but most new logic should not be notebooks. Notebooks should mostly be used for attack setup and documentation. For example, configuring the components and putting them together is a good use of a notebook, but new logic for an attack should be moved to one or more components. +- A TypeScript + React single-page app built with Vite and Fluent UI. +- Dev workflow via `dev.py` / npm scripts orchestrates both servers together; tested with Jest (unit) and Playwright (e2e).