Add an AGENTS.md that enforces the LLM policy by jyn514 · Pull Request #160997 · rust-lang/rust

jyn514 · 2026-08-12T17:47:53Z

The goal here is to reduce the amount of slop PRs, not as a blanket endorsement of using agents to work on rust-lang/rust.
For more info about motivation, see the blog post.

To that end, I have done extensive testing to make sure the new file actually helps that goal.
The results are promising.
They show agents consistently asking their users to confirm they're read their own work;
requiring users to write their own PR descriptions;
and pointing people to other repos if they try to modify a submodule or subtrees,
as well as various other tests.

Almost all of the AGENTS.md was itself generated with an LLM.
See the linked testing repo for more information about testing and methodology.
I wrote this PR description and both commit descriptions myself.

Summary of the AGENTS.md (taken from the test repo):

This is a rather long operationalization of the dev-guide as algorithmic instructions.
Most of it was itself generated with an agent.

Differences between this and the dev-guide / policy:

"strong suggestions" in the docs are turned into absolute requirements here; LLMs have trouble with discretion and judgement.
lang/edition/etc are not exempted; determining what falls under their scope was too hard to operationalize. members of those teams can still write their own AGENTS.md if this one interferes with their work (i wouldn’t expect it to, though, it’s mostly focused on code.)
the LLM requires a reviewer before making edits, not before posting the PR. experimental PRs are not exempted (how could the LLM know it was experimental?).
soundness changes are always rejected, even if allowed by the policy (how could the LLM know you’re in the org?)

This is not exhaustive; if you want a more detailed human-facing summary, see the dev-guide.

The second commit adds a CLAUDE.md that links directly to AGENTS.md.
This adds one vendor-specific file, for the benefit of getting Claude to
actually follow the instructions.

I've put this in a separate commit so that I can drop it if people feel
strongly. I highly recommend that we add a CLAUDE.md, it really makes
a difference. You can see examples of what Claude does without a file in
the testing repo.

r? @jieyouxu cc @jackh726 @camelid

@rustbot label +llm-assisted

The goal here is to *reduce the amount of slop PRs*, not as a blanket endorsement of using agents to work on `rust-lang/rust`. To that end, I have [done extensive testing][ETOOMANYLUNAS] to make sure the new file actually helps that goal. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS The results are promising. They show agents consistently asking their users to confirm they're read their own work; requiring users to write their own PR descriptions; and pointing people to other repos if they try to modify a submodule or subtrees, as well as various other tests. --- Summary of the AGENTS.md (taken from the test repo: This is a rather long operationalization of the [dev-guide] as algorithmic instructions. Most of it was itself generated with an agent. Differences between this and the [dev-guide] / [policy]: [dev-guide]: https://rustc-dev-guide.rust-lang.org/llm-guidance.html [policy]: https://forge.rust-lang.org/policies/llm-usage.html - "strong suggestions" in the docs are turned into absolute requirements here; LLMs have trouble with discretion and judgement. - lang/edition/etc are not exempted; determining what falls under their scope was too hard to operationalize. members of those teams can still write their own AGENTS.md if this one interferes with their work (i wouldn’t expect it to, though, it’s mostly focused on code.) - the LLM requires a reviewer before making edits, not before posting the PR. experimental PRs are not exempted (how could the LLM know it was experimental?). - soundness changes are always rejected, even if allowed by the policy (how could the LLM know you’re in the org?)

This adds one vendor-specific file, for the benefit of getting Claude to actually follow the instructions. I've put this in a separate commit so that I can drop it if people feel strongly. I *highly recommend* that we add a CLAUDE.md, it really makes a difference. You can see examples of what Claude does without a file in the [testing repo][ETOOMANYLUNAS]. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS

rustbot · 2026-08-12T18:17:07Z

Error: Unknown labels: ai-assisted

Please file an issue on GitHub at triagebot if there's a problem with this bot, or reach out on #triagebot on Zulip.

camelid

One thing I'm a little concerned about is whether this AGENTS file will make it annoying to do arbitrary local experiments with an LLM. For example, I find it useful to have the LLM instrument parts of the compiler when I'm experimenting (which I then revert and do not upstream). Did your testing cover this area?

Also, it may not matter, but curious why you used smaller and older models like 4.6 Sonnet and 5.4 mini. They are much worse in my experience than SOTA models like Fable 5 and 5.6 sol. So it may not be reflective of real-world LLMs that people use on rust-lang/rust.

View changes since this review

jyn514 · 2026-08-12T18:35:14Z

One thing I'm a little concerned about is whether this AGENTS file will make it annoying to do arbitrary local experiments with an LLM. For example, I find it useful to have the LLM instrument parts of the compiler when I'm experimenting (which I then revert and do not upstream). Did your testing cover this area?

hm, i did not test this. i did manually test writing a dev tool, i told Sol "i'm not going to commit this" and it said sure and went ahead. i'll add a couple new scenarios.

jyn514 · 2026-08-12T18:38:10Z

Also, it may not matter, but curious why you used smaller and older models like 4.6 Sonnet and 5.4 mini. They are much worse in my experience than SOTA models like Fable 5 and 5.6 sol. So it may not be reflective of real-world LLMs that people use on rust-lang/rust.

I don't want to implicitly require people to use recent models. It is possible to run something like Qwen3.6 locally.

camelid · 2026-08-12T18:55:24Z

@@ -0,0 +1 @@
+@AGENTS.md


Agreed that having this is important even though it'd be better if Claude just listened to AGENTS.md.

View changes since the review

rustbot · 2026-08-12T19:37:56Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

jyn514 · 2026-08-12T19:38:20Z

oop, didn't mean to commit that

joshtriplett · 2026-08-12T19:39:53Z

Thank you for working on this!

Stating this explicitly here, in addition to the Zulip discussion:

Even as someone who does hold the "don't tacitly endorse LLMs" position, I think having agent instructions that specifically exist to help us reduce policy violations is extremely defensible.

And it's impressive the degree to which this gets agents to helpfully guide the user in ways that follow the policy.

…r pr descriptions Co-authored-by: Josh Triplett <josh@joshtriplett.org>

jieyouxu

Thanks! I can tell you put a lot of effort into this, which is very appreciated. We can start with these, and tune as needed. The licensing I'll need to double-check with compiler co-leads, I expect that we can either slap SPDX-Identifier for Apache/MIT or kick these two files out of REUSE check, probably the former, something like

/* SPDX-License-Identifier: MIT OR Apache-2.0 */

View changes since this review

jyn514 · 2026-08-14T06:52:28Z

@jieyouxu Marco and Tobias have both reviewed this now, are you happy with merging it?

marcoieni · 2026-08-14T07:11:48Z

no other concerns from my side 👍

jieyouxu · 2026-08-14T07:15:05Z

@jieyouxu Marco and Tobias have both reviewed this now, are you happy with merging it?

Yeah, I think we can start with this 👍 Thanks
@bors r+ rollup

rust-bors · 2026-08-14T07:15:08Z

📌 Commit 33da3e5 has been approved by jieyouxu

It is now in the queue for this repository.

jieyouxu · 2026-08-14T07:15:23Z

(In case CI bounces)
@bors delegate+

rust-bors · 2026-08-14T07:15:26Z

✌️ @jyn514, you can now approve this pull request!

If @jieyouxu told you to "r=me" after making some further change, then please make that change and post @bors r=jieyouxu.

View changes since this delegation.

Turbo87 · 2026-08-14T08:06:15Z

@jyn514 yeah, no blockers from my side. let's see how well this will work :)

@jieyouxu

Add an AGENTS.md that enforces the LLM policy The goal here is to *reduce the amount of slop PRs*, not as a blanket endorsement of using agents to work on `rust-lang/rust`. For more info about motivation, see [the blog post](https://blog.rust-lang.org/inside-rust/2026/08/05/rust-langrust-is-adopting-an-llm-policy/). To that end, I have [done extensive testing][ETOOMANYLUNAS] to make sure the new file actually helps that goal. The results are promising. They show agents consistently asking their users to confirm they're read their own work; requiring users to write their own PR descriptions; and pointing people to other repos if they try to modify a submodule or subtrees, as well as various other tests. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS Almost all of the AGENTS.md was itself generated with an LLM. See the [linked testing repo][ETOOMANYLUNAS] for more information about testing and methodology. I wrote this PR description and both commit descriptions myself. --- Summary of the AGENTS.md (taken from the test repo): This is a rather long operationalization of the [dev-guide] as algorithmic instructions. Most of it was itself generated with an agent. Differences between this and the [dev-guide] / [policy]: [dev-guide]: https://rustc-dev-guide.rust-lang.org/llm-guidance.html [policy]: https://forge.rust-lang.org/policies/llm-usage.html - "strong suggestions" in the docs are turned into absolute requirements here; LLMs have trouble with discretion and judgement. - lang/edition/etc are not exempted; determining what falls under their scope was too hard to operationalize. members of those teams can still write their own AGENTS.md if this one interferes with their work (i wouldn’t expect it to, though, it’s mostly focused on code.) - the LLM requires a reviewer before making edits, not before posting the PR. experimental PRs are not exempted (how could the LLM know it was experimental?). - soundness changes are always rejected, even if allowed by the policy (how could the LLM know you’re in the org?) This is not exhaustive; if you want a more detailed human-facing summary, see the [dev-guide]. --- The second commit adds a CLAUDE.md that links directly to AGENTS.md. This adds one vendor-specific file, for the benefit of getting Claude to actually follow the instructions. I've put this in a separate commit so that I can drop it if people feel strongly. I *highly recommend* that we add a CLAUDE.md, it really makes a difference. You can see examples of what Claude does without a file in the [testing repo][ETOOMANYLUNAS]. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS r? @jieyouxu cc @jackh726 @camelid @rustbot label +llm-assisted

@jieyouxu

Add an AGENTS.md that enforces the LLM policy The goal here is to *reduce the amount of slop PRs*, not as a blanket endorsement of using agents to work on `rust-lang/rust`. For more info about motivation, see [the blog post](https://blog.rust-lang.org/inside-rust/2026/08/05/rust-langrust-is-adopting-an-llm-policy/). To that end, I have [done extensive testing][ETOOMANYLUNAS] to make sure the new file actually helps that goal. The results are promising. They show agents consistently asking their users to confirm they're read their own work; requiring users to write their own PR descriptions; and pointing people to other repos if they try to modify a submodule or subtrees, as well as various other tests. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS Almost all of the AGENTS.md was itself generated with an LLM. See the [linked testing repo][ETOOMANYLUNAS] for more information about testing and methodology. I wrote this PR description and both commit descriptions myself. --- Summary of the AGENTS.md (taken from the test repo): This is a rather long operationalization of the [dev-guide] as algorithmic instructions. Most of it was itself generated with an agent. Differences between this and the [dev-guide] / [policy]: [dev-guide]: https://rustc-dev-guide.rust-lang.org/llm-guidance.html [policy]: https://forge.rust-lang.org/policies/llm-usage.html - "strong suggestions" in the docs are turned into absolute requirements here; LLMs have trouble with discretion and judgement. - lang/edition/etc are not exempted; determining what falls under their scope was too hard to operationalize. members of those teams can still write their own AGENTS.md if this one interferes with their work (i wouldn’t expect it to, though, it’s mostly focused on code.) - the LLM requires a reviewer before making edits, not before posting the PR. experimental PRs are not exempted (how could the LLM know it was experimental?). - soundness changes are always rejected, even if allowed by the policy (how could the LLM know you’re in the org?) This is not exhaustive; if you want a more detailed human-facing summary, see the [dev-guide]. --- The second commit adds a CLAUDE.md that links directly to AGENTS.md. This adds one vendor-specific file, for the benefit of getting Claude to actually follow the instructions. I've put this in a separate commit so that I can drop it if people feel strongly. I *highly recommend* that we add a CLAUDE.md, it really makes a difference. You can see examples of what Claude does without a file in the [testing repo][ETOOMANYLUNAS]. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS r? @jieyouxu cc @jackh726 @camelid @rustbot label +llm-assisted

…uwer Rollup of 11 pull requests Successful merges: - #158510 (Enable `static_position_independent_executables` on all gnu targets) - #160441 (PowerPC inline ASM: Fix scalar floats being in the wrong vector lane on little endian) - #160760 (Miri: give the incremental session a chance to finish) - #160892 (Inline some hot new-solver functions) - #160821 (Updates the expect messages in library/std/src/net/udp.rs to follow the style guide) - #160997 (Add an AGENTS.md that enforces the LLM policy) - #161057 (Silence unecessary errors on `Vec<foo::Bar>` to `Vec<foo:Bar>` typo) - #161079 (bootstrap: Clean up imports related to `define_config!` and `check_ci_llvm!`) - #161080 (propagate `RerunNonErased` in `projection_may_match`) - #161085 (bootstrap: Restrict the normalization of relative paths) - #161086 (add crashtests [3/N])

@jieyouxu

Rollup merge of #160997 - jyn514:agents-md, r=jieyouxu Add an AGENTS.md that enforces the LLM policy The goal here is to *reduce the amount of slop PRs*, not as a blanket endorsement of using agents to work on `rust-lang/rust`. For more info about motivation, see [the blog post](https://blog.rust-lang.org/inside-rust/2026/08/05/rust-langrust-is-adopting-an-llm-policy/). To that end, I have [done extensive testing][ETOOMANYLUNAS] to make sure the new file actually helps that goal. The results are promising. They show agents consistently asking their users to confirm they're read their own work; requiring users to write their own PR descriptions; and pointing people to other repos if they try to modify a submodule or subtrees, as well as various other tests. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS Almost all of the AGENTS.md was itself generated with an LLM. See the [linked testing repo][ETOOMANYLUNAS] for more information about testing and methodology. I wrote this PR description and both commit descriptions myself. --- Summary of the AGENTS.md (taken from the test repo): This is a rather long operationalization of the [dev-guide] as algorithmic instructions. Most of it was itself generated with an agent. Differences between this and the [dev-guide] / [policy]: [dev-guide]: https://rustc-dev-guide.rust-lang.org/llm-guidance.html [policy]: https://forge.rust-lang.org/policies/llm-usage.html - "strong suggestions" in the docs are turned into absolute requirements here; LLMs have trouble with discretion and judgement. - lang/edition/etc are not exempted; determining what falls under their scope was too hard to operationalize. members of those teams can still write their own AGENTS.md if this one interferes with their work (i wouldn’t expect it to, though, it’s mostly focused on code.) - the LLM requires a reviewer before making edits, not before posting the PR. experimental PRs are not exempted (how could the LLM know it was experimental?). - soundness changes are always rejected, even if allowed by the policy (how could the LLM know you’re in the org?) This is not exhaustive; if you want a more detailed human-facing summary, see the [dev-guide]. --- The second commit adds a CLAUDE.md that links directly to AGENTS.md. This adds one vendor-specific file, for the benefit of getting Claude to actually follow the instructions. I've put this in a separate commit so that I can drop it if people feel strongly. I *highly recommend* that we add a CLAUDE.md, it really makes a difference. You can see examples of what Claude does without a file in the [testing repo][ETOOMANYLUNAS]. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS r? @jieyouxu cc @jackh726 @camelid @rustbot label +llm-assisted

@jieyouxu

Rollup merge of #160997 - jyn514:agents-md, r=jieyouxu Add an AGENTS.md that enforces the LLM policy The goal here is to *reduce the amount of slop PRs*, not as a blanket endorsement of using agents to work on `rust-lang/rust`. For more info about motivation, see [the blog post](https://blog.rust-lang.org/inside-rust/2026/08/05/rust-langrust-is-adopting-an-llm-policy/). To that end, I have [done extensive testing][ETOOMANYLUNAS] to make sure the new file actually helps that goal. The results are promising. They show agents consistently asking their users to confirm they're read their own work; requiring users to write their own PR descriptions; and pointing people to other repos if they try to modify a submodule or subtrees, as well as various other tests. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS Almost all of the AGENTS.md was itself generated with an LLM. See the [linked testing repo][ETOOMANYLUNAS] for more information about testing and methodology. I wrote this PR description and both commit descriptions myself. --- Summary of the AGENTS.md (taken from the test repo): This is a rather long operationalization of the [dev-guide] as algorithmic instructions. Most of it was itself generated with an agent. Differences between this and the [dev-guide] / [policy]: [dev-guide]: https://rustc-dev-guide.rust-lang.org/llm-guidance.html [policy]: https://forge.rust-lang.org/policies/llm-usage.html - "strong suggestions" in the docs are turned into absolute requirements here; LLMs have trouble with discretion and judgement. - lang/edition/etc are not exempted; determining what falls under their scope was too hard to operationalize. members of those teams can still write their own AGENTS.md if this one interferes with their work (i wouldn’t expect it to, though, it’s mostly focused on code.) - the LLM requires a reviewer before making edits, not before posting the PR. experimental PRs are not exempted (how could the LLM know it was experimental?). - soundness changes are always rejected, even if allowed by the policy (how could the LLM know you’re in the org?) This is not exhaustive; if you want a more detailed human-facing summary, see the [dev-guide]. --- The second commit adds a CLAUDE.md that links directly to AGENTS.md. This adds one vendor-specific file, for the benefit of getting Claude to actually follow the instructions. I've put this in a separate commit so that I can drop it if people feel strongly. I *highly recommend* that we add a CLAUDE.md, it really makes a difference. You can see examples of what Claude does without a file in the [testing repo][ETOOMANYLUNAS]. [ETOOMANYLUNAS]: https://github.com/jyn514/ETOOMANYLUNAS r? @jieyouxu cc @jackh726 @camelid @rustbot label +llm-assisted

jyn514 added 2 commits August 12, 2026 19:42

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 12, 2026

rustbot assigned jieyouxu Aug 12, 2026

jyn514 changed the title ~~Agents md~~ Add an AGENTS.md that enforces the LLM policy Aug 12, 2026

clarfonthey mentioned this pull request Aug 12, 2026

Project-wide LLM policy rust-lang/rfcs#3959

Open

rustbot added the llm-assisted An LLM-assisted PR as defined by the LLM policy. Requires ahead-of-time consent by assignee. label Aug 12, 2026

This comment has been minimized.

Sign in to view

camelid reviewed Aug 12, 2026

View reviewed changes

Comment thread AGENTS.md Outdated

link to dev guide

6ab09fa

don't use real names in AGENTS.md

2f1245d

camelid reviewed Aug 12, 2026

View reviewed changes

joshtriplett reviewed Aug 12, 2026

View reviewed changes

Comment thread AGENTS.md

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Aug 12, 2026

relax reviewer constraint for devtools and debugging

1bddf2d

jyn514 force-pushed the agents-md branch from 65f82cf to 1bddf2d Compare August 12, 2026 19:39

This comment has been minimized.

Sign in to view

add an extra reminder that agent explanations are not a substitute fo…

5a58c55

…r pr descriptions Co-authored-by: Josh Triplett <josh@joshtriplett.org>

This comment has been minimized.

Sign in to view

jieyouxu approved these changes Aug 13, 2026

View reviewed changes

add license annotations

65426fd

rustbot added the A-meta Area: Issues & PRs about the rust-lang/rust repository itself label Aug 13, 2026

This comment was marked as off-topic.

Sign in to view

Turbo87 reviewed Aug 13, 2026

View reviewed changes

Comment thread AGENTS.md

Comment thread AGENTS.md Outdated

Comment thread AGENTS.md Outdated

This comment was marked as off-topic.

Sign in to view

marcoieni reviewed Aug 13, 2026

View reviewed changes

Comment thread AGENTS.md

marcoieni reviewed Aug 13, 2026

View reviewed changes

Comment thread AGENTS.md Outdated

jyn514 added 2 commits August 14, 2026 07:41

move links closer to the places they're used

ba616d9

recommend ./x, not global x

ea0aa2b

jyn514 mentioned this pull request Aug 14, 2026

Give better examples of how to run x rust-lang/rustc-dev-guide#2964

Merged

use compact bullet lists for external doc references

33da3e5

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 14, 2026

JonathanBrouwer mentioned this pull request Aug 14, 2026

Rollup of 10 pull requests #161089

Closed

This was referenced Aug 14, 2026

Rollup of 11 pull requests #161092

Closed

Rollup of 11 pull requests #161093

Merged

rust-bors Bot merged commit beff62f into rust-lang:main Aug 14, 2026
13 checks passed

rustbot added this to the 1.99.0 milestone Aug 14, 2026

jyn514 deleted the agents-md branch August 14, 2026 17:14

		@@ -0,0 +1 @@
		@AGENTS.md

Uh oh!

Uh oh!

Conversation

jyn514 commented Aug 12, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Aug 12, 2026

Uh oh!

This comment has been minimized.

camelid left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jyn514 commented Aug 12, 2026

Uh oh!

jyn514 commented Aug 12, 2026

Uh oh!

camelid Aug 12, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rustbot commented Aug 12, 2026

Uh oh!

jyn514 commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshtriplett commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

jieyouxu left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as off-topic.

Uh oh!

Uh oh!

jyn514 commented Aug 14, 2026

Uh oh!

marcoieni commented Aug 14, 2026

Uh oh!

jieyouxu commented Aug 14, 2026

Uh oh!

rust-bors Bot commented Aug 14, 2026

Uh oh!

jieyouxu commented Aug 14, 2026

Uh oh!

rust-bors Bot commented Aug 14, 2026

Uh oh!

Turbo87 commented Aug 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

jyn514 commented Aug 12, 2026 •

edited by rustbot

Loading

camelid left a comment •

edited by rustbot

Loading

camelid Aug 12, 2026 •

edited by rustbot

Loading

jyn514 commented Aug 12, 2026 •

edited

Loading

joshtriplett commented Aug 12, 2026 •

edited

Loading

jieyouxu left a comment •

edited

Loading