Project-wide LLM policy

[clarfonthey]

View all comments

Preface

A lot of discussion has occurred in private about the topic of LLM policy, and while some of that context has been included in the prior art, most of it is intentionally omitted here.

To keep things focused on policy, there are two broad categories of comments we'd like to request you avoid:

• Simply stating your viewpoint on LLMs, even if you provide reasons. While these arguments can be useful for the RFC, they are better worded as explicit suggestions to specific areas of the RFC, rather than as just general comments.
• Debating others' viewpoints on LLMs, or calling others out on their viewpoints. Again, we would rather these be focused on specific policy points and the merits of those points, than just general bickering.

In general, defer to the code of conduct.

Important

Since RFCs involve many conversations at once that can be difficult to follow, please use review comment threads on the text changes instead of direct comments on the RFC.

If you don't have a particular section of the RFC to comment on, you can click on the "Comment on this file" button on the top-right corner of the diff, to the right of the "Viewed" checkbox. This will create a separate thread even if others have commented on the file too.

Existing policies

Right now, the primary LLM policy for the Rust project is the Forge policy which has been adopted by a large number of teams but importantly, not all teams. The current plan by the Leadership Council is to create an LLM team to formalise project-wide policy, and whether this policy is ratified by the LC or their LLM subteam does not strictly matter.

This RFC does intend to replace all team-specific policies. It doesn't try to rephrase or rescope them because it was proposed before they were even accepted, even though it has grown a lot next to them. Feedback given to other policies has been incorporated here too.

Summary

This policy details the requirements for using generative Artificial Intelligence (AI) models, particularly Large Language Models (LLMs), in all aspects of the Rust project. This includes (but is not limited to) contributions of code, documentation, chat messages, issue descriptions, etc.

1. Trivial LLM usage is always allowed, and thus irrelevant to this policy.
   We allow changes made by LLMs are indistinguishable from those made by humans, where the LLM didn't have any creative input.
2. Slop, also known as vibe-coding, is spam and moderated accordingly.
   We disallow changes made by LLMs with minimal human intervention.
3. Potentially non-trivial LLM usage must be disclosed, ideally with as much detail as possible.
   (RFC-only note: This may necessitate additional tooling to notify new contributors about the policy and explain how disclosure works.)
4. Non-trivial usage, if correctly disclosed, is merely rejected or closed without any additional action.
   There are no formal punishments for non-trivial usage with disclosure.
5. Non-trivial usage without proper disclosure can become a Code of Conduct violation.
   There are formal punishments for frequent non-trivial usage without disclosure; this is seen as intentionally refusing to honor the boundaries we've set.
6. If a contributor does not fully understand the code they submit, their contribution may be rejected for that reason alone.
   This is similar to, but not always slop. For example, they may understand a large portion, but not all of it, which shows that they still put in a lot of effort.

In general, as long as contributors are demonstrating an earnest effort to reduce LLM usage, even if said reduction is not total, then they should be commended for that instead of punished, even if the project itself draws the line at non-trivial usage for accepted contributions.

RFC-only note: In terms of additional tooling for disclosure, this RFC encourages the creation of a bot that automatically replies to contributions from new users informing them of the LLM policy and what constitutes sufficient disclosure. As mentioned, in general, going into as much detail as possible (e.g. prompts used, etc.) is preferred, but not always required. The RFC leaves the exact details of such implementation unspecified and up for revision later.

Rendered

GitHub is being weird. I already merged these changes, so, it's weird it's showing them as unaddressed.

[imyxh]

Whatever it means to you to hear this from the outside: I sympathize greatly with your frustrations here and I appreciate your work more than you can imagine. Please take care, please feel better.

[lumi-me-not]

Thank you for all the work you've done. I also really appreciate it.

Take a break if you need to, take care of yourself. If you wish to vent, feel free to reach out.

Wishing you the best.

[clarfonthey]

So, it appears that somehow the root of this thread got hidden because of a direct report to GitHub, which is extremely concerning. For now, you can read it on rustbot's view, since it appears that GitHub implemented this (likely new) feature very poorly. But note that not even the moderators can unhide this comment, and I'm not even sure org admins can either. Will have to investigate, but, since Rust Week is this week, might take a while.

[oli-obk]

We (leadership council) are taking this off our nomination list. We don't have something concrete to discuss about it at present, but it can be renominated once the rust-lang/rust policy is merged (or otherwise handled) and there is a concrete ask of the council.

[clarfonthey]

I was honestly not expecting it to get nominated in the first place, so, that makes sense, although I would assume that the LC would be the ones on the FCP unless I'm mistaken.

[clarfonthey]

Hello, everyone!

You may have noticed that all the active discussion threads have been resolved, minus two:

• One thread for people to express that they would not be comfortable supporting the project if it does not restrict LLM usage.
• One thread where, upon sharing my personal exprerience with the vitriol surrounding this policy discussion, GitHub decided to block my comment under the "acceptable use" policy, even though it does not. They've refused to even respond to my support requests, even after it has been privately brought up through other channels to them.

Both of these remain relevant throughout the policy discussion and emphasise the need to put our feet down and unilaterally condemn and restrict LLM usage. Note that I still say restrict, not ban, because LLM usage is entirely unrestricted up until the point where it affects the content being written. If LLMs are still useful for accessibility, or if they are still usefull for sifting through data and other tasks, these usages are unaffected. See the RFC itself for more motivation and details.

If you're wondering where the old Motivation section went, a lot of people complained that it should be a blog post, and now it is. Note: effectively all the links that were shared in that motivation section are still included as relevant citations, summarised in the newer, shorter motivation section, and the blog post version is linked alongside it for context. I decided to concede that this context was an unnecessary barrier to reading the RFC, but importantly, all of the relevant citations are still included. Technically, the justification is all there, just in a form that offers additional reading instead of constituting required reading.

Side note: unfortunately, because GitHub doesn't like accessible semantic HTML, specifically on the GitHub view, you will need to expand the citations explicitly for links within them to work. (In plain HTML, <details> are automatically opened when anchors within them are linked. GitHub hates this for some reason.)

The RFC now is almost entirely the actual policy, a timeline of events in the Rust project, and examples of other policies in effect. There are only minor wording changes to the actual policy, plus one additional clause: Co-Authored-By trailers are explicitly forbidden, and Assisted-By and similar trailers are discouraged but not forbidden. This is for two reasons: one, that the presence of these trailers generally indicates nontrivial usage, and two, that they effectively provide advertisements for LLM providers in contributor lists.

Side note: I won't talk too much on the outstanding discussion to create a committee to shepherd the policy discussion. It is my opinion that this policy can continue to solicit feedback from project members regardless of who ultimately ratifies it. I believe that any accepted policy will have a shape similar to this one, similar to the proposed Forge policy, etc. and also believe that the committee should work with existing proposals rather than try to shut them down, which has already been expressed in that discussion.

View changes since the review

[clarfonthey]

Messing with the GH API, I managed to find a way to isolate the counts of members who have reacted to the RFC (at the top level) from T-all (with ❤️ being used if both 👍 and ❤️ are present):

❤️ ×12
👎 ×3
👍 ×2

Don't wanna explicitly call people out given the small numbers, but just in case folks are worried that the overwhelmingly positive response is purely due to outside observers: it's not.

For reproducibility

curl https://team-api.infra.rust-lang.org/v1/rfcbot.json | jq .all_members[] > members  # new API!
gh api /repos/rust-lang/rfcs/issues/3959/reactions --paginate --jq '.[] | { login: .user.login, content }' > reactions
jq --slurpfile members members --slurpfile reactions reactions --null-input '[$reactions[] | select([.login] | inside($members))] | group_by(.login) | map({key: .[0].login, value: [.[].content]}) | from_entries | map_values(if length == 1 then .[0] elif . == ["+1", "heart"] then "heart" else . end) | to_entries | group_by(.value) | map({ key: .[0].value, value: [.[].key]}) | from_entries'

[clarfonthey]

Another small update: @jyn514's work on creating an AGENTS.md file to help enforce the current policy for rust-lang/rust demonstrates that models can be pushed to encourage users to disclose and even reduce the extent of their usage, and so, should be allowed within this policy. I've added a dedicated section for this to encourage similar work.

You can see the work on this here: rust-lang/rust#160997

[MusicalNinjaDad]

To my mind, the current summary is convoluted and open to too much interpretation. I get the feeling that it is designed to be worded "nicely" rather than "clearly", and it leaves open the potential for too many cases of "I thought this was trivial".

Code is easier than words (the current wording would require a series of out-of-logical-order if/match blocks). I would propose simplifying to:

bitflags! {
    struct LlmUsage {
        const NONE               = 0,
        const TEXT_TO_SPEECH     = 1,
        const RESEARCH           = 2,
        const BOILERPLATE        = 4,
        const IDEATION           = 8,
        const CODING             = 16,
        const DOCUMENTATION      = 32,
        const TESTS              = 64,
        
        const DISCLOSURE_EXEMPT  = 1,
        const TRIVIAL            = 7,
    }
}

/// Raising a PR *requires* disclosure of LLM usage.
///
/// The PR template includes a standard disclosure checklist ("choose all that apply")
/// and pop-up / bot is in place to highlight cases where no checkbox is selected.
pub fn raise_pr(name: &str, description: &str, llm_disclosure: LlmUsage) -> PrResult {
    if llm_disclosure > LlmUsage::TRIVIAL {
        return Err(LlmViolation);
    }
    ...
}

/// This is optionally available to members, to avoid the concerns of
/// "I always have to tick `None`"
///
/// The trust extended to members, is that they *will* disclose any non-exempt usage.
/// Voilation of this trust can lead sanctions including to revocation of membership.
///
/// [LlmUsage::DISCLOSURE_EXEMPT] specifies disclosure-exempt usages.
pub(members) fn raise_nonllm_pr(name: &str, description: &str) -> PrResult {
    ...
}

Based on this I would suggest the summary list be adjusted to:

1. All LLM usage is subject to disclosure with the following exemptions:
   • Text-to-speech (for accessibility purposes)
2. Trivial LLM usage, is allowed, but should be disclosed. We allow changes made by LLMs are indistinguishable from those made by humans, where the LLM didn't have any creative input. In particular this covers:
   • Research (please provide details to help us delineate "resarch" from "ideation")
   • Boilerplate (please provide details to help us understand exactly what was generated)
3. Non-trivial LLM usage, if correctly disclosed, will rejected or closed without any additional action.
   There are no formal punishments for non-trivial usage with disclosure. In particular this includes, but is not limited to:
   • Ideation
   • Code generation
   • Writing documentation
   • Writing tests
4. Non-trivial usage without proper disclosure can become a Code of Conduct violation.
   There are formal punishments for frequent non-trivial usage without disclosure; this is seen as intentionally refusing to honor the boundaries we've set.
5. Slop, also known as vibe-coding, is spam and moderated accordingly.
   We disallow changes made by LLMs with minimal human intervention.

[kennytm]

@MusicalNinjaDad I'm replying to #3959 (comment) here to avoid polluting the main thread.

Sorry I find the pseudo-code even harder to digest, like normally bitflags won't be compared using >, and DISCLOSURE_EXEMPT is used nowhere, and the main difference between are two functions¹ are erased behind ....

My understanding of your comment is like this:

Category	Examples	Allowed?	Correspondence in 01d100c
1/ Accessibility	Speech-to-text2	Yes (disclosure not required)	-
2/ Trivial	Research Boilerplate	Yes (disclosure encouraged)	Summary pt. 1, 3
3,4/ Non-trivial	Ideation Coding Documentation Tests	No Intentional concealment = CoC violation	Summary pt. 4, 5
5/ Slop	(Vibe-coding)	No — treated as spam	Summary pt. 2, 6

This differs from the actual RFC text which "Research" is classified as "Potentially non-trivial", and there is no independent "Accessibility" category out of "Trivial". Effectively you're promoting boilerplate generation to "Potentially non-trivial" too, and rebranding it "trivial".

Additionally it seems your re-summary is focusing on disclosure requirement, whereas the original summary focuses on what can and cannot be accepted. IMO if you think disclosure requirement should be addressed first, you don't just need to change the Summary but reorder the entire Guide-level Explanation as well.

View changes since the review

Footnotes

1. Why we have a pub(members) visibility in the first place? The RFC explicitly said "Whatever policy is adopted, it should be symmetric between project members and new contributors.". ↩

2. I'm pretty sure you mean "speech-to-text" rather than "text-to-speech", and the RFC also discussed about the former only. ↩

[MusicalNinjaDad]

Thanks for adding a comment @kennytm , to address each of your points:

1. Pseudo-code - each to their own, I'm not suggesting including pseudo-code in the actual policy, just found it the easiest way to consolidate my thoughts.
2. Re-summary - intention - A good policy summary needs to immediately and clearly convey the fundamentals of the policy to the widest possible audience. The goal is that if someone only reads the summary then their actions should be expected to conform to the policy. I do not believe that the current policy summary achieves this goal.¹²³
3. My concern - The current summary allows for the following thought chain without any malice on the contributor's part:
   1. "Trivial LLM usage is always allowed, and thus irrelevant to this policy." - Ok, that's fine, my usage was trivial
   2. "Slop, also known as vibe-coding, is spam" - Agree, definitely not what I did, I'm fine
   3. "Potentially non-trivial LLM usage must be disclosed, ideally with as much detail as possible." - There's no way my usage was non-trivial, I'm confident of that
   4. "Non-trivial usage, if correctly disclosed, is merely rejected or closed without any additional action." - Ok, that makes sense, I'll remember not to use LLMs in any significant way for rust-lang/rust
   5. "Non-trivial usage without proper disclosure can become a Code of Conduct violation." - fine, makes sense
   6. "If a contributor does not fully understand the code they submit, their contribution may be rejected" - nope, I've got a good handle on this code. I'm good to submit
4. Re-ordering - The suggested order aims to:
   • Be clearly structured by level-of-usage. From most to least trivial.
   • For each "level" answer both "what falls in this category?" (examples) and "what do I need to do?" (the key action required is usually disclosure-related, hence the prevalence)
   • Leave much less room for interpretation when read by a contributor (or reviewer) and allow for use as a simple series of instructions to follow/consider
5. Removal of "potentially non-trivial" - I found the category was not actually needed in the summary and open for confusion. I'd see no problem with rebranding LlmUsage::TRIVIAL to LlmUsage::REQUIRES_REVIEW in the pseudo-code and updating the summary to reflect this rather than categorically claiming examples to be "trivial" - that would likely be a good change to make to my suggestion.
6. Ordering of guide-level implementation - The full RFC text serves a different purpose than the policy summary and the current ordering makes sense. Full policy text needs to fully explain the motivation, nuances and logical reasoning behind the policy the current text & ordering achieves this.
7. pub(members) fn raise_nonllm_pr - aims to highlight and ensure discussion on how to handle the tool-support for this policy.
   • If a bot enforces everyone to always answer "No LLM usage", this could quickly become tiring for those who contribute very regularly and have done for a long time.
   • If there is no "did you read the policy and can you confirm no usage?" prompt for anyone then the risk of the policy being missed, and the extra work on reviewers to read & reject PRs goes up.
   • My suggestion, was that given a certain level of trust, we can expect someone to a) have read the policy b) follow the policy and therefore switch off the "please confirm ..." tool.
   • There's a bug in the code, which I've now fixed, by removing the Option<LlmUsage>
   • LlmUsage::DISCLOSURE_EXEMPT is purely provided to clearly document when (no) disclosure is required
8. Accessibility - yes, I switched text-to-speech and speech-to-text in my head when writing this.

Footnotes

1. Both of us have read, and spend time considering, the full text. We both came to different conclusions on how to interpret "potentially non-trivial" and what exactly falls into each category. ↩

2. The current policy contains the following text on boilerplate: "Even the writing of certain code or text can be considered trivial, if little creative input is required to write it. "Boilerplate" code is a good example of this." I now realise that I have inadvertently moved Boilerplate from "trivial" to "potentially non-trivial" (I actually think it belongs in "potentially non-trivial", but would possibly be happier to see it in "non-trivial" where you expected to find it than in "trivial" where it currently sits) ↩

3. The current policy contains the following text on research: "If LLM usage falls in the gray area of "research," then disclosure is only requested if a maintainer is confused or asks what your process was." ↩