Add an LLM policy for rust-lang/rust

[jyn514]

Rendered

View all comments

FCP link

Summary

This document establishes a policy for how LLMs can be used when contributing to rust-lang/rust. Subtrees, submodules, and dependencies from crates.io are not in scope. Other repositories in the rust-lang organization are not in scope.

This policy is intended to live in Forge as a living document, not as a dead RFC. It will be linked from CONTRIBUTING.md in rust-lang/rust as well as from the rustc- and std-dev-guides.

Ethical issues

See this thread.

Moderation guidelines

This PR is preceded by an enormous amount of discussion on Zulip. Almost every conceivable angle has been discussed to death; there have been upwards of 3000 messages, not even counting discussion on GitHub. We initially doubted whether we could reach consensus at all.

Therefore, we ask to bound the scope of this PR specifically to the policy itself. In particular, we mark several topics as out of scope below. We still consider these topics to be important, we simply do not believe this is the right place to discuss them.

So, the following are considered off topic for this PR specifically:

• Long-term social or economic impact of LLMs
• The environmental impact of LLMs
• Anything to do with the copyright status of LLM output. We have gotten initial confirmation from US lawyers that LLM output likely doesn't cause issues under US law. We still need to confirm this with EU lawyers and in other parts of the world.
• Moral judgements about people who use LLMs

We have asked the moderation team to help us enforce these rules. For an extended rationale, please see this comment.

Feedback guidelines

We are aware that parts of this policy will make some people very unhappy. As you are reading, we ask you to consider the following.

• Can you think of a concrete improvement to the policy that addresses your concern? Consider:
  • Whether your change will make the policy harder to moderate
  • Whether your change will make it harder to come to a consensus
• Does your concern need to be addressed before merging or can it be addressed in a follow-up?
  • Keep in mind the cost of not creating a policy.

If your concern is for yourself or for your team

• What are the specific parts of your workflow that will be disrupted?
  • In particular we are only interested in workflows involving rust-lang/rust. Other repositories are not affected by this policy and are therefore not in scope.
• Can you live with the disruption? Is it worth blocking the policy over?

---

Previous versions of this document were discussed on Zulip, and we have made edits in responses to suggestions there.

Motivation

• Many people find LLM-generated code and writing deeply unpleasant to read or review.
• Many people find LLMs to be a significant aid to learning and discovery.
• rust-lang/rust is currently dealing with a deluge of low-effort "slop" PRs primarily authored by LLMs.
  • Having a policy makes these easier to moderate, without having to take every single instance on a case-by-case basis.

This policy is not intended as a debate over whether LLMs are a good or bad idea, nor over the long-term impact of LLMs. It is only intended to set out the future policy of rust-lang/rust itself.

Drawbacks

• This bans some valid usages of LLMs. We intentionally err on the side of banning too much rather than too little in order to make the policy easy to understand and moderate.
• This intentionally does not address the moral, social, and environmental impacts of LLMs. These topics have been extensively discussed on Zulip without reaching consensus, but this policy is relevant regardless of the outcome of these discussions.
• This intentionally does not attempt to set a project-wide policy. We have attempted to come to a consensus for upwards of a month without significant progress. We are cutting our losses so we can have something rather than adhoc moderation decisions.
• This intentionally does not apply to subtrees of rust-lang/rust. We don't have the same moderation issues there, so we don't have time pressure to set a policy in the same way.

Rationale and alternatives

• We could create a project-wide policy, rather than scoping it to rust-lang/rust. This has the advantage that everyone knows what the policy is everywhere, and that it's easy to make things part of the mono-repo at a later date. It has the disadvantage that we think it is nigh-impossible to get everyone to agree. There are also reasons for teams to have different policies; for example, the standard for correctness is much higher within the compiler than within Clippy.
• We could have different standards for people in the Rust project than for new contributors. That would make moderation much easier, and allow us to experiment with additional LLM use. However, it reinforces existing power structures, creates more of a gap between authors and reviewers, and feels "unfriendly" to new contributors.
• We could have a more lenient policy that allows "responsible and appropriate" use of LLMs. This raises the question of what "responsible and appropriate" means. The usual suggestion is "self-review, and judging the change by the same standard as any other change"; but this neglects the reputational and social harm of work that "feels" LLM generated. It also makes our moderation policy much harder to understand, and increases the likelihood of re-litigating each moderation decision.
• We could have a more lenient policy that abandons the "experiment" framing and removes the "circuit breaker" clause. This makes the rules much easier to predict and understand, makes it less likely that we drive away corporate funding, and avoids trying to numerically quantify problems that are qualitative. However, it has the very large downside that it does not force us to resolve our governance issues. I believe it is critically important that we don't just have a policy, but a policy that we can adapt in response to changing circumstances. By having a "forcing function", we ensure that we don't just forget about the policy, but update it over time as facts on the ground change.
• We could have a more strict policy that removes the threshold of originality condition. This has the advantage that our policy becomes easier to moderate and understand. It has the disadvantage that it becomes easy for people to intend to follow the policy, but be put in a position where their only choices are to either discard the PR altogether, rewrite it from scratch, or tell "white lies" about whether an LLM was involved.
• We could have a more strict policy that bans LLMs altogether. It seems unlikely we will be able to agree on this, and we believe attempting it will cause many people to leave the project.
• We could have no policy at all. This avoids banning valid use cases; avoids implicitly legitimizing the use of LLMs by setting a policy; and saves all of us a great deal of time and effort. However, it greatly increases the baseline level of distrust within the project; makes review assignment even more of a dice roll than it is already; wastes a great deal of contributor and moderator time dealing with LLM-authored PRs on a case-by-case basis; and gives no guidance for people who really do want to use an LLM in a responsible way.

Prior art

This prior art section is taken almost entirely from Jane Lusby's summary of her research, although we have taken the liberty of moving the Rust project's prior art to the top. We thank her for her help.

Rust

• Moderation team's spam policy
• Compiler team's "burdensome PRs" policy

Other organizations

These are organized along a spectrum of AI friendliness, where top is least friendly, and bottom is most friendly.

• full ban
  • postmarketOS - also explicitly bans encouraging others to use AI for solving problems related to postmarketOS - multi point ethics based rational with citations included
  • zig
    • philosophical, cites Profession (novella)
    • rooted in concerns around the construction and origins of original thought
  • servo
    • more pragmatic, directly lists concerns around ai, fairly concise
  • qemu
    • pragmatic, focuses on copyright and licensing concerns
    • explicitly allows AI for exploring api, debugging, and other non generative assistance, other policies do not explicitly ban this or mention it in any way
  • forgejo
    • bans AI for review, code, documentation, and communication
    • mentions "legal uncertainties" as a motivating factor
    • explicitly excludes machine translation
• allowed with supervision, human is ultimately responsible
  • scipy
    • strict attribution policy including name of model
  • llvm
  • blender
  • linux kernel
    • quite concise but otherwise seems the same as many in this category
  • mesa
    • framed as a contribution policy not an AI policy, AI is listed as a tool that can be used but emphasizes same requirements that author must understand the code they contribute, seems to leave room for partial understanding from new contributors.

      Understand the code you write at least well enough to be able to explain why your changes are beneficial to the project.

  • firefox
  • ghostty
    • pro-AI but views "bad users" as the source of issues with it and the only reason for what ghostty considers a "strict AI policy"
  • fedora
    • clearly inspired and is cited by many of the above, but is definitely framed more pro-ai than the derived policies tend to be
• curl
  • does not explicitly require humans understand contributions, otherwise policy is similar to above policies
• linux foundation
  • encourages usage, focuses on legal liability, mentions that tooling exists to help automate managing legal liability, does not mention specific tools
• In progress
  • NixOS
    • How do we deal with AI-generated issues? NixOS/nixpkgs#410741

Unresolved questions

See the "Moderation guidelines" and "Drawbacks" section for a list of topics that are out of scope.

[rustbot]

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Fallback group: @Mark-Simulacrum, internal-sites
• @Mark-Simulacrum, internal-sites expanded to Mark-Simulacrum, Urgau, ehuss, jieyouxu
• Random selection from Mark-Simulacrum, Urgau, ehuss, jieyouxu

[jyn514]

@rustbot label T-libs T-compiler T-rustdoc T-bootstrap

[jieyouxu]

I really like this version, and thanks a ton for working on it. Specifically:

• It doesn't try to dump entire walls of text, which is unfortunately a good way to be sure nobody reads it. Instead, it gives you concrete examples, and a guiding rule-of-thumb for uncovered scenarios, and acknowledges upfront that it surely cannot be exhaustive.
• I also like where it points out the nuance and recognizes the uncertainties.
• I like that it covers both "producers" and "consumers" (with nuance that reviewers can also technically use LLMs in ways that are frustrating to the PR authors!)

I left a few suggestions / nits, but even without them this is still a very good start IMO.

(Will not leave an explicit approval until we establish wider consensus, which likely will take the form of 4-team joint FCP.)

View changes since this review

[ChayimFriedman2]

The links to Zulip are project-private, FWIW.

[jyn514]

The links to Zulip are project-private, FWIW.

I'm aware. This PR is targeted towards Rust project members moreso than the broad community.

[davidtwco]

I'm happy with this as an initial policy for the rust-lang/rust repository.

View changes since this review

[workingjubilee]

@tmandry The perspectives advocated by that position during this entire debate has been a major obstruction to my, and several other contributor's, time and desire to keep working on Rust. Google, et alia, are the ones putting out their hand to halt the flywheel of contributions, while claiming they are responsible for it. Absurd. They are responsible for your paycheck, yes, but what of others? We aren't all on that payroll, not even counting just the ones who work on making sure the features they want to land, correctly.

Corporations like Google get more than just an abstract benefit of "upstreaming code" when they do. They get to offload risk. By taking on the implementation entirely for themselves, they internalize the risk of its correctness. A risk they take on while using tools that are known to have errors and disclaimed to have factual inaccuracies, to boot.

It is madness that we are being told that we must be exceedingly and indefinitely hospitable to things for which the engineering consequences of their use at scale have largely not yet been discovered, in order to perpetually appease the companies trying to sell people on unchecked usage of such tools. Why would doing so actually increase the likelihood those companies will invest in us, if it just threatens the one value proposition we actually have, of giving a damn above and beyond their corporate interests? What is there left for those corporations to invest in if we make ourselves just like them? We have to be actually different.

They will need to change something when they discover the consequences of their experiments. We will need to change something when we discover the consequences of our experiments.

If you want to know how to sell this up your chain, because you can't think of how, then consider paying for a better wordsmith.

[phaylon]

Companies that require employees to use provided AI tooling do so because they believe it makes work more efficient. Since they're paying those people for work, that's a legitimate thing to do. Characterizing this as abusive is unhelpful.

Just to add: It will be common for organizations, groups or people that are in general already regular AI users (for whatever reason) to already have significant AI work done at the point where upstreaming issues or fixes are considered. So this is less about being forced to use AI for upstream work, and more about how much additional effort they have to spend after they have arrived at a solution.

In that case the developer would have to convince their employer to spend additional resources (potentially even to fully clean room the solution?) and then also convince the project/team members that it is clean enough. And that's without the additional element of dealing with an issue that potentially has moderation involvement.

[jyn514]

In that case the developer would have to convince their employer to spend additional resources (potentially even to fully clean room the solution?) and then also convince the project/team members that it is clean enough

That is not what the policy says, no. The policy says you may have to wait up to 10 days.

[phaylon]

That is not what the policy says, no. The policy says you may have to wait up to 10 days.

Is that so? That depends on if you make it into the window built-up by the non-AI contributions in that 10-day time window, right? So the incentive is already to deploy locally until-we-get-around-to-it. If you don't want that hassle, you'd have to produce a non-AI contribution. (Apologies if I misread anything. It's 29°C inside here.)

I'm not saying everyone will be that unlucky, just reinforcing that the incentive structures at that point don't rely on any kind of extra force or push from the employer towards upstream AI contributions. Because the current disagreement seems to be about where and how the effects of the hurdles come in.

[jyn514]

So the incentive is already to deploy locally until-we-get-around-to-it.

the incentive is always that. there is no way we can make it easier to upstream code than to just run your own fork, other than completely discarding the idea of a review process. we are quibbling about the degree of review required, not about structural incentives.

That depends on if you make it into the window built-up by the non-AI contributions in that 10-day time window, right?

... which is why the point of the window is to have a discussion, not just to blindly follow the rules we made months ago. i know people don't believe me but i really do mean it when i say that i want us to be able to change the policy.

[steffahn]

Hey everyone, please make sure to open review threads¹ whenever possible²; any longer back-and-forth on specific discussion points held in the top level thread is increasingly making the discussion very hard to navigate (and also burying existing threaded discussion in the conversation tab).

Footnotes

1. by creating a review-comment on a specific line, or line-range, or even on the whole file ↩

2. GitHub doesn't allow us to move anything into threads retroactively! So this is easiest if you start discussing a new thing; it’s second easiest if you reply to something the author may not have intended to become a longer discussion point; it's harder but still valuable to pivot discussions into a dedicated review thread at a later point

   e.g. for any additional commentary on the circuit breaker topic now that there's been a new commit might be a good time ↩

[traviscross]

(This continues the top-level thread at #1040 (comment) through #1040 (comment).)

I know people don't believe me but I really do mean it when I say that I want us to be able to change the policy.

I believe you.

Meaning that you want us to be able to change the policy is different, though, than the forward-looking factual question of how hard it will prove.

The policy says you may have to wait up to 10 days.

If the natural rate of some group of PRs (that we have the review capacity for and would otherwise accept) is 60% and we impose a 50% rate limit, then (absent backpressure) that will produce an unbounded backlog and unbounded latency, not 10 day latency.

It is madness that we are being told that we must be exceedingly and indefinitely hospitable to things for which the engineering consequences of their use at scale have largely not yet been discovered, in order to perpetually appease the companies trying to sell people on unchecked usage of such tools.

Nobody is suggesting that we must be hospitable or appeasing. The choices we make simply affect the help we can expect to receive.

We should act with knowledge of the consequences of our actions. @tmandry, who has spent of himself for years to attract help for us, is offering valuable insight into that when it would be easier for him to stay silent and keep out of this unpleasant discussion.

I'd suggest we'd do well to be appreciative and curious.

View changes since the review

[PLeVasseur]

As someone in a similar role as @tmandry, working to attract support to the Rust Project as lead of the Safety-Critical Rust Consoritum and guiding Rust adoption at Toyota Group, I feel the concern @tmandry raised sharply.

Even within safety critical spaces, where there's potential for loss of human life, we're working on how AI tooling can be incorporated in our processes while ensuring quality, verification, and compliance. If, after doing that substantial work, we couldn't use our own validated processes for working on Rust (without incurring arbitrary and unmanagable delays), that would impair my ability to attract support for the Project.

[AthenaEryma]

Why are conflicts with the policies of large corporations and contributor alienation that may cause considered an on-topic motivation for policy here, when contributor alienation caused by the usage of LLM-generated code at all due to things in the Official List of Off-Topic Topics in the OP is not?

[lumi-me-not]

I don't think @tmandry is wrong, and I do think it is valuable to let us know about this, but I just don't think we should be sacrificing our values as a project to appease corporate interests. Having boundaries is good, and there are some we should not cross, no matter what potential gains we could get.

[clarfonthey]

My longer comment was deleted after a mutual agreement that it was unhelpful. And so, I'm going to include my shorter bit to at least address something important:

Companies that require employees to use provided AI tooling do so because they believe it makes work more efficient. Since they're paying those people for work, that's a legitimate thing to do. Characterizing this as abusive is unhelpful.

Forcing people to do something that does not necessarily make them productive, that sometimes makes them less productive, that sometimes makes them less happy, even in circumstances where other projects have explicitly drawn boundaries around its usage, is abusive. Full stop, not debatable. I have experienced enough abuse in my days to know what it looks like, and this is it. This is not sensationalising; this is correctly using a term to describe something that's happening.

Also, keeping this analogy in because I find it apt: imagine if companies instituted mandatory pomadoro timers and apps upon all employees, because one employee thought they were helpful for productivity. For some people, it would be incredible to have productivity tools supported by their company like this. For other people, it would be literal hell on earth and absolutely make them miserable and worse at their job. We can call this what it is: abuse. There are enough people who've talked about how LLM mandates at their jobs demonstrably made their working life miserable that I don't think it's debatable that calling this abuse is correct.

Again, if individual employees are aided by LLM usage, if it makes them more productive, if it makes them happier, that is a valid argument that I feel should be on the table for allowing its use. That people are forced to use it, is not a valid argument. This is going to be my final message here.

[nnethercote]

This sounds reasonable to me, and I'm willing to trust Jack and Josh's viewpoints.

[6TELOIV]

@joshtriplett just throwing ideas out, not sure if something like this has/is being discussed elsewhere, so appologies if I'm reinventing a discarded idea.

1. Set a recuring time interval for experiment review ¹
2. Leadership Council delegates review to a trusted subset of individuals ²³
3. That group's responsibilties are:
   • Review the experiment progress and outcomes
   • Make changes to the experiment, including but not limited to
     • Tightening or relaxing experiment requirements ⁴
     • Procedural changes to facilitate better experimentation (data gathering, methods, etc)
   • Propose termination of the experiment, with discussion of conculsions reached, and the results reflected in non-experimental policy.
4. This group has full control to make these changes to the experiment, with the important exception that termination of the experiment (including termination in-effect) requires MCP ⁵

The overall idea here is that this group acts as human circuit breakers. I trust people (especially people not entrenched in either camp) to be able to recognize and remediate issues better than any automated metric we could come up with.

Of course, none of this precludes anyone else from proposing changes, including termination of the experiment, following the "PR for minor change, MCP for major" part of this policy.

Also, thinking some more, the policy should specify that the ultimate goal of the group is to terminate the experiment once enough information, both quantitative and qualitative, has been gathered to solidify the LLM policy, and once that policy work has occurred.

Footnotes

1. Maybe 18 weeks? That's 3*6 week intervals. The point is that whatever length is chosen needs to be frequent enough to keep the experminet moving, and infrequent enough to keep it out-of-the way of day-to-day operation. ↩

2. This avoids the issue of either extreme getting an automatic veto to terminate/indefinitely continue the experiment, as this group ↩

3. The group could possibly be the LLM Committee proposed in https://github.com/rust-lang/leadership-council/issues/308 ↩

4. But not fully in either direction; the intent is they have the power to prevent harm if LLM use gets out of hand, and also open novel avenues for experimentation ↩

5. Requiring MCP to terminate the expermient could be interpreted as the experiment being de-facto policy and never ending. But, I trust those with power to raise concerns ("veto") termination to use it responsibly, and that the experiment should only be sunset once it has informed any necessary policy changes, either here or elsewhere, and MCP can be reached that it can be sunset. ↩

[lumi-me-not]

Another avenue for reducing the scope of the experiment could be to disallow anything done using "agent mode" (as in, telling an agent to do things via a prompt, like in Claude Code, Codex, etc). This would then allow things like LLM autocomplete, but not much more.

While I am completely against all GenAI/LLM for any purpose, there are things that are more (or less) objectionable, to me. And some people in the same camp may be okay compromising a little in this dimension and allowing experimentation with LLM autocomplete.

[camelid]

IMO, restricting usage to autocomplete is a strange and arbitrary line to draw in the sand. I also find autocomplete to be rather useless for any large codebase except for trivial things that would already be allowed by the main policy.

Also, I would rather have a clear boundary like this PR proposes, where in general only trivial LLM-based changes are allowed, where we judge the result as trivial rather than the process to get there. And then within the experiment part, we have the freedom to, well, experiment as much as we want with how the LLM is used, as long as it stays within the area carved out for experimentation (reviewer on board, non-critical part of codebase, etc.).

[jyn514]

I've thought about this for a long time, and chatted with Tyler, Boxy, Josh, and various other people.

I understand that this clause is not popular.
I agree with Tyler and Pete that there's potential for this to affect the ability for us to get funding and maintainer time from members of the Foundation, and with Ralf that this is probably not the most technically effective way for the clause to achieve its own goals.

However, I want to caution very strongly against trying to achieve perfection here. The goal of this policy is not to last forever; it's to get a policy that we can agree on, without being locked in limbo forever. I think in practice the chances of this clause ever being triggered are very low, and if it is, that's a sign that we have failed to update the policy in response to changing circumstances, not a sign that the policy itself has issues. In that case, I think it is in fact a good thing that we're forced to come back to the table and discuss further.

See the thread above for my comment on how I think we can update the policy in response to changing circumstances.
See the edited PR description for my attempt to write up the tradeoffs of removing this clause.

[traviscross]

(The top-level thread on the subject raised in #1040 (comment) now continues at #1040 (comment).)

[nnethercote]

Given the recent (and non-trivial) changes to the policy after all the boxes had been checked I think it's worth giving this a bit of additional time over the normal FCP period just to make sure everyone is aware and had the time to review those changes.

@rfcbot concern wait-time-for-josh-policy-changes

I don't know how long is good to wait here, maybe a week or two before going into the final comment period again?

@BoxyUwU: it's been almost two weeks. Do you think that's enough time? People who have ticked a box have had plenty of time to revisit their decision if they want. Additional waiting doesn't seem like it will achieve anything more here.

[BoxyUwU]

@rfcbot resolve wait-time-for-josh-policy-changes

I do think enough time has passed that people have now seen the changes and there's been discussion about it. I do also think that we shouldn't merge this policy while there is ongoing discussion about these changes and there are people with concerns.

I felt hesitant to actually resolve my concern here because no concerns had been formally raised about josh's changes even though there is certainly informal pushback. I don't want this policy to just "go forwards" past this feedback. I also don't really want register a concern of the form "some people have feedback" because that feels too close to proxying other peoples concerns for my own comfort, but I'm not sure how else this is supposed to work

@rfcbot concern unresolved-informal-feedback-on-josh-policy-changes

It feels fairly clear to me that this policy is getting slightly screwed over by the fact that it can't be a real "experiment" because people (myself included) don't think it'll actually be able to be changed. rust-lang/leadership-council#308 seems like a nice way to resolve that and would obviate this PR (and its associated nightmarish billion person FCP).

I don't know how much appetite there really is in the project for delaying a policy by a bunch of time while we propose a committee, form the committee, have the committee actually do the policy, etc. Things feel really delicate and I don't have a good sense of what the right way forward is (I wish we'd done something like the committee idea from the start Tbh).

I think at minimum it doesn't really makes sense for this policy to be moving forward with rust-lang/leadership-council#308 existing? we should be waiting until that either gets accepted or doesn't get accepted 😅

@rfcbot concern what-is-going-on-with-llm-committee

[joshtriplett]

I think at minimum it doesn't really makes sense for this policy to be moving forward with rust-lang/leadership-council#308 existing? we should be waiting until that either gets accepted or doesn't get accepted 😅

I don't think we should delay one to wait on the other. Several of the folks on the Council who are talking about the committee proposal are saying that we don't want to block or slow down other concurrent policy work. For instance:

So many people have worked hard on drafting, reviewing, discussing and updating 1040 that I also wouldn't want this to stop the current process.

I don't believe we should preempt the current process. Rather, I think we should allow them to proceed in parallel. [...] let's not throw a crowbar into the current policy process at the last minute, rather than allowing it to complete if it's going to complete.

AFAICT, the goal of the committee is to provide paths for policy evolution/adaptation, and to write policy that will be more broadly applicable across the project, but that shouldn't stop us from enacting policy for rust-lang/rust if we can.

[nnethercote]

I don't think we should delay one to wait on the other.

I agree. This issue was created almost three months ago. It has reached the required N-2 approval threshold for four different teams. That is 26 separate approvals, an extraordinary number. I understand that some people have disengaged from discussion; pausing to delay for vague reasons is not going to change anything of note. Let's move forward.

[Kobzol]

@BoxyUwU If you are waiting for the LLM committee, I would suggest.. don't 😆 That is orthogonal to this issue, and if you want to continue with it, then feel free to. Creating the committee might take a lot of time anyway.

[traviscross]

Agreed with @Kobzol. What I would humbly suggest for moving this forward is for the leads of the five teams (one lead per team) involved to get together with the author of the policy and agree on its final form, within the scope of those teams, taking into consideration the feedback that's been received, then ask their members to resolve and withhold concerns.¹ I'm sure that our program managers would be happy to handle the logistics of setting up this meeting.

Footnotes

1. I'd earlier proposed that the council make this suggestion, but we weren't able to cover this in the Friday council meeting, so the suggestion here is mine alone. ↩

[jyn514]

@BoxyUwU

It feels fairly clear to me that this policy is getting slightly screwed over by the fact that it can't be a real "experiment" because people (myself included) don't think it'll actually be able to be changed. rust-lang/leadership-council#308 seems like a nice way to resolve that and would obviate this PR (and its associated nightmarish billion person FCP).

I've explicitly noted that the LLM committee is allowed to overrule this policy; see this thread.

I felt hesitant to actually resolve my concern here because no concerns had been formally raised about josh's changes even though there is certainly informal pushback. I don't want this policy to just "go forwards" past this feedback.

I've explicitly left a response to Tyler and Pete's concerns in this thread. I agree we shouldn't just ignore the concerns, but I also think for a policy this controversial we're simply not going to find something that everyone is happy with. I think we should move forward with this as a starting point, with the explicit plan to improve our governance so that it's easier to change this going forward.

View changes since this review

[jyn514]

I agree with Tyler that we should have a bias towards action here, and with TC that we have to deal with the facts of how hard it will be to change the policy, not our ideals of how easy we wish it should be. To that effect, I've added a new condition for modification or dissolution:

If the proposed LLM committee is created, it would have governing authority over this policy.

That avoids needing unanimous consent from ~30 different people while still while still ensuring people are represented. One concern I've heard is that team leads and council members have a very broad mandate and wouldn't have proper time to dedicate to the policy; I think having a dedicated committee avoids that.

[jyn514]

I've thought about this for a long time, and chatted with Tyler, Boxy, Josh, and various other people.

I understand that this clause is not popular.
I agree with Tyler and Pete that there's potential for this to affect the ability for us to get funding and maintainer time from members of the Foundation, and with Ralf that this is probably not the most technically effective way for the clause to achieve its own goals.

However, I want to caution very strongly against trying to achieve perfection here. The goal of this policy is not to last forever; it's to get a policy that we can agree on, without being locked in limbo forever. I think in practice the chances of this clause ever being triggered are very low, and if it is, that's a sign that we have failed to update the policy in response to changing circumstances, not a sign that the policy itself has issues. In that case, I think it is in fact a good thing that we're forced to come back to the table and discuss further.

See the thread above for my comment on how I think we can update the policy in response to changing circumstances.
See the edited PR description for my attempt to write up the tradeoffs of removing this clause.

[BoxyUwU]

In retrospect I wish I hadn't filed the concern about the interacts with the LLM committee. While I was unsure what was going on there, it wasn't really the crux of my concern and I certainly wasn't confused enough to warrant blocking the policy. The convo after I raised this concern has also cleared this up for me :3

@rfcbot resolve what-is-going-on-with-llm-committee

In general I dislike "unactionable" concerns on FCPs that don't really leave a way forwards. My concern about the informal feedback about Josh's changes was very much not actionable. Unforchies!

The thing I care about isn't "everyone in the project is in favour of this policy" as I don't think that's particularly attainable, the project is simply too big and LLMs are simply too controversial ✨ Instead, really what I care about is that people feel like their voices were heard and that they can understand why controversial decisions were made.

I think one part of doing this is to make sure that feedback is always "considered"/"acknowledged". We shouldn't finish FCPs with feedback which the responsible team never explicitly took into account (especially for something as controversial as this policy).

Taking feedback into account could mean resolving the concerns (i.e. the concerned party no longer is concerned), or it could just mean the responsible team writing down why they disagree with the feedback and moving forwards.

With jyn's comment and edits to the PR description I do feel that the feedback about Josh's changes has been considered/acknowledged even if there are still people concerned about the added clause.

@rfcbot resolve unresolved-informal-feedback-on-josh-policy-changes

I expect us to find a way to more easily update this policy in the short/medium term, for example the LLM committee proposal (rust-lang/leadership-council#308). This makes me much more willing to accept this policy even while it contains parts which feel somewhat suboptimal :>

@rfcbot reviewed

[theemathas]

Is FCP considered to have started? If not, what is currently blocking it?

[BoxyUwU]

it's morally final comment period yeah