Optimize SPFx release pages for agentic upgrade workflows#10921
Conversation
|
Learn Build status updates of commit 90e3802: ❌ Validation status: errorsPlease follow instructions here which may help to resolve issue.
For more details, please refer to the build report. Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them. |
|
Learn Build status updates of commit 90e3802: ❌ Validation status: errorsPlease follow instructions here which may help to resolve issue.
For more details, please refer to the build report. Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them. |
90e3802 to
1dd128e
Compare
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit 1dd128e:
|
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit f2ddca8:
|
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit c9853a9: 💡 Validation status: suggestions
This comment lists only the first 25 files in the pull request. docs/spfx/release-1.22.0.md
For more details, please refer to the build report. Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them. |
|
Set to draft, found some inconsistencies that I need to investigate. |
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit 520e4c6: 💡 Validation status: suggestions
This comment lists only the first 25 files in the pull request. docs/spfx/release-1.22.0.md
For more details, please refer to the build report. Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them. |
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit 1d58183: ✅ Validation status: passed
This comment lists only the first 25 files in the pull request. |
andrewconnell
left a comment
There was a problem hiding this comment.
WRT one file in this PR: docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md ...
While there is value in providing the CLI upgrade steps, but want to get approval from @VesaJuvonen about removing all detailed steps & documentation on what changed.
Some companies can't use open source/community tools, while others can't unless it goes through a rigorous security review. Only providing instructions for a tool that some people might not be allowed to use removes a resource. IMHO, both options should be present, not just the CLI option.
Flagging & requesting @VesaJuvonen review to make the decsision.
|
@andrewconnell - badly communicated from my side. This is for improving agents in the upgrade scenarios and based on detailed testing with multiple models we currently have a challenge with the upgrades and agent(s) start doing wrong things. Calling out the challenge of the CLI as open-source is valid and we might want to adjust that embedded message a bit to be calling that out in same ways as we always callout PnP tooling to be officially not supported by Microsoft, even though it's mentioned. We do not really have good upgrade guidance otherwise except the CLI tooling - and when it's used - upgrade results were significantly better with agents. So - on the TIP - let's be specific that it's an option, but not the only way - as that would be wrong as well. Include has the open-source note already nicely. |
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit 71c2153: ❌ Validation status: errorsPlease follow instructions here which may help to resolve issue.
docs/spfx/release-1.23.2.md
For more details, please refer to the build report. Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them. |
71c2153 to
f41ca2e
Compare
|
Learn Build status updates of commit f41ca2e: ✅ Validation status: passed
This comment lists only the first 25 files in the pull request. |
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
- Replace shared TIP with WARNING about upgrade complexity - Add version-specific TIP with npx upgrade command to each release page - Add CLI for Microsoft 365 community disclaimer (pnp-o365cli snippet) - Rename snippet from spfx-release-upgrade-tip to spfx-release-upgrade-warning - Rewrite migrate-gulptoolchain-hefttoolchain.md for CLI-driven approach - Restructure release-1.22.0 upgrade section - Fix broken bookmark in release-1.15.2 (azure-ad -> entra-id) - Fix absolute link in release-1.22.0 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
f41ca2e to
2937ceb
Compare
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit 2937ceb:
|
| File | Status | Preview URL | Details |
|---|---|---|---|
| docs/spfx/release-1.15.2.md | View | Details | |
| docs/spfx/release-1.22.0.md | 💡Suggestion | View | Details |
| docs/spfx/release-0.0.0.md.template | ✅Succeeded | ||
| docs/spfx/release-1.0.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.1.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.10.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.11.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.12.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.13.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.13.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.14.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.15.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.16.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.16.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.2.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.3.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.4.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.2.md | ✅Succeeded | View | |
| docs/spfx/release-1.19.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.20.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.21.0.md | ✅Succeeded | View |
This comment lists only the first 25 files in the pull request.
docs/spfx/release-1.15.2.md
- Line 64, Column 44: [Warning: bookmark-not-found - See documentation]
Cannot find bookmark '#requesting-permissions-to-azure-ad-applications-in-another-tenant' in 'spfx/use-aadhttpclient.md', did you mean '#requesting-permissions-to-entra-id-applications-in-another-tenant'?
docs/spfx/release-1.22.0.md
- Line 12, Column 3: [Suggestion: docs-link-absolute - See documentation]
Absolute link 'https://learn.microsoft.com/javascript/api/overview/sharepoint?view=sp-typescript-latest' will be broken in isolated environments. Replace with a relative link. - Line 12, Column 3: [Suggestion: preserve-view-not-set - See documentation]
You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL. Otherwise, remove the view parameter. URL: https://learn.microsoft.com/javascript/api/overview/sharepoint?view=sp-typescript-latest
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
|
Thanks @andrewconnell and @VesaJuvonen for the review. Findings from testingBefore discussing next steps, I want to share the reasoning behind the changes in this PR, because none of them were arbitrary. How agents approach upgradesFrom our trajectory analysis, agents form their upgrade plan before they read any documentation. By the time the agent fetches the release notes or migration guide, it's already decided on a manual approach — read package.json, work out what needs changing, apply edits. It fetches docs to confirm versions, not to discover an approach. The 1.22.0 release notes page links to the Gulp-to-Heft migration guide. When an agent reads the 1.22.0 page, it follows that link because a detailed migration guide aligns with its pre-formed manual plan. Once on the migration page, the detailed step-by-step instructions confirm the manual approach and the agent ignores the CLI recommendation from the release page — we tested this and upgrade quality was significantly worse than when the agent used the CLI. This is why the changes in this PR work as a combined intervention. The WARNING challenges the agent's pre-formed manual plan. The TIP gives it a concrete alternative. And the conceptual migration page reinforces the CLI recommendation when the agent follows the link, instead of providing manual steps that let it revert to its original plan. The TIP wording matters too. Framing the CLI as optional ("You can use...") scored 0/5 across 7 scenarios (35 runs). When the agent sees two optional approaches, it picks the one that aligns with its pre-formed manual plan — every time. Why the CLI produces better upgrade outcomesThe CLI computes the full delta for an upgrade: it detects the source version, calculates every change across each intermediate version, and produces a complete, project-specific report. Given the prompt "upgrade to v1.22.2", agents consistently fetch two release notes pages — DecisionThe core question is: should SPFx upgrades performed by agents be driven by manual steps from documentation, or by the CLI? Based on our results:
ActionsDone
Needs decisionIf the decision is CLI-driven upgrades: the PR as-is is the right shape. @andrewconnell's concern about companies that can't use open-source tools is valid — the manual steps could live on a separate reference page rather than being removed entirely, so they're available without competing with the CLI recommendation on the migration page. If the decision is documentation-driven upgrades: the PR changes to the migration page and TIP wording should be reverted, and the investment shifts to making the docs comprehensive enough to cover multi-version upgrade paths — covering all incremental changes and guiding agents through chaining version-specific docs together. |
- add manual migration article for gulp to heft toolchain - link manual guide from automated migration article - soften contraction wording in migration guidance - reword section heading and script description
andrewconnell
left a comment
There was a problem hiding this comment.
Duped old page migration with manual steps & added as footnote on to the original page updated.
|
Learn Build status updates of commit 89e9e4c:
|
| File | Status | Preview URL | Details |
|---|---|---|---|
| docs/spfx/release-1.15.2.md | View | Details | |
| docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain-manual.md | View | Details | |
| docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md | View | Details | |
| docs/spfx/release-1.22.0.md | 💡Suggestion | View | Details |
| docs/spfx/release-0.0.0.md.template | ✅Succeeded | ||
| docs/spfx/release-1.0.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.1.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.10.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.11.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.12.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.13.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.13.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.14.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.15.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.16.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.16.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.2.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.3.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.4.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.2.md | ✅Succeeded | View | |
| docs/spfx/release-1.19.0.md | ✅Succeeded | View |
This comment lists only the first 25 files in the pull request.
docs/spfx/release-1.15.2.md
- Line 64, Column 44: [Warning: bookmark-not-found - See documentation]
Cannot find bookmark '#requesting-permissions-to-azure-ad-applications-in-another-tenant' in 'spfx/use-aadhttpclient.md', did you mean '#requesting-permissions-to-entra-id-applications-in-another-tenant'?
docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain-manual.md
- Line 7, Column 1: [Warning: duplicate-h1s - See documentation]
H1 'Migrate from the Gulp-based to the Heft-based Toolchain' is duplicated with other articles: 'spfx/toolchain/migrate-gulptoolchain-hefttoolchain-manual.md(7,1), spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md(7,1)'. First level headings must be unique within a docset. - Line 0, Column 0: [Suggestion: filename-incomplete - See documentation]
The filename contains an incomplete word, misspelled word, acronym, or abbreviation that is disallowed.
docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md
- Line 7, Column 1: [Warning: duplicate-h1s - See documentation]
H1 'Migrate from the Gulp-based to the Heft-based Toolchain' is duplicated with other articles: 'spfx/toolchain/migrate-gulptoolchain-hefttoolchain-manual.md(7,1), spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md(7,1)'. First level headings must be unique within a docset.
docs/spfx/release-1.22.0.md
- Line 12, Column 3: [Suggestion: docs-link-absolute - See documentation]
Absolute link 'https://learn.microsoft.com/javascript/api/overview/sharepoint?view=sp-typescript-latest' will be broken in isolated environments. Replace with a relative link. - Line 12, Column 3: [Suggestion: preserve-view-not-set - See documentation]
You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL. Otherwise, remove the view parameter. URL: https://learn.microsoft.com/javascript/api/overview/sharepoint?view=sp-typescript-latest
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
Added AI usage note to migration documentation.
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit cd534ba:
|
| File | Status | Preview URL | Details |
|---|---|---|---|
| docs/spfx/release-1.15.2.md | View | Details | |
| docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain-manual.md | View | Details | |
| docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md | View | Details | |
| docs/spfx/release-1.22.0.md | 💡Suggestion | View | Details |
| docs/spfx/release-0.0.0.md.template | ✅Succeeded | ||
| docs/spfx/release-1.0.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.1.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.10.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.11.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.12.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.13.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.13.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.14.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.15.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.16.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.16.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.2.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.3.md | ✅Succeeded | View | |
| docs/spfx/release-1.17.4.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.0.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.1.md | ✅Succeeded | View | |
| docs/spfx/release-1.18.2.md | ✅Succeeded | View | |
| docs/spfx/release-1.19.0.md | ✅Succeeded | View |
This comment lists only the first 25 files in the pull request.
docs/spfx/release-1.15.2.md
- Line 64, Column 44: [Warning: bookmark-not-found - See documentation]
Cannot find bookmark '#requesting-permissions-to-azure-ad-applications-in-another-tenant' in 'spfx/use-aadhttpclient.md', did you mean '#requesting-permissions-to-entra-id-applications-in-another-tenant'?
docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain-manual.md
- Line 7, Column 1: [Warning: duplicate-h1s - See documentation]
H1 'Migrate from the Gulp-based to the Heft-based Toolchain' is duplicated with other articles: 'spfx/toolchain/migrate-gulptoolchain-hefttoolchain-manual.md(7,1), spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md(8,1)'. First level headings must be unique within a docset. - Line 0, Column 0: [Suggestion: filename-incomplete - See documentation]
The filename contains an incomplete word, misspelled word, acronym, or abbreviation that is disallowed.
docs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md
- Line 8, Column 1: [Warning: duplicate-h1s - See documentation]
H1 'Migrate from the Gulp-based to the Heft-based Toolchain' is duplicated with other articles: 'spfx/toolchain/migrate-gulptoolchain-hefttoolchain-manual.md(7,1), spfx/toolchain/migrate-gulptoolchain-hefttoolchain.md(8,1)'. First level headings must be unique within a docset.
docs/spfx/release-1.22.0.md
- Line 12, Column 3: [Suggestion: docs-link-absolute - See documentation]
Absolute link 'https://learn.microsoft.com/javascript/api/overview/sharepoint?view=sp-typescript-latest' will be broken in isolated environments. Replace with a relative link. - Line 12, Column 3: [Suggestion: preserve-view-not-set - See documentation]
You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL. Otherwise, remove the view parameter. URL: https://learn.microsoft.com/javascript/api/overview/sharepoint?view=sp-typescript-latest
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
Summary
Updates all SPFx release notes pages and the Gulp-to-Heft migration guide to improve upgrade guidance by directing developers to use CLI for Microsoft 365.
Changes
All release pages (40 files)
package.jsonalone will cause build failures.npxcommand to runm365 spfx project upgrade --toVersion <version>for each release page.pnp-o365clisnippet) clarifying CLI for Microsoft 365 is open-source with no Microsoft SLA.spfx-release-upgrade-tip.md→spfx-release-upgrade-warning.mdto reflect its new content.docs/spfx/release-1.22.0.mddocs/spfx/toolchain/migrate-gulptoolchain-hefttoolchain.mdm365 spfx project upgradeto generate the full set of changes automatically.