Added "View as markdown" button#3161
Conversation
|
ibexa-workflow-automation-1
Bot
requested review from
adriendupuis,
dabrt and
julitafalcondusza
and removed request for
a team
|
@mnocon We could, for example, have icons at the top right, with hover-on labels, wdys? Or, alternatively, put the section with purely technical buttons at the bottom, after the article itself? 
|
|
I've chosen this implementation based on other docs, which is (quickly) becoming the norm. Other pages using this approach of "buttons under h1":
As the number of action grows, they start using dropdowns:
As we only have 3 buttons right now - I'd keep this as is |
adriendupuis
left a comment
adriendupuis
left a comment
There was a problem hiding this comment.
I don't like having the action button bar below the page title, it's like the title should be out of the export or that all section headings should have this bar.
The new lines converted to commas in the table cells are breat, they're not duplicated when already there like in Built-in views' table from https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/commerce/shopping_list/shopping_list_design/#built-in-views / https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/commerce/shopping_list/shopping_list_design/index.md
| <link rel="llms" href="{{ 'llms.txt' | url }}" title="LLM-friendly index"> | ||
| <link rel="llms" href="{{ 'llms-full.txt' | url }}" title="LLM-friendly full content"> |
There was a problem hiding this comment.
Important notice here (for myself and other reviewers), yes, this a .txt file with Markdown in it, not a .md file.
The file names to use are not clear on https://llmstxt.org/
but this is the name formats used by Anthropic on Claude doc:
Interesting… It works better when there is a baseline like on https://docs.stripe.com/get-started/account/set-up than when there isn't one like in https://developers.cloudflare.com/docs-for-agents/ It also seem to work better with a different layout spacing. So I would fix that by isolating a bit more the page title (with actions when available) from the content and get it closer to other page metadata like the breadcrumbs and page pills. |
Summary
|
Changed breadcrumbs to Can you please re-review? |
There was a problem hiding this comment.
For all files in perso: exclude them from the Markdown-generated content (llms.txt, llms-full.txt, page actions)
| The `match` configuration matches both the content type and the identifier of the Content query field. | ||
|
|
||
| Finally, in the template `templates/themes/<my_theme/content_query/blog_posts.html.twig`, render all results of the query: | ||
| Finally, in the template `templates/themes/<my_theme>/content_query/blog_posts.html.twig`, render all results of the query: |
There was a problem hiding this comment.
The lack of closing > broke the Markdown rendering (discovered when checking the output manually)
| </div> | ||
| <div class="notification__image"> | ||
| <img src="images/notification-latest-release.svg" alt="The latest release" /> | ||
| <img src="images/notification-latest-release.svg" alt="" /> |
There was a problem hiding this comment.
Icons don't need alt text
| @@ -0,0 +1,54 @@ | |||
| from llmstxt_preprocess import absolutize_image_urls | |||
There was a problem hiding this comment.
I've asked Claude to add tests for the changes to:
- make it easier to understand the code
- make it easier to fix the edge cases we discover in the future without worrying about regressions (I'm sure I didn't discover all of them)
| exit 1 | ||
| fi | ||
|
|
||
| python-tests: |
There was a problem hiding this comment.
Running tests for the LLM Markdown converter
| @@ -0,0 +1,153 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
One important one for the generated llms.txt file.
Our MkDocs file uses a nested structure (with multiple levels), but the output of this one is a list that goes 1-level deep at most.
Compare these two:
- https://github.com/ibexa/documentation-developer/blob/5.0/mkdocs.yml
- https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/llms.txt
I though about creating a nested structure, but this goes against the "spec":
Zero or more markdown sections delimited by H2 headers, containing “file lists” of URLs where further detail is available
Each “file list” is a markdown list, containing a required markdown hyperlink [name](url), then optionally a : and notes about the file.
and against all the examples available in https://directory.llmstxt.cloud/ - they all use a "flatter" structure than our MkDocs.
| </svg> | ||
| Edit on GitHub | ||
| </a> | ||
| </div> |
There was a problem hiding this comment.
The edit_uri is changed to blob/{ref}/docs in #3280 so this labels don't fully match anymore.
- the labels could be changed to something like "View on GitHub" and "View and eventually edit this page on GitHub" (sadly lose the invitation to edit, but this is my favorite)
- It could stay like this, the main reason to view on GitHub is to consider editing (OK-ish)
edit_uricould be set back to editedit/{ref}/docs(it's a bit stressful to land directly in an editor on a new website)
* Added "View as Markdown" buttons * Review feedback + added tests * Before re-review * View on GitHub
3 participants
This PR adds support for https://llmstxt.org/ (tool helping AI agents navigate webpages), and introduces a Markdown version of the doc.
How it looks:
The "View on GitHub" button is replaced by:
Why Markdown version of the doc, is the content on GitHub not enough?
No, because the macros we use (for example, embedding files) make the content not readable.
Compare:
The output Markdown contains the full code sample, making it easier for the AI agents to understand what's going on.
For more example see the list of cases I had to cover below.
LLMS.txt files
llms.txt file
llms-full.txt file
Pages can be excluded from these lists by marking them with
exclude_from_llmstxt: true(excludes from llms.txt, llms-full.txt, and does not show the "View as Markdown" buttons).The content of these files is autogenerated by
mkdocs buildis run - no manual changes needed.For now, HTML files are excluded from this list - I want to tackle API References in a follow-up.
Notes
Release notes page:
HTML: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/release_notes/ibexa_dxp_v5.0/
Markdown: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/release_notes/ibexa_dxp_v5.0/index.md (supports badges)
Added styling changes to make the title header look the same as on other pages.
Exclusion of Personalization
Personalization pages are excluded from this solution.
Example page: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/personalization/personalization/
Image alt text
AI agents cannot see the images, similarly as people using screen readers.
HTML: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/infrastructure_and_maintenance/request_lifecycle/
Markdown: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/infrastructure_and_maintenance/request_lifecycle/index.md
If an image has alt-text, it's transformed into:
Absolute vs relative urls
I've went with absolute URLs, as these fill safer, but they use more token space of the agents.
Example page with absolute linls: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/discounts/discounts/index.md
Admonition boxes
Converted to blockquotes.
HTML: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/infrastructure_and_maintenance/background_tasks/#start-worker
Markdown: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/infrastructure_and_maintenance/background_tasks/index.md
Tabs
Converted to bolded text (can't use headers as making the headers match the page structure would be a challenge).
Example:
HTML https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/getting_started/requirements/
Markdown: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/getting_started/requirements/index.md
Tables
Tables actually look great 😄 The only problem is tables where a call contains a
|- this is guarded by Markdownlint already.For tables with linebreaks within cells, a comma is used instead.
Examples:
HTML: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/ibexa_products/editions/
Markdown: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/ibexa_products/editions/index.md
With linebreaks:
HTML: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/api/event_reference/content_events/
Markdown: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/api/event_reference/content_events/index.md
Cards macro
Converted to a list of links:
HTML: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/ibexa_products/editions/
Markdown: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/ibexa_products/editions/index.md
Lists numbering
Example: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/infrastructure_and_maintenance/clustering/clustering_with_ddev/index.md
Edition pills
In content (inline):
Mutliple badges: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/commerce/shopping_list/shopping_list_guide/index.md
Single badge:
HTML: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/update_and_migration/from_5.0/update_from_5.0/#form-builder-performance-fix-missing-indexes-on-form-submission-data
Markdown: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/update_and_migration/from_5.0/update_from_5.0/index.md
Release notes: https://ez-systems-developer-documentation--3161.com.readthedocs.build/en/3161/release_notes/ibexa_dxp_v5.0/index.md