fix: retry enrollment agent-doc write on ES 409 version conflict

[ycombinator]

What is the problem this PR solves?

During high-load enrollment (e.g. 100k agents), a primary shard relocation on .fleet-agents causes in-flight CREATE bulk operations to fail with a 409 version_conflict_engine_exception. Fleet Server previously had no retry logic for this case and propagated the error directly as HTTP 500 to the enrolling agent.

Observed in a 100k-agent scale test: 661 enrollment requests returned 500 over a ~90-second window following a single shard relocation. p50 request duration was 52s, p99 was 69s (Fleet Server held the connection while retrying the raw ES call rather than resolving the conflict).

How does this PR solve the problem?

Modified createFleetAgent in internal/pkg/api/handleEnroll.go to retry up to three times on a 409 version conflict. Before each retry the function re-reads the agent document from ES to determine what actually happened:

Re-read result	Action
Document absent	Create was not committed on the old primary; retry the Create
Document found, same access_api_key_id	Create was committed before the 409 was reported; return success without a second write
Document found, different access_api_key_id	A concurrent writer created the document; surface the 409 as an error

The updateFleetAgent path (re-enrollment of existing agents) already uses retry_on_conflict: 3 at the Elasticsearch bulk-API level and is unaffected.

How to test this PR locally

Unit tests cover the two new retry paths:

go test ./internal/pkg/api/... -run TestEnrollCreateVersionConflict

• TestEnrollCreateVersionConflict_DocumentNotFound_Retries – 409 on first Create, document absent on re-read → retries and succeeds
• TestEnrollCreateVersionConflict_DocumentFound_SameAPIKey_Succeeds – 409 on Create, document found with matching access_api_key_id → returns success without a second write

Design Checklist

• I have ensured my design is stateless and will work when multiple fleet-server instances are behind a load balancer.
• I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.
• I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool

Related issues

[mergify]

This pull request does not have a backport label. Could you fix it @ycombinator? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.