Skip to content

Implement quota fetch all mode and use actual ranged query#76

Draft
rustybee42 wants to merge 6 commits into
mainfrom
hackathon/quota
Draft

Implement quota fetch all mode and use actual ranged query#76
rustybee42 wants to merge 6 commits into
mainfrom
hackathon/quota

Conversation

@rustybee42

@rustybee42 rustybee42 commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator
  • When no ids to query are configured, uses the previously unused All fetch mode.
  • Actually use the ranged query when a range is configured (instead of combining it into a list). Does a separate request to still allow combination with the list query (from the file and system ids) if configured.
  • Refactoring

With this change, quota performance and user experience should significantly improve. It allows the storage server to use incremental enumeration for ranges and the new "all/automatic" mode, querying only entries that actually exist. The old list query remains, querying exactly the ids specified. The new, default automatic mode doesn't need any configuration, no more id files updated by cronjobs needed.

This should be widely compatible with non-updated storage nodes: The range query handler was implemented before, and "all" would just return nothing. Also the other way around: Previous management always queried by list, which remains intact on the patched storage servers.

There is only one little issue (not breaking change): For ZFS, it depends on the version of installed library on whether the incremental enumeration works or not. While there can be a fallback for the range mode (just use the old "try each id in range"), we can't do that for the "all" mode. In that case, the request will fail and error out. Which should be fine, since all mode is new. Note that for ext4 and xfs, this doesn't really apply because it comes with the kernel since 4.6, which includes all our supported kernels.

#72 might not be necessary in this form anymore. Maybe with an only slightly increased timeout.

Related to https://github.com/ThinkParQ/beegfs-core/pull/4756

@rustybee42 rustybee42 self-assigned this Jul 8, 2026
Comment thread mgmtd/src/quota.rs Fixed
Comment thread mgmtd/src/quota.rs Dismissed
@rustybee42 rustybee42 changed the title Implement quota fetch all mode Implement quota fetch all mode and use actual ranged query Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants