feat(backend): group file sharing over MLS - #532
Open
Olorunfemi20 wants to merge 3 commits into
Open
Conversation
feat(backend): MLS group state recovery and new-device join
A newly-linked device now joins existing groups through a proposal/commit
and receives every epoch from its join onwards. Group messages from before
its join surface as unavailable placeholders rather than as errors.
Schema:
- mls_groups holds the public group identifiers and current epoch, one row
per group conversation
- mls_group_members records a device leaf as an epoch interval
[joinedAtEpoch, removedAtEpoch) rather than a boolean, so a device
decryption window is derivable and a re-added device does not regain the
epochs it was absent for
- mls_commits is the append-only epoch log a device replays to catch up
- mls_welcomes holds Welcome messages for devices that are offline when
they are added
- messages.mls_epoch records which epoch encrypted a group message
Routes under /conversations/:id/mls:
- POST /group records group identifiers, seats the founder at 0
- GET /group epoch, membership window, pending Welcome and the
epoch this device history starts at
- GET /pending-devices member devices with no leaf yet
- POST /commits publishes a commit plus its adds, removes and
Welcomes; epoch must be currentEpoch + 1 and the
group row is locked so a concurrent commit loses
with 409 instead of both applying
- GET /commits replay, clamped to the device join epoch
- GET /welcome claims the pending Welcome
Read paths:
- GET /conversations/:id/messages and the socket message_history event both
blank the ciphertext of messages outside the device epoch window and tag
them with unavailable plus a reason code, preserving sender and timestamp
so the client renders a placeholder in position
- the send path accepts mlsEpoch; an MLS group message carries one group
ciphertext instead of per-device envelopes, so the envelope requirement
and the sibling-coverage check do not apply to it
Adds docs/mls-group-membership.md documenting the join flow, the endpoints
and the no-history-for-new-device property.
@
feat(backend): group file sharing over MLS A file shared with a group is encrypted once, and the random file key that protects it is distributed inside the MLS group message that references the file. Every member derives the same key from group state, the server never holds one, and object storage keeps a single ciphertext instead of one copy per recipient device. - GET /files/:fileId now gates MLS group files on the requesting device holding an epoch window that covers the reference, using the same reason codes as the message history path. A device removed at epoch M keeps what it could already decrypt and loses everything shared from M onwards, so the server stops handing out ciphertext whose key was only distributed post-rekey - access considers every message referencing the file, so re-sharing a file into a later epoch is how a group deliberately makes an older file readable to members who joined since; the response reports which epoch granted access - POST /uploads requires the uploading device to hold an active leaf when the conversation has an MLS group, and returns the current epoch so the client knows which epoch to encrypt the file key to - non-MLS conversations are untouched: the group lookup is skipped and the response carries a null epoch Adds docs/mls-group-files.md covering the single-ciphertext model, the access rules, and the removed-member behaviour.
|
@Olorunfemi20 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #371
Summary
A file shared with a group is encrypted once, and the random file key that protects it is distributed by putting it inside the MLS group message that references the file. Every member derives the same key from group state, the server never holds one, and object storage keeps a single ciphertext instead of one copy per recipient device.
This reuses the existing file subsystem — presigned upload,
filesrow, presigned download — and changes only who is allowed to fetch the ciphertext.Why one ciphertext
The alternative is encrypting the file separately for each recipient device. A 30-person group with 3 devices each means 90 copies of the same bytes, and another copy every time someone links a device.
MLS already solves the key-distribution half: a message encrypted to the group's epoch secrets is readable by exactly the devices in the tree at that epoch. Putting the file key in such a message gives every member the key without the server learning it, and storage holds one object.
What changed
Nothing in the schema. The file key was already never a column — the existing comment on
filessays it "lives exclusively inside the E2EE envelope ciphertext".messages.mls_epoch(from #372) on the referencing message is what ties a file to an epoch, so no epoch column is needed onfileseither.GET /files/:fileIdAccess now requires both:
The second rule mirrors the message read path exactly. A device that could not decrypt the message carrying the file key has no use for the ciphertext, so the server does not hand it out. Denials use the same reason codes as the history endpoint:
{ "error": "This device has no key for this file", "reason": "mls_no_key_before_join" }Removed members lose access going forward. The commit that removes a member rekeys the group, so every file key distributed from that epoch on is encrypted to secrets the removed device does not have. The epoch check makes the server refuse the ciphertext too, closing the gap between "cannot decrypt" and "cannot download".
Files shared before the removal are still served while the device's user remains a conversation member. That device already held those file keys — withholding bytes it has already been able to read is theatre, not forward secrecy. Removing the user from the conversation cuts off everything, via the membership check that was already there.
Re-sharing an old file to newer members. A file can be referenced by more than one message, and re-sending it in a later epoch is how a group deliberately makes an older file available to members who joined since. Access is therefore granted if any reachable reference falls inside the device's window, and the response reports which epoch granted it:
{ "url": "https://...", "mlsEpoch": 6 }When nothing is readable, the denial reports the reason from the earliest reference, so the message is about the original share rather than an incidental re-share.
The route previously used
findFirston the referencing message, which picked an arbitrary one; it now loads all references, which is what makes the re-share case correct.POST /uploadsRequires the uploading device to hold an active leaf when the conversation has an MLS group. A device that cannot send into the group cannot distribute a file key either, so letting it claim a slot would only create an orphaned object.
The response gains
mlsEpoch— the group's current epoch — so the client knows which epoch to encrypt the accompanying message to:{ "fileId": "uuid", "uploadUrl": "https://...", "mlsEpoch": 7 }DMs and non-MLS conversations
Unchanged. When a conversation has no MLS group the epoch lookup is skipped entirely and
mlsEpochcomes backnull. A file with any non-MLS reference also bypasses the gate, so a file shared into both a DM and a group stays reachable from the DM.Tests
src/__tests__/mlsGroupFiles.test.ts— 18 cases:uploads.test.tsgains a stub for the group lookup and adeviceIdon its auth double — the route now consults MLS state, so its db double had to provide it. All existing assertions in that file are unchanged and still pass.Suite goes from 446 to 464 passing on top of #531. The three files that fail (
file.messages,presence.typing,gateway.integration) already fail onmainbecauseweb-push,socket.io-clientandioredis-mockare not installed in the local workspace; the failure set is unchanged.eslint srcreports no errors.Docs
apps/backend/docs/mls-group-files.mdcovers the single-ciphertext rationale, the full send/receive flow, the access rules with reason codes, the removed-member semantics and why pre-removal files stay readable, re-sharing, and thumbnails.