Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Diffusion SFT on (video, prompt) datasets, reusing the PR #63 hook architecture end to end. No sglang engines are involved:
--loss-type sft_lossflips on the existingdebug_train_onlywiring, so placement groups, weight sync, and the driver loop need no changes.How
loss_hub/sft.py:prepare_sft_batchsamples grid sigmas (phase-pure per micro-batch for dual-expert Wan2.2 viacomponent_for_timestep), corrupts cached clean latents withx_t = (1-σ)x₀ + σε, and collates cached cond through the familyTrainPipelineConfig;sft_loss_formulais velocity MSE againstε - x₀. Both plug into the existing--custom-prepare-train-batch-path/--custom-loss-function-pathslots (auto-filled by--loss-type sft_loss).miles/ray/sft_data_manager.py: replacesRolloutManagerbehind the same driver interface.generate(rollout_id)is stateless — per-epoch seeded shuffle, contiguous slice,TrainDataDPSplitter, and a flow-shifted training sigma grid emitted as scheduler meta — so resume needs no cursor state.scripts/sft_encode_wan.py: one-time Ray encode job (UMT5 + Wan VAE from diffusers) producing one.ptper sample with the pair schema the prepare hook consumes.scripts/run-diffusion-sft-wan22.sh: 4-GPU Wan2.2 dual-expert LoRA SFT recipe.--sft-data-path; sft_loss validation rejects RL-only combos (--eval-interval,--diffusion-kl-beta,--diffusion-recompute-old-log-prob,--ref-mode,--ema-shadow,--n-samples-per-prompt != 1) and requires--diffusion-flow-shiftfor the training grid.The rollout-side
data_conversion_hub(expand hook) is intentionally not used: its(samples, rewards)signature is rollout-specific and SFT has neither.Validation
tests/fast/backends/fsdp_utils/test_loss_hub_sft.py(5 tests, pass): corruption/target identity, timestep scaling, dual-expert phase purity, loss values.SftDataManagerexercised directly: shift math (σ(0.5) → 0.7507 at shift 3.0), terminal sigma, DP shard shapes, exact one-epoch coverage, different shuffle across epochs.parse_argsround-trip for the recipe arg set (hooks auto-filled,debug_train_only=True, gbs derivation) plus rejection of each invalid combo.