2026-spring-littleotherut-T2-2-1#186
Open
littleotherut wants to merge 8 commits into
Open
Conversation
…nt and unit tests
…with tensor initialization
chen2021673
reviewed
Jul 15, 2026
|
|
||
| // Process-wide default CPU generator (single shared random state source). | ||
| Generator &MutableDefaultCPUGenerator() { | ||
| static Generator generator(std::make_shared<CPUGeneratorImpl>()); |
Contributor
There was a problem hiding this comment.
CPU只创建了一个进程级 generator，在单进程多线程 DDP 场景下，各 rank 串行消费同一个进程级 RNG，拿到的随机数并不确定。建议在 DDP 模型 to device 后、第一次 forward 前，从 DP group rank 0 原地 broadcast 所有参数。
| for (int64_t i = 0; i < num_elements; ++i) { data[i] = (values[i] < p) ? 0.0f : data[i] * scale; } | ||
|
|
||
| impl->MemcpyAsync(tensor->DataPtr(), data.data(), num_elements * sizeof(float), | ||
| device.IsCPU() ? core::MemcpyKind::kD2D : core::MemcpyKind::kH2D, impl->GetStream(device)); |
Contributor
There was a problem hiding this comment.
这也要SynchronizeStream一下吧
chen2021673
requested changes
Jul 16, 2026
chen2021673
left a comment
chen2021673
left a comment
Contributor
There was a problem hiding this comment.
统一 Generator 抽象、CPU/CUDA 实现、默认 Generator、seed/state 管理、算子接入以及测试框架都已经搭建起来，代码结构也比较清晰。可以继续补齐 CUDA Generator 未完成功能和对应测试。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
2026春大赛训推赛道T2-2-1 中的 Generator 抽象设计。非报名正式参赛选手，最后几天尝试遵循任务要求做了一些很粗糙的设计，做了核心架构抽象和部分后端落地。
报告.pdf