Add NCCL GIN / symmetric memory tutorial

[d4l3k]

Adds a new unstable tutorial on GPU-Initiated Networking (GIN) with NCCL and PyTorch distributed, via the NCCL backend of torch.distributed._symmetric_memory.

Contents

• Introduction to host-initiated vs GPU-initiated communication and the NCCL 2.28 device API (LSA, Multimem, GIN)
• Enabling the NCCL symmetric memory backend (symm_mem.set_backend("NCCL") / TORCH_SYMMMEM=NCCL), including eager process group init and the warm-up collective requirement
• Complete torchrun-able example of a device-initiated one_shot_all_reduce on symmetric tensors
• One-sided communication with torch.ops.symm_mem.nccl_put_with_signal / nccl_wait_for_signal, plus notes on nccl_put/nccl_get and the handle-level put_signal/wait_signal (NCCL 2.29+)
• Multi-node launch and GIN hardware/software requirements (RDMA NICs, GPUDirect RDMA, DMA-BUF), including the GDAKI and CPU proxy transports
• Writing custom communication kernels in Python with the CuTe DSL over symmetric memory peer buffers (hdl.get_buffer), adapted from the NVIDIA CUTLASS distributed examples
• Pointers to the raw ncclGin C++ device API for extension authors

Since GIN itself is a device-side API not directly exposed in Python, the tutorial is framed around symmetric memory as the user-facing API, with GIN presented as the transport that services window operations across nodes.

All Python API calls in the examples were verified against pytorch/pytorch main (op registrations in nccl_extension.cu, pybind signatures, and patterns from test/distributed/test_nccl.py); the CuTe DSL example is adapted from the official NVIDIA CUTLASS examples/python/CuTeDSL distributed examples.

The tutorial is a static .rst in unstable_source/ (multi-GPU torchrun code cannot execute in the docs build), registered with a card and toctree entry in unstable_index.rst. Validated with make html-noplot (clean build, no new warnings) and lintrunner.

🤖 Generated with Claude Code

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3932

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 1d236c3 with merge base cb473bc ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.