Discussions

• 1 You must be logged in to vote

  🙏

  NVIDIA 5060 incompatibility

  Spartbrine asked Dec 10, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Must the sp_size be equal to the total_gpus in UlyssesSPAttentionHF?

  NiuMa-1234 asked Nov 5, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Large all reduce call before optimizer

  shubhagr-qc asked Sep 25, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Compatibility of Progressive Layer Dropping & Pipeline Parallelism

  benearnthof asked Sep 23, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  The MoE spends too much time on the gate network.

  zhaozheng09 asked Aug 23, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙌

  Interactive diagrams for DeepSpeed

  brovatten started Aug 15, 2025 in Show and tell

  

  

  0
• 1 You must be logged in to vote

  🙏

  How to compute per sample gradient in deepspeed

  vermouthdky asked Aug 15, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Allocating buffers for different tensor sizes

  colorSand asked Jul 24, 2025 in Q&A · Unanswered

  

  

  1
• 6 You must be logged in to vote

  🙏

  Gradient not accumulated across ranks 😭

  yahyunee asked Jul 8, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Multi-gpu in Zero3 mode, no parameter fragmentation is observed

  1227600298 asked Jul 7, 2025 in Q&A · Unanswered

  

  

  1
• 2 You must be logged in to vote

  🙏

  How to obtain all parameters during training?

  qibin0506 asked Jun 23, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  What happened to "Ulysses Sequence Parallelism Plus for HF Transformers"?

  vecorro asked Jun 20, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Why DeepSpeed ZeRO paper says "reduce-scatter (or all-gather) has data movement of Ψ elements"?

  civat asked Oct 1, 2024 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  💬

  Does DeepSpeed allow to use Gloo backend?

  Arya-Mayank started Jun 16, 2025 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  Does DeepSpeed support tensor parallel for training?

  yiliu30 asked Oct 10, 2023 in Q&A · Answered

  

  

  

  

  6
• 1 You must be logged in to vote

  🙏

  Is Autotuning compatible with AutoTP training?

  cynricfu asked May 28, 2025 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Same model with/without deepspeed different results

  BernardoPalmer asked May 30, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  RNG APIs called in Inference engine even when do_sample is passed as False.

  shubhagr-qc asked May 19, 2025 in Q&A · Answered

  

  

  7
• 1 You must be logged in to vote

  🙏

  Which models does AutoTP support for training

  KENTHD8mg asked May 12, 2025 in Q&A · Answered

  

  

  3
• 1 You must be logged in to vote

  💬

  autotuning failed to run: pdsh@GCRSANDBOX324: localhost: ssh exited with exit code 255

  dunalduck0 started Feb 24, 2022 in General

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  Deepspeed: Tensor Parallelism with 2 GPUs Falls Short of Single GPU Performance

  thri5ha asked Apr 24, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  DeepSpeed on Apple Silicon

  minhquoc0712 started Apr 11, 2025 in General

  

  

  0
• 2 You must be logged in to vote

  🙏

  Deepspeed ZeRo3 Error 'RuntimeError: 'weight must be 2-D' Please help

  shing100 asked Oct 18, 2024 in Q&A · Unanswered

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  pipeline parallelism activation memory

  ghadialhajj asked Apr 3, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Scatter-reduce in stage 2

  ghadialhajj asked Apr 2, 2025 in Q&A · Unanswered

  

  

  0