Could you provide some reference code?
Using the trainer, I'm confused by the dataloader and DistributedSampler.
Different ranks in the same sp_group always fail to obtain the same data idx es.
Locke Li
Locke
·
AI & ML interests
None yet
Recent Activity
commented on
an
article
3 months ago
Ultra-Long Sequence Parallelism: Ulysses + Ring-Attention Technical Principles and Implementation
new activity
5 months ago
baichuan-inc/Baichuan-Audio-Instruct:ImportError (vector_quantize) when loading the model
liked
a model
7 months ago
deepseek-ai/DeepSeek-Prover-V2-671B
Organizations
None yet
commented on
Ultra-Long Sequence Parallelism: Ulysses + Ring-Attention Technical Principles and Implementation
3 months ago
ImportError (vector_quantize) when loading the model
3
#2 opened 7 months ago
by
Jeronymous
liked
a
model
7 months ago
commented
a
paper
8 months ago
upvoted
a
paper
10 months ago
liked
a
dataset
over 1 year ago