Post
175
How many 3090 is too many 3090? Trick question - never enough 🤖.
Working on rewiring for a 5th this weekend so need the quad in 4U to make this happen. Definitely pushing air cooling to its limit but NVLink requires aggressive spacing between the cards.
Fortunately up here in Canada our power is quite cheap, so stacking compute this way works. While being annoying in terms of physical constraint the NVlink bridges really helps squeeze these cards - on smaller models at big batch the difference at -tp 2 can be as much as 50%. It's not bandwidth, it's latency!
Working on rewiring for a 5th this weekend so need the quad in 4U to make this happen. Definitely pushing air cooling to its limit but NVLink requires aggressive spacing between the cards.
Fortunately up here in Canada our power is quite cheap, so stacking compute this way works. While being annoying in terms of physical constraint the NVlink bridges really helps squeeze these cards - on smaller models at big batch the difference at -tp 2 can be as much as 50%. It's not bandwidth, it's latency!
