Fitting Four 32 GB Resizable Bars Behind a PCIe Switch on Linux

(doug.sh)

20 points by dougcalobrisi1 hour ago

3 comments

  • kkaske43 minutes ago
    The bottleneck with AI and hardware is really pushing things.<p>Even with six capable GPUs that have plenty of VRAM with enough PCIe bandwidth, you then discover that memory address allocation is preventing the hardware from doing what it&#x27;s physically capable of.
  • dougcalobrisi1 hour ago
    This is my post. Basically, I needed P2P between four R9700s for tensor parallel serving, which means a 32 GB BAR on each card. With all six GPUs in, the BIOS couldn&#x27;t map them, and Linux sized the switch&#x27;s window too small, so only two cards got their BARs. The module in the post works around that. With P2P off, prefill was about half as fast.
  • bombela46 minutes ago
    The word or acronym BAR is mentioned 34 times but never defined.