Would you like to run 235B parameter data-analytics-id=”inline-link” href=”https://www.tomshardware.com/tech-industry/artificial-intelligence/amd-launches-gaia-open-source-project-for-running-llms-locally-on-any-pc” data-mrf-recirculation=”inline-link” data-before-rewrite-localise=”https://www.tomshardware.com/tech-industry/artificial-intelligence/amd-launches-gaia-open-source-project-for-running-llms-locally-on-any-pc”>LLMs at home, but your lowly $10,000 budget restricts you to “consumer GPUs that can barely handle 70B parameter models”? This was the situation developer David Noel Ng found himself in, until he stumbled across an “obviously fake” Nvidia data-analytics-id=”inline-link” href=”https://www.tomshardware.com/news/nvidia-details-grace-hopper-cpu-superchip-design-144-cores-on-4n-tsmc-process” data-mrf-recirculation=”inline-link” data-before-rewrite-localise=”https://www.tomshardware.com/news/nvidia-details-grace-hopper-cpu-superchip-design-144-cores-on-4n-tsmc-process”>Grace-Hopper platform being sold on Reddit, of all places. Ng took the gamble and, according to his data-analytics-id=”inline-link” href=”https://dnhkng.github.io/posts/hopper/” target=”_blank” data-url=”https://dnhkng.github.io/posts/hopper/” referrerpolicy=”no-referrer-when-downgrade” data-hl-processed=”none” data-mrf-recirculation=”inline-link”>blog post, it paid off royally. He’s managed, with a bit of tinkering and fixing up, to get an data-analytics-id=”inline-link” href=”https://www.tomshardware.com/tag/enterprise” data-auto-tag-linker=”true” data-mrf-recirculation=”inline-link” data-before-rewrite-localise=”https://www.tomshardware.com/tag/enterprise”>enterprise system that…
