Improving LLM prefill on the NVIDIA GeForce RTX 4060 mobile

The Acer Nitro V15 has a discrete nvidia GeForxe RTX 4060 mobile. Turns out, the default firmware leaves some performance untapped [GPU].

1. follow the project instructions [GPU] to download, build and install.
2. specifically, the 4060m requires a kernel parameter in grub: “gpu_unlock.targets=0000:01:00.0 gpu_unlock.ad107_experimental=1”

You’ll have to find the exact address with lspci. Yes, it needs the address, the “all” shortcut won’t work. The 4060m is ad107

3. Unfortunately, power management resets the locks. Thus, add a modprobe configuration: “options nvidia NVreg_DynamicPowerManagement=0x00”

4. update-grub and initramfs -u

5. cold-reboot

6. afterwards, “journalctl -k -b | grep gpu_unlock” should show a clear trace that the GPU was unlocker

On the Acer this pays a handsome x2 prefill and +25% decode. Verdict: nice.

Resources

[GPU] GPU unlock ada github project
https://github.com/JustAResearcher/gpu-unlock-ada/

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.