OK everybody is talking about deepseek and I wanted to see for myself.
I am following the note from Xihan Li.
Specs of my desktop.
- CPU(
free -h): 128G - GPU(
lspci | grep -i vga): two RTX A4500, each has 20 GB of GDDR6 memory.
I guess I will try the smallest one (1.58-bit, 131GB).
That was Feb 4th, 2025. Today is Feb 27, 2025, and I have discovered LM Studio, end of story…
OK a few more tips:
- Add the
--no-sandboxflag to bypass the sandbox requirement (use cautiously, as this reduces security) if you run into the SUID sandbox error. - Remember to eject the model when you are done running it to free up the memory.
- It is much slower than the website in my case…
Who would know that one day when you say LM it refers to language models (or large models?) instead of linear models…
Have fun! No more “The server is busy. Please try again later.”