Yeah I can run any open weight models which leave me enough vram to not crash. But its a bit of a gotcha because you also need enough system ram to load the model. I use it to heavily parallelize training tasks… Honestly, I need to tinker with it more but I’m pretty annoyed at how ollama has gone deep in the paint as basically being a tool for accessing cloud models.
Yeah I can run any open weight models which leave me enough vram to not crash. But its a bit of a gotcha because you also need enough system ram to load the model. I use it to heavily parallelize training tasks… Honestly, I need to tinker with it more but I’m pretty annoyed at how ollama has gone deep in the paint as basically being a tool for accessing cloud models.
Someday TM
Why not llama.cpp? Ollama is just a wrapper over it.