SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 11 days agoDo you host your own AI?message-squaremessage-square185linkfedilinkarrow-up10file-text
arrow-up10message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 11 days agomessage-square185linkfedilinkfile-text
minus-squaresurewhynotlem@lemmy.worldlinkfedilinkEnglisharrow-up0·11 days agoI have to recommend switching to llamacpp. It’s SO much faster than ollama.
minus-squarecurbstickle@anarchist.nexusMlinkfedilinkEnglisharrow-up0·11 days agoOn the list but haven’t gotten to it yet, but I know I should. I could probably get a bit more out of that box with it, expand the context windows a bit…
I have to recommend switching to llamacpp. It’s SO much faster than ollama.
On the list but haven’t gotten to it yet, but I know I should. I could probably get a bit more out of that box with it, expand the context windows a bit…