What hardware that needs? My issue with running local models was that it’s too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.
Mac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn’t free, but at least it’s just a single upfront payment.
What hardware that needs? My issue with running local models was that it’s too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.
64GB unified memory. I run it (and a lot more) on a dgx spark, but a Mac mini would suffice also.
You could prob run 4-bit version on a RTX card with 32g. Maybe even 24g. Like a 5090 or 4090 or such.
So much info out there.
Mac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn’t free, but at least it’s just a single upfront payment.