qaz@lemmy.world to Programmer Humor@programming.devEnglish · 30 days agoSeptlemmy.worldimagemessage-square64linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1imageSeptlemmy.worldqaz@lemmy.world to Programmer Humor@programming.devEnglish · 30 days agomessage-square64linkfedilink
minus-squaremcv@lemmy.ziplinkfedilinkarrow-up0·28 days agoBut if that’s how you’re going to run it, why not also train it in that mode?
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up0·28 days agoThat is a thing, and it’s called quantization aware training. Some open weight models like Gemma do it. The problem is that you need to re-train the whole model for that, and if you also want a full-quality version you need to train a lot more. It is still less precise, so it’ll still be worse quality than full precision, but it does reduce the effect.
minus-squaremudkip@lemdro.idlinkfedilinkEnglisharrow-up0·25 days agoIs it, or is it not, AI slop? Why are you using so heavily markdown formatting? That is a telltale sign of an LLM being involved
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up0·25 days agoI am not using an llm but holy bait Hop off the reddit voice
But if that’s how you’re going to run it, why not also train it in that mode?
That is a thing, and it’s called quantization aware training. Some open weight models like Gemma do it.
The problem is that you need to re-train the whole model for that, and if you also want a full-quality version you need to train a lot more.
It is still less precise, so it’ll still be worse quality than full precision, but it does reduce the effect.
Your response reeks of AI slop
4/10 bait
Is it, or is it not, AI slop? Why are you using so heavily markdown formatting? That is a telltale sign of an LLM being involved
I am not using an llm but holy bait
Hop off the reddit voice