codeinabox@programming.dev to Programming@programming.devEnglish · 17 days agoTurn off Cursor, turn on your mindallvpv.orgexternal-linkmessage-square17linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkTurn off Cursor, turn on your mindallvpv.orgcodeinabox@programming.dev to Programming@programming.devEnglish · 17 days agomessage-square17linkfedilink
minus-squarethinkercharmercoderfarmer@slrpnk.netlinkfedilinkarrow-up0·16 days agoYou sent me down a bit of a rabbit hole, but it turned up an interesting answer. Turns out they are nondeterministic, and why they aren’t deterministic is still an open question https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
minus-squareMichal@programming.devlinkfedilinkarrow-up0·16 days agoInteresting, I had assumed that turning down temperature to 0, or hardcoding a seed would make LLM inference deterministic. Especial after watching this video https://youtu.be/J9ZKxsPpRFk
You sent me down a bit of a rabbit hole, but it turned up an interesting answer. Turns out they are nondeterministic, and why they aren’t deterministic is still an open question https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
Interesting, I had assumed that turning down temperature to 0, or hardcoding a seed would make LLM inference deterministic.
Especial after watching this video https://youtu.be/J9ZKxsPpRFk