RSS Bot@lemmy.bestiver.seBM to Hacker News@lemmy.bestiver.seEnglish · 11 hours agoLaya (OS Jev) on Mac M4 CoreML Offline (45 decisions per second)gist.github.comexternal-linkmessage-square1linkfedilinkarrow-up11file-text
arrow-up11external-linkLaya (OS Jev) on Mac M4 CoreML Offline (45 decisions per second)gist.github.comRSS Bot@lemmy.bestiver.seBM to Hacker News@lemmy.bestiver.seEnglish · 11 hours agomessage-square1linkfedilinkfile-text
minus-squaremira_tokarevlinkfedilinkEnglisharrow-up1·6 hours agoWhether hosted Jev or on-device Laya, high-frequency decisions amplify whatever you put in state. Before optimizing another decisions/sec, log tokens(state) separately and run a boring deterministic Tier-1 shrink. Complementary to the model itself — not a replacement. contextpress: https://github.com/Taha-azizi/contextpress · https://pypi.org/project/contextpress/
Whether hosted Jev or on-device Laya, high-frequency decisions amplify whatever you put in
state.Before optimizing another decisions/sec, log tokens(state) separately and run a boring deterministic Tier-1 shrink. Complementary to the model itself — not a replacement. contextpress: https://github.com/Taha-azizi/contextpress · https://pypi.org/project/contextpress/