For text, if you have a decent probability distribution of the next piece of text based on the past context, then you can create a much more efficient encoding, so in that case you can create a good lossless compression
decompression would be terribly slow though, and it doesn’t easily extend to images
Are people downvoting just because LLM? It’s basic information theory, i’m not suggesting we start using it, it would be slow to the point of uselessness
Yeah i was just pointing out one case where it can be used for lossless compression, for lossy compression you can use autoencoders and stuff, but they are lossy and less efficient than jpeg and the like
For text, if you have a decent probability distribution of the next piece of text based on the past context, then you can create a much more efficient encoding, so in that case you can create a good lossless compression
decompression would be terribly slow though, and it doesn’t easily extend to images
Are people downvoting just because LLM? It’s basic information theory, i’m not suggesting we start using it, it would be slow to the point of uselessness
To be clear I’m not speculating on something that could potentially be cool, this already exists and is improving
Yeah i was just pointing out one case where it can be used for lossless compression, for lossy compression you can use autoencoders and stuff, but they are lossy and less efficient than jpeg and the like