They’re using domain-specific LLMs to compress narrow-domain data. Their text compression LLM was trained on, and then tested on, legal text and medical text.
There’s no reason one couldn’t apply the domain-specific-compressor idea to a conventional lossless text compressor, essentially moving much of the dictionary from the compressed file to the program itself. I don’t know if anyone’s tried that. I’d like to know how that compares.
Well actually…
https://www.nature.com/articles/s42256-025-01033-7
Lossless compression by LLM
I’d be more interested in lossy compression tho. Few % of image quality loss for multiplying compression ratio is mega useful.
Is it a compression when you need presumably gigabites of a model to reconstruct the data?
It’s like saying rainbow tables are compression.
A roughly 340MB model at fp16 is all it takes to beat xz and zstd by 2x compression on text
Non paywalled
https://arxiv.org/abs/2407.07723
They’re using domain-specific LLMs to compress narrow-domain data. Their text compression LLM was trained on, and then tested on, legal text and medical text.
There’s no reason one couldn’t apply the domain-specific-compressor idea to a conventional lossless text compressor, essentially moving much of the dictionary from the compressed file to the program itself. I don’t know if anyone’s tried that. I’d like to know how that compares.
There’s a number of compression projects working on this, this is the first that comes to mind
https://bellard.org/nncp/
https://bellard.org/ts_zip/