Lossy compression, don’t mind.
It’s humorous, but last I checked, the best general-purpose compressors with the highest levels of compression—even lossless, which is probably not what most people think of when they think of neural nets—are neural net based.
Neural net-based compressors are computationally expensive, which is why we don’t normally use them for most day-to-day tasks, but they really can produce really small outputs.
I’m going to take the text of the US Constitution and stick it in a text file.
$ wget https://www.gutenberg.org/cache/epub/5/pg5.txt $ stat -c %s pg5.txt 48326Okay, so 48326 bytes.
Let’s do lzo. You’d expect a limited amount of compression — LZO is “fast” compression, usually only used where compression speed is really important, like where you want to be compressing stuff that’s going to be decompressed once and your bottleneck is throughput to disk:
$ lzop <pg5.txt >pg5.txt.lzo $ stat -c %s pg5.txt.lzo 24843Okay, how about gzip? That’s Deflate, an older, but pretty-widely-used general-purpose compression algorithm.
$ gzip <pg5.txt >pg5.txt.gz $ stat -c %s pg5.txt.gz 16660Okay, what about LZMA? That’s a newer, more-CPU-intensive thing that’s probably a good general-purpose choice that’ll generally give better compression ratios. It’s the kind of thing that I’d probably use in a lot of cases. (Personally, these days, I tend to use
pixz, which provides both indexed access for tarballs and parallel compression and decompression, which is important for modern processors.)$ xz <pg5.txt >pg5.txt.xz $ stat -c %s pg5.txt.xz 15488Okay, now PAQ, a neural-net-based compressor:
$ zpaq a pg5.txt.zpaq a pg5.txt -method 5 $ stat -c %s pg5.txt.zpaq 13063“Neutral net based” compression isn’t even in the same universe as “compressed to a prompt” via LLM
It actually is. I mean, it’s building a dictionary off of a variety of content ahead-of-time, rather than training it on the specific item in question, but that’s not uncommon for non-general-purpose compressors.
I mean, doing so to a (probably short) prompt is (a) lossy (and I gave a lossless example) and (b) lossy to an extreme degree, to where it’s probably not incredibly useful option for the kinds of systems that exist today.
But…existing diffusion models aren’t actually intended for this, either. I’d bet that you could train a model to do image compression along these lines, with a large dictionary, that could do usable compression along the lines of what is (jokingly) described in the article. Probably have a larger compressed form than what they’re thinking of.
EDIT: At one point in time, about over a quarter-century ago now, I went out and banged on a neural net post-processor for JPEG artifacts. The idea here is that JPEG very probably isn’t optimally representing the final image, as a human, using their knowledge of what the world looks like, can manually (if time-consumingly) clean these up. I didn’t meet with a lot of success; I only wanted to put a small amount of time into it, and I was working with much weaker hardware than people are running neural nets on today. But that generated a pre-existing dictionary, a pre-trained neural net, off a training corpus of uncompressed images. It didn’t try to reconstruct the image from scratch, the way something like this would, just clean up artifacts, but it has that same pre-generated neural net approach.
I did not know about PAQ actually, thats cool. :)
laughs when reading this
cries when paying thousands of dollars for DLSS
I suspected everything the moment I read 13-year-old-boy
This is not funny, you don’t understand how long and how many nights i spend overengineering compression pipelines for family photos and videos…
Then I bet your stuff is actually good.
I’ve got mine down to 32 byte string to describe the location, datetime, and people in the photograph, with flags for who’s smiling and/or blinking.
I want to see your porn metadata.
Absolutely debauched. Especially when you haven’t even bought them coffee yet.
brilliant satire that critiques all of magazine journalism.
i love going to [insert publication here] and reading another article about ‘so and so is ready for their next chapter in life.’ the so and so always an uninteresting, overly wealthy fuckwit that hasn’t accomplished anything other than going to college and having a wealthy parent.
The sad thing is that this has been possible for decades using convolutional autoencoders, but with LLMs we forgot that AI architectures other than transformers still exist.
Yeah, it’s really awful. With any luck, AI winter will follow AI summer, like usual, and the serious people can come out again. Although, aren’t CNNs more of a this century thing? I guess two decades is still decades…
IIRC autoencoders actually produce the same image to within our ability to notice, as well.
New Math and vibe coding. Am I right? (insert canned laughter here).
🤢
That’s … redefining the word I would think.
“First, it uses AI”
Nope. Stopped reading. Don’t care.
Looks like someone got Onioned.
Shrug. Still don’t care.
Perhaps you’re not in the right community then, this one is for humor not serious discussion.
Remember that Sloot thing from late 90s?
No, what was it?
found it - here’s a detailed account https://corecursive.com/sloot-digital-coding-system/
Very interesting piece of history. Thank you!
you’re welcome
Wow that was quite the read, thanks!
so basically there was a dutch tech bro in the late 90s who claimed he had developed a compression technique that could turn a movie into just a couple of kilobytes and right when he was going to sign with some investors he mysteriously passed away leaving no documents explaining his invention.
why would someone kill him over it? what do you think?
Not saying that’s what happened, but if I ever find myself in a similar situation, I’m faking my death and move halfway across the world in heartbit. After I get the money of course
most likely he just overstressed because he sold folks a bill of goods
Yup. It seems a lot more likely than solving a problem that we still can’t solve, and that might not even be possible, in the 90’s.
Lossless -> lossy -> mindless.
-> mindy?
Probably not very.
mindly?
This is so sad, Chat GPT generate me an email responding to this article
