It actually is. I mean, it’s building a dictionary off of a variety of content ahead-of-time, rather than training it on the specific item in question, but that’s not uncommon for non-general-purpose compressors.
I mean, doing so to a (probably short) prompt is (a) lossy (and I gave a lossless example) and (b) lossy to an extreme degree, to where it’s probably not incredibly useful option for the kinds of systems that exist today.
But…existing diffusion models aren’t actually intended for this, either. I’d bet that you could train a model to do image compression along these lines, with a large dictionary, that could do usable compression along the lines of what is (jokingly) described in the article. Probably have a larger compressed form than what they’re thinking of.
EDIT: At one point in time, about over a quarter-century ago now, I went out and banged on a neural net post-processor for JPEG artifacts. The idea here is that JPEG very probably isn’t optimally representing the final image, as a human, using their knowledge of what the world looks like, can manually (if time-consumingly) clean these up. I didn’t meet with a lot of success; I only wanted to put a small amount of time into it, and I was working with much weaker hardware than people are running neural nets on today. But that generated a pre-existing dictionary, a pre-trained neural net, off a training corpus of uncompressed images. It didn’t try to reconstruct the image from scratch, the way something like this would, just clean up artifacts, but it has that same pre-generated neural net approach.
“Neutral net based” compression isn’t even in the same universe as “compressed to a prompt” via LLM
It actually is. I mean, it’s building a dictionary off of a variety of content ahead-of-time, rather than training it on the specific item in question, but that’s not uncommon for non-general-purpose compressors.
I mean, doing so to a (probably short) prompt is (a) lossy (and I gave a lossless example) and (b) lossy to an extreme degree, to where it’s probably not incredibly useful option for the kinds of systems that exist today.
But…existing diffusion models aren’t actually intended for this, either. I’d bet that you could train a model to do image compression along these lines, with a large dictionary, that could do usable compression along the lines of what is (jokingly) described in the article. Probably have a larger compressed form than what they’re thinking of.
EDIT: At one point in time, about over a quarter-century ago now, I went out and banged on a neural net post-processor for JPEG artifacts. The idea here is that JPEG very probably isn’t optimally representing the final image, as a human, using their knowledge of what the world looks like, can manually (if time-consumingly) clean these up. I didn’t meet with a lot of success; I only wanted to put a small amount of time into it, and I was working with much weaker hardware than people are running neural nets on today. But that generated a pre-existing dictionary, a pre-trained neural net, off a training corpus of uncompressed images. It didn’t try to reconstruct the image from scratch, the way something like this would, just clean up artifacts, but it has that same pre-generated neural net approach.