This is effectively the same issue as what’s going on in the paper and why I used it as an analogy.
Much like how maladaptive genes can piggyback on good genes, but then become overrepresented in an endogenous sample pool, small errors in the diffusion model end up exacerbated through subsequent generations without enough difference in ‘genes.’
There’s definitely good ‘genes’ in the diffusion model, but it’s not the frequency or abundance of the good genes that’s at issue, but the frequency of maladaptive traits in subsequent generations. Much like the issues with human reproduction.
Right, but the primary difference is that the AI is both creating errors and magnifying them in a horrifying Cronenberg feedback loop, where incest doesn’t actually introduce errors.
That said, there’s a known trait called inbreeding depression where fitness is reduced as a result of repeated inbreeding, however it can result is purifying selection that removes deleterious genes and recessive alleles that are unmasked by the inbreeding and actually increase fitness. If they could adapt some sort of testing algorithm to prevent rampancy maybe they could “breed” diffusion algorithms or just curtail the outputs of the current ones.
Though there’d probably be some strange feedback loops if it was set up as two adversarial models where one is trained to slap down weird outputs and the other is trained to adapt to rejected outputs.
Well, the ideal would probably be to train a discriminator based on human ratings of generated outputs.
Take generation 0 (G0), produce output which is accepted or rejected based on humans, train a discriminator to predict those ratings off output, and then use the combined accepted outputs from humans and trained discriminator to train G1.
Repeat again for G1, G2, G3, etc.
My guess would be that the end result would continue to get better and better rather than worse.
The problem is if the diffusion model can’t properly reject weird hands or pupils, those magnify in subsequent rounds.
But there’s likely adaptive and maladaptive tendencies in the diffusion model, and adding a halfway decent filter between human selection and synthetic selection of outputs separate from the diffusion model itself would effectively curb the magnification here.
It seems like a simple enough fix, though also setting a weird precedent. Instead of directly fixing things, just keep adding layers of machine learning to produce improved outputs.
The future of AI isn’t spaghetti code, but spaghetti AI chains lol. Probably why people much smarter than me are the ones working on machine learning.
This is effectively the same issue as what’s going on in the paper and why I used it as an analogy.
Much like how maladaptive genes can piggyback on good genes, but then become overrepresented in an endogenous sample pool, small errors in the diffusion model end up exacerbated through subsequent generations without enough difference in ‘genes.’
There’s definitely good ‘genes’ in the diffusion model, but it’s not the frequency or abundance of the good genes that’s at issue, but the frequency of maladaptive traits in subsequent generations. Much like the issues with human reproduction.
Right, but the primary difference is that the AI is both creating errors and magnifying them in a horrifying Cronenberg feedback loop, where incest doesn’t actually introduce errors.
That said, there’s a known trait called inbreeding depression where fitness is reduced as a result of repeated inbreeding, however it can result is purifying selection that removes deleterious genes and recessive alleles that are unmasked by the inbreeding and actually increase fitness. If they could adapt some sort of testing algorithm to prevent rampancy maybe they could “breed” diffusion algorithms or just curtail the outputs of the current ones.
Though there’d probably be some strange feedback loops if it was set up as two adversarial models where one is trained to slap down weird outputs and the other is trained to adapt to rejected outputs.
Well, the ideal would probably be to train a discriminator based on human ratings of generated outputs.
Take generation 0 (G0), produce output which is accepted or rejected based on humans, train a discriminator to predict those ratings off output, and then use the combined accepted outputs from humans and trained discriminator to train G1.
Repeat again for G1, G2, G3, etc.
My guess would be that the end result would continue to get better and better rather than worse.
The problem is if the diffusion model can’t properly reject weird hands or pupils, those magnify in subsequent rounds.
But there’s likely adaptive and maladaptive tendencies in the diffusion model, and adding a halfway decent filter between human selection and synthetic selection of outputs separate from the diffusion model itself would effectively curb the magnification here.
It seems like a simple enough fix, though also setting a weird precedent. Instead of directly fixing things, just keep adding layers of machine learning to produce improved outputs.
The future of AI isn’t spaghetti code, but spaghetti AI chains lol. Probably why people much smarter than me are the ones working on machine learning.