Kind of. It’s more complicated (for example in 5.3 of the paper it discussed how a little bit of AI generated data mixed with new human data actually improved outputs over only human data).
Under the hood it has to do with sample diversity. The more apt comparison than Xerox (where it’s lowering quality because of necessary fidelity loss) is genetic reproduction.
Even if you have great genes, after a few generations of sex with siblings you’re going to end up with messed up kids.
But if you have great genes, a small degree of over-representation of your genes in a larger mixed gene pool would be better than only new random genes.
This is basically saying that AI models shouldn’t have incest levels of recursion moreso than it is saying that they shouldn’t have ANY recursive data (which would be the case if it worked like a Xerox).
I can’t believe my second comment on Lemmy is gonna be about incest.
If you only have great genes, multiple generations of sister-wives will produce children with those exact same great genes. The problem with incest is that if you carry alleles for recessive disorders (and most people do), inbreeding makes it more and more likely that two copies of the recessive gene will be inherited and expressed since family members generally carry the same recessive genes. That’s why banging strangers is generally a good idea, since they usually carry a different set of recessive disorders than you do.
If there were a brother and sister (or any pairing) with a pristine genetic code, then as long as they remained inbred the first birth defect or genetic disorder to affect their family line will be a completely novel random mutation that formed as a result of pure time and chance over dozens or hundreds of generations. It’s also why inbreeding is a standard tool for animal and plant husbandry.
This is effectively the same issue as what’s going on in the paper and why I used it as an analogy.
Much like how maladaptive genes can piggyback on good genes, but then become overrepresented in an endogenous sample pool, small errors in the diffusion model end up exacerbated through subsequent generations without enough difference in ‘genes.’
There’s definitely good ‘genes’ in the diffusion model, but it’s not the frequency or abundance of the good genes that’s at issue, but the frequency of maladaptive traits in subsequent generations. Much like the issues with human reproduction.
Right, but the primary difference is that the AI is both creating errors and magnifying them in a horrifying Cronenberg feedback loop, where incest doesn’t actually introduce errors.
That said, there’s a known trait called inbreeding depression where fitness is reduced as a result of repeated inbreeding, however it can result is purifying selection that removes deleterious genes and recessive alleles that are unmasked by the inbreeding and actually increase fitness. If they could adapt some sort of testing algorithm to prevent rampancy maybe they could “breed” diffusion algorithms or just curtail the outputs of the current ones.
Though there’d probably be some strange feedback loops if it was set up as two adversarial models where one is trained to slap down weird outputs and the other is trained to adapt to rejected outputs.
Well, the ideal would probably be to train a discriminator based on human ratings of generated outputs.
Take generation 0 (G0), produce output which is accepted or rejected based on humans, train a discriminator to predict those ratings off output, and then use the combined accepted outputs from humans and trained discriminator to train G1.
Repeat again for G1, G2, G3, etc.
My guess would be that the end result would continue to get better and better rather than worse.
The problem is if the diffusion model can’t properly reject weird hands or pupils, those magnify in subsequent rounds.
But there’s likely adaptive and maladaptive tendencies in the diffusion model, and adding a halfway decent filter between human selection and synthetic selection of outputs separate from the diffusion model itself would effectively curb the magnification here.
It seems like a simple enough fix, though also setting a weird precedent. Instead of directly fixing things, just keep adding layers of machine learning to produce improved outputs.
The future of AI isn’t spaghetti code, but spaghetti AI chains lol. Probably why people much smarter than me are the ones working on machine learning.
Kind of. It’s more complicated (for example in 5.3 of the paper it discussed how a little bit of AI generated data mixed with new human data actually improved outputs over only human data).
Under the hood it has to do with sample diversity. The more apt comparison than Xerox (where it’s lowering quality because of necessary fidelity loss) is genetic reproduction.
Even if you have great genes, after a few generations of sex with siblings you’re going to end up with messed up kids.
But if you have great genes, a small degree of over-representation of your genes in a larger mixed gene pool would be better than only new random genes.
This is basically saying that AI models shouldn’t have incest levels of recursion moreso than it is saying that they shouldn’t have ANY recursive data (which would be the case if it worked like a Xerox).
I can’t believe my second comment on Lemmy is gonna be about incest.
If you only have great genes, multiple generations of sister-wives will produce children with those exact same great genes. The problem with incest is that if you carry alleles for recessive disorders (and most people do), inbreeding makes it more and more likely that two copies of the recessive gene will be inherited and expressed since family members generally carry the same recessive genes. That’s why banging strangers is generally a good idea, since they usually carry a different set of recessive disorders than you do.
If there were a brother and sister (or any pairing) with a pristine genetic code, then as long as they remained inbred the first birth defect or genetic disorder to affect their family line will be a completely novel random mutation that formed as a result of pure time and chance over dozens or hundreds of generations. It’s also why inbreeding is a standard tool for animal and plant husbandry.
This is effectively the same issue as what’s going on in the paper and why I used it as an analogy.
Much like how maladaptive genes can piggyback on good genes, but then become overrepresented in an endogenous sample pool, small errors in the diffusion model end up exacerbated through subsequent generations without enough difference in ‘genes.’
There’s definitely good ‘genes’ in the diffusion model, but it’s not the frequency or abundance of the good genes that’s at issue, but the frequency of maladaptive traits in subsequent generations. Much like the issues with human reproduction.
Right, but the primary difference is that the AI is both creating errors and magnifying them in a horrifying Cronenberg feedback loop, where incest doesn’t actually introduce errors.
That said, there’s a known trait called inbreeding depression where fitness is reduced as a result of repeated inbreeding, however it can result is purifying selection that removes deleterious genes and recessive alleles that are unmasked by the inbreeding and actually increase fitness. If they could adapt some sort of testing algorithm to prevent rampancy maybe they could “breed” diffusion algorithms or just curtail the outputs of the current ones.
Though there’d probably be some strange feedback loops if it was set up as two adversarial models where one is trained to slap down weird outputs and the other is trained to adapt to rejected outputs.
Well, the ideal would probably be to train a discriminator based on human ratings of generated outputs.
Take generation 0 (G0), produce output which is accepted or rejected based on humans, train a discriminator to predict those ratings off output, and then use the combined accepted outputs from humans and trained discriminator to train G1.
Repeat again for G1, G2, G3, etc.
My guess would be that the end result would continue to get better and better rather than worse.
The problem is if the diffusion model can’t properly reject weird hands or pupils, those magnify in subsequent rounds.
But there’s likely adaptive and maladaptive tendencies in the diffusion model, and adding a halfway decent filter between human selection and synthetic selection of outputs separate from the diffusion model itself would effectively curb the magnification here.
It seems like a simple enough fix, though also setting a weird precedent. Instead of directly fixing things, just keep adding layers of machine learning to produce improved outputs.
The future of AI isn’t spaghetti code, but spaghetti AI chains lol. Probably why people much smarter than me are the ones working on machine learning.