Clive Robinson • July 5, 2023 12:46 PM

@ Bruce, ALL,

Re : Descent into chaos and noise.

“A recent paper showed that using AI generated text to train another AI invariably “causes irreversible defects.””

As I’ve indicated before that is to be expected when you understand how these neural network based systems work.

There is not the space on this blog to go through the maths and the effort to make formula via UTF-8 glyphs is beyond most mortal flesh and blood can stand.

So an analogy instead[1]…

We know that various things like gongs, wine glasses, bottles and certain types of edges can cause musical notes, due to the build up and loss of energy in resonators.

The thing is appart from the repeyative banging on the gong, all of these resonators gain their energy from near random chaotic input.

You can see this with the wet finger on the wine glass rim. If you move your finger too quickly or too slowely then the body of the glass does not resonate. You can calculate the best speed fairly simply, but it’s even simpler just to get a little practice in. //

Well those nueral networks kind of work that way. You give them a stochastic –random– source and the network in effect resonates –parrots– to it which produces the ouput. Whole musical phrases and entire tunes can be held in the weights.

The weights come about by feeding in tunes and finding the errors and feeding the errors back to adjust the weights.

The point is the network can become “tuned” to a type of music or even just a composer. Which means the filter selects out of the random stream characteristics that match the type of music or the composer.

But each output from the network has differences, to the original music based on residual errors in the system. Yes it sounds to us like the type of music or in the style of the composer, but it’s different by those errors.

Feed that error laden output in as training data and the errors will build up over each iteration, as you would expect.

It’s like the “photocopy of the photocopy” or the “boot-leg tape of the boot-leg tape” each generation adds more noise that changes the network.