So now the time has come… In a new paper(https://lnkd.in/eADdp5r6), some colleagues from Standord and Rice University prove what has been bothering me for some time:
๐๐ฟ๐ฎ๐ถ๐ป๐ถ๐ฒ๐ฟ๐ ๐บ๐ฎ๐ป ๐๐ ๐บ๐ถ๐ ๐๐-๐ด๐ฒ๐ป๐ฒ๐ฟ๐ถ๐ฒ๐ฟ๐๐ฒ๐ป ๐ง๐ฟ๐ฎ๐ถ๐ป๐ถ๐ป๐ด๐๐ฑ๐ฎ๐๐ฒ๐ป, ๐๐ฒ๐ฟ๐ฑ๐ฒ๐ป ๐ฑ๐ถ๐ฒ ๐๐ฟ๐ด๐ฒ๐ฏ๐ป๐ถ๐๐๐ฒ ๐๐ฐ๐ต๐น๐ฒ๐ฐ๐ต๐๐ฒ๐ฟ (i.e. they are becoming more and more similar).
This effect is impressively demonstrated in the paper using the example of image generation, but in general this applies to any type of generative AI!
So also, for example, if you train
Chat-GPT with data generated by Chat-GPT…
AI then cannibalizes itself at some point.
We are currently living in an age in which the ratio of generated to real data is still very favorable (AI is only just beginning).
However, this is changing rapidly.
And that means that we will soon have nothing left with which we can train the AIs in a meaningful way (almost all available data is already trained in the large language models anyway).
Because even if new information is constantly being produced – if we cannot distinguish between what is human-generated and what is AI-generated, then we will no longer be able to use anything qualified for training, i.e. our AIs will no longer improve at some point.
๐๐ฒ๐ฟ๐ฎ๐ฑ๐ฒ ๐ณ๐ฬ๐ฟ ๐จ๐ป๐๐ฒ๐ฟ๐ป๐ฒ๐ต๐บ๐ฒ๐ป ๐ถ๐๐ ๐ฑ๐ฎ๐ ๐ฑ๐ฒ๐ฟ ๐ฏ๐น๐ฎ๐ป๐ธ๐ฒ ๐๐ผ๐ฟ๐ฟ๐ผ๐ฟ!
As soon as your company’s employees start using generative AI in an uncontrolled/unguided manner, you are digging your own potential data grave, because at some point you will no longer be able to rely on your data.
And your data is your capital…
๐๐ฎ๐ต๐ฒ๐ฟ ๐บ๐๐๐ ๐ฎ๐ธ๐๐๐ฒ๐น๐น ๐ฑ๐ถ๐ฒ ๐ผ๐ฏ๐ฒ๐ฟ๐๐๐ฒ ๐ฃ๐ฟ๐ถ๐ผ๐ฟ๐ถ๐๐ฎฬ๐ ๐๐ฒ๐ถ๐ป, ๐๐-๐๐ฎ๐๐ฒ๐ป ๐๐ ๐ธ๐ฒ๐ป๐ป๐๐ฒ๐ถ๐ฐ๐ต๐ป๐ฒ๐ป!
Difficult to impossible in public, but fortunately feasible within the company.
If you don’t know how to do this, please contact me!
—
P.S.: Of course, there are also applications where synthetic data is very helpful.
However, these are isolated exceptions and not the general rule.
P.P.S.: here are some of my previous posts on this topic:
https://lnkd.in/eY8rC8C7
https://lnkd.in/e3bcJ_92
https://lnkd.in/efAex_M2
https://lnkd.in/eHZMm6KZ
P.P.P.S.: I generated the cover picture with Midjourney.
Because our Generative AI is still working ๐
