This is definitely alarming and should give every one who believes in the scientific method a sense of foreboding. After all, how can an evidence based approach work if the evidence can be fabricated?
However, I think there is also an opportunity here to use technology to fight problems created by technology. The idea of adversarial machine learning is not new – we already rely on it in fraud protection, for example in detecting fake or synthetic identities involved in payment fraud. If deep learning can create made up facts and entities, deep learning can also be used to detect such made up entities. It is definitely an arms race, but just because it is an arms race does not mean the good side is doomed to failure. We need to arm our selves with the right tools and technologies to fight the “fake” pandemic. It can be done if we give this the proper attention and investment of effort and resources.
In general, and especially in the world of business, I think the time may have come that we need to have a different mindset going forward. “Presume bad, and then strive to prove good.” That means that assume that every putative customer trying to sign up to your service or trying to buy your product is likely malicious and then use data to prove innocence and enable a good experience for the “provably” good customers. Many online business are already seeing fraud attacks out number good customers. (Facebook closed 1.3 Billion fake accounts in 6 months!)
I know this sounds Orwellian, but we are talking of an existential threat to democracies and to scientific thinking!
Anand's musings 