GPT-2 was notorious for plagiarizing. GPT-3 is better, from gwern.net's blog post [0] and my limited experience of playing AI Dungeon, it appears to be really good at generating original writings by mixing, matching, and rewriting existing ideas. Sometimes they are verbatim copies, but more often they are rephased. For example, this is the output from GPT-3 after gwern fed a large number of quotes into the system,
> Real is right now, this second. This might seem trivially true, but it is trivial only if you identify the real with the material.
> Cameras may document where we are, but we document what the camera cannot see: the terror we experience in the silence of our eyes.
> No society values truth-tellers who reveal unpleasant truths. Why should we value them when they are our own?
Almost everything written by GPT-3 gives you a perception of "I've read something similar before", and of course, the ideas and meanings must come from pre-existing works. Yet, you often cannot find the original text, it has been rewritten beyond recognition. It's almost like Jean Baudrillard's Simulacra and Simulation [1] - everything is a copy, yet with no original.
> The first stage is a faithful image/copy [...] The second stage is perversion of reality, this is where we come to believe the sign to be an unfaithful copy [...] The third stage masks the absence of a profound reality, where the sign pretends to be a faithful copy, but it is a copy with no original. Signs and images claim to represent something real, but no representation is taking place and arbitrary images are merely suggested as things which they have no relationship to. [...] The fourth stage is pure simulacrum, in which the simulacrum has no relationship to any reality whatsoever. Here, signs merely reflect other signs and any claim to reality on the part of images or signs is only of the order of other such claims.
> Real is right now, this second. This might seem trivially true, but it is trivial only if you identify the real with the material.
> Cameras may document where we are, but we document what the camera cannot see: the terror we experience in the silence of our eyes.
> No society values truth-tellers who reveal unpleasant truths. Why should we value them when they are our own?
Almost everything written by GPT-3 gives you a perception of "I've read something similar before", and of course, the ideas and meanings must come from pre-existing works. Yet, you often cannot find the original text, it has been rewritten beyond recognition. It's almost like Jean Baudrillard's Simulacra and Simulation [1] - everything is a copy, yet with no original.
> The first stage is a faithful image/copy [...] The second stage is perversion of reality, this is where we come to believe the sign to be an unfaithful copy [...] The third stage masks the absence of a profound reality, where the sign pretends to be a faithful copy, but it is a copy with no original. Signs and images claim to represent something real, but no representation is taking place and arbitrary images are merely suggested as things which they have no relationship to. [...] The fourth stage is pure simulacrum, in which the simulacrum has no relationship to any reality whatsoever. Here, signs merely reflect other signs and any claim to reality on the part of images or signs is only of the order of other such claims.
[0] https://www.gwern.net/GPT-3
[1] https://en.wikipedia.org/wiki/Simulacra_and_Simulation