There is a lot of controversy surrounding the use of AI in writing, and a huge discussion going on right now in one of my Facebook writing groups about whether you can consider yourself a writer if you use AI. Then there is the use of AI detectors to determine whether your writing is genuine. Two months ago, an AI detector flagged an article of mine from 2009. Since I knew it wasn’t in fact, AI generated I wondered just how well the detectors work?
So out of pure boredom and maybe a small pinch of curiosity, I put together a little test. First, I had ChatGPT generate some writing for an article that I am working on for my blog “Behind the Crime” and then I typed out a paragraph from the book “On Writing” by Stephen King. The first detector I used didn’t flag King’s book as AI-generated, but the next two detectors did. They also flagged the AI-written work as AI generated but look at the screenshots below. The first is King’s writing, and the second is my Chat GPT generated text.
Text Guard listed King’s writing as 88% AI generated, while it listed my AI-generated writing as only 91% AI generated. I also posted the same two examples in another AI detector, NoteGPT, which showed King’s writing as 0% AI-generated and my AI-generated writing as being 77% probability AI-generated.
Undetectable AI showed that King’s writing was 99% human, and the ChatGPT writing was 31% human, but it flagged just 12 sentences as AI, implying a human wrote the remaining document, although AI fully authored it.
I cut down to the first paragraph and cut out the highlighted text, and the same program suggested the text that was completely AI-generated had 0 AI sentences, though it only increased to a 69% probability that a human wrote it. Keep in mind that this was completely 100% AI-generated.
Do you have an AI detector that you think is more accurate? What about using AI in your writing? Is it a good idea or a bad idea?
