As I mentioned last week, researchers unveiled an approach to detecting AI called IdeaLens. Here’s the results of putting IdeaLens to the test.

 “Putting IdeaLens to the test” means testing it against both human and AI created documents. I conducted a test back in June of last year on ZeroGPT, which still touts itself as “the most Advanced and Reliable Chat GPT, GPT6 & AI Content Detector” and wrote about it here, proving that it’s not that accurate.

Then, in March of this year, I tested ChatGPT’s own AI detector, Zero GPT Detector (not to be confused with ZeroGPT) and wrote about it here. Again, not great – it told me that my blog post comparing the rise and fall of the BlackBerry to the emergence of LLMs and GenAI in eDiscovery that was totally written by me (I swear!) was 92% likely to be AI-generated. Fail!

As for IdeaLens, the researchers who developed it say it’s a novel approach to detecting AI provenance that largely avoids the shortcomings of other AI detectors – it focuses on whether a paper’s ideas – and not just the prose – came from a human or a machine.

I decided to use the same three examples from the previous two tests in putting IdeaLens to the test, then added a fourth new test. Here are the results. My analysis is in italics below.

Test #1: Content written by me.

For this test, I chose this blog post I wrote three years ago comparing the rise and fall of the BlackBerry to the emergence of LLMs and GenAI in eDiscovery. I loaded in the plain text of the title and almost the entire blog post – down to (but not including) the paragraph beginning with “P.S.”, which seemed like a large clue that this was written by a person. Again, this was totally written by me – no AI. What did IdeaLens say about this test? Here’s a screenshot of the results:

Human ideas, human prose! I am a real boy after all – take that, Pinocchio! 😉 Seriously though, IdeaLens noted that the probability that the ideas were AI – P(AI) – was only 1.6% and the probability that the prose was AI was less than 0.1%. It’s pretty hard to be more human than that!

A couple of comments: 1) IdeaLens provides a progress window that shows the steps it goes through to analyze your text (example below), which enables you to see how long it takes to process the results. Interestingly, the test that took the most time was the test with the least amount of text – test #3 – which took over 3 minutes. 2) IdeaLens also creates a unique link for each analysis, so that you can go back to it. It provides not only the determination seen in the image above, but also the “Idea outline” that it used to help make its decisions. So far, so good!

Here’s the link to the analysis for the first test: https://ideadetector.ai/r/75va5amca9

And here’s an example of the progress window (which is the example for test #2.

Test #2: Content written by NotebookLM from Google (now known as Gemini Notebook)

For this test, I chose the Briefing Document portion of this blog post about DeepSeek that I wrote back in January 2025. This was a post where I loaded 15 articles into NotebookLM and had it generate a briefing document. The text I loaded for this one was the paragraph starting with “DeepSeek: AI’s Sputnik Moment” (but not the info in parenthesis in the first paragraph) down through the “Conclusion” paragraph. As I noted in the post, this text is all AI-generated. This is what IdeaLens said:

Well, at 91.9%, it definitely thinks the prose is AI-generated! Alas, the Ideas are still considered “Human”, but (notably) the threshold is higher for this document. I’m not sure why the threshold is different than the previous document, I have raised the question to the researchers but haven’t heard back yet. Still, IdeaLens considers the probably of the ideas to be AI at 83.8%. That seems pretty AI to me. Threshold aside, this seems like a pretty solid result too.

Here’s the link to the analysis for the second test: https://ideadetector.ai/r/5ayjkj4aee

Test #3: Content written by ChatGPT

As I’ve done on the previous AI-checkers. I wanted to see what happens when I’m putting IdeaLens on something ChatGPT wrote. This was the validation protocol example in the In re Broiler Chicken Antitrust Litigation case from 2018 that I used in the first two tests test (text of the ChatGPT output available here). So, this text is also all AI-generated. What happened here?

Again, the prose came in at 99.5% likely to be AI! Definitely correct. The ideas came in at 87.7% probably of it being AI, but the threshold is 90.1%. So, it looks like it’s a bit of a sliding scale in terms of determining the ideas to be AI – both examples I tested were 100% AI, and IdeaLens stated a high probability that they were, but still ultimately considered them to be human ideas. One thing: as I noted, this test took longer than the other tests, possibly because it contained the least amount of text (only 511 words). I would imagine the less words, the harder it is to make a determination.

Here’s the link to the analysis for the third test: https://ideadetector.ai/r/zzjmf22jv6

Test #4: Content written by me.

Since the results were pretty darn good for the first three tests, I decided to conduct one additional test. This test was on the blog post I published just last week about the “Da Silva Moore of Generative AI”. With several references to “I”, “me” and “my”, it should be obvious (but I’ll state it anyway) that this was totally written by me – no AI.

Human ideas, human prose once again! Both probabilities that it was written by AI are less than 0.1%. Can’t be any more human than that. Notably the two thresholds returned to the numbers on the first test, which was also human text. Again, not sure why the thresholds vary.

Here’s the link to the analysis for the fourth test: https://ideadetector.ai/r/7tsi9jcq2d

Once again, this was, admittedly, not a scientific case of putting IdeaLens to the test. But this is the first AI-checker I’ve tested that actually did a good job of discerning human and AI-generated text.

A couple of final points: 1) While the researchers “trained our detectors on 1M datapoints from the web”, my fourth document analyzed was only the 5,221st text checked so far, so there will undoubtedly be a lot more testing from the general public which will hopefully provide additional insights. 2) There’s a 53-page paper titled IdeaLens: Detecting AI Ideas in Long-form Writing (available here) with a lot more information about their approach and testing.

So, what do you think? Have any of you been putting IdeaLens to the test? If so, what did you find? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using ChatGPT, using the term “robot and a human high-fiving each other”.

Disclaimer: The views represented herein are exclusively the views of the authors and speakers themselves, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.