I Tested 3 Literature Review AIs - Only One Didn’t Lie to Me

Key Concepts:

• AI Literature Review Tools
• Hallucination (AI Fabrication of References)
• Manis
• Gen Spark
• Gemini AI (Google)
• Peer-Reviewed Sources
• In-text Citations (IEEE Format)
• Literature Review Structure (Themes, Evolution, Studies, Conflicts, Gaps, Future Research)
• Reference Verification
• Failure Rate

Manis Analysis:

• Process: The presenter used the prompt: "Generate a structured literature review on the topic [specified topic]. Please include the key research themes, how they've evolved, important studies, conflicting viewpoints, gaps in the current literature, suggestions for future research. Please use peer-reviewed sources, present the output in a format suitable for inclusion into the introduction of a thesis, add in-text citations in IEEE format (even if they are placeholders), and make the review as long as it needs to be to cover the literature effectively."
• Speed: Manis was the fastest, completing the task in approximately 3 minutes.
• Output: Generated a 14-page document with 38 references.
• Content Quality: The literature review was generally good, covering important aspects of critical analysis and research themes.
• Hallucination Rate: 16%.
• Verification Method: Each reference was checked on Google Scholar and, if not found, on the journal's website.
• Types of Errors: Included references with the wrong journal and year but the correct title, repeated references, and completely fabricated references.
• Example: A reference to "morphology control" in the Journal of Physical Chemistry could not be found on the journal's website.
• Observation: The hallucination rate did not increase as the tool processed more references.

Gen Spark Analysis:

• Process: The same prompt as used for Manis was used for Gen Spark.
• Speed: Completed the task in approximately 5-7 minutes.
• Output: Provided 19 references.
• Content Quality: Understood the request for IEEE referencing.
• Format Issue: Extracting the output into a usable file format was difficult.
• Hallucination Rate: 26%.
• Verification Method: Same as Manis (Google Scholar and journal website).
• Types of Errors: Primarily completely fabricated references.
• Observation: Gen Spark had a higher hallucination rate than Manis.

Gemini AI (Google) Analysis:

• Process: The same prompt as used for Manis and Gen Spark was used for Gemini AI with the "deep research" setting enabled.
• Speed: Took the longest, approximately 20 minutes.
• Output: Generated a 61-page document with 105 references.
• Content Quality: Very detailed and deep, including tables to present information.
• Citation Format: Did not strictly adhere to IEEE format; used numbered citations.
• Source Types: Included theses and websites in addition to peer-reviewed papers.
• Hallucination Rate: 1%.
• Verification Method: Each reference was checked for existence and accessibility.
• Types of Errors: One reference was to a book that was not accessible.
• Positive Aspects: Included up-to-date (2025) research.
• Drawbacks: Difficult to export references to reference management software (e.g., Mendeley, EndNote).
• Notable Quote: "If you were wanting to do a literature review at the moment and you are only interested in sort of like the uh finding the literature getting themes um coming up with a generated document there's no doubt at the moment that Gemini AI is just doing so so well in that space."

Comparison and Ranking:

• Gemini AI: Ranked first due to its low hallucination rate (1%), detailed output, and inclusion of up-to-date research, despite not strictly adhering to peer-reviewed sources and IEEE format.
• Manis: Ranked second due to its faster speed and lower hallucination rate (16%) compared to Gen Spark.
• Gen Spark: Ranked third due to its higher hallucination rate (26%) and fewer references.

Synthesis/Conclusion:

The video evaluates three AI tools (Manis, Gen Spark, and Gemini AI) for their ability to generate literature reviews, focusing on the accuracy of their references. Gemini AI performed the best with a very low hallucination rate, producing a detailed and comprehensive review, although it didn't strictly adhere to the prompt's requirements for IEEE formatting and exclusive use of peer-reviewed sources. Manis was a decent second, while Gen Spark had the highest rate of fabricated references. The key takeaway is that while AI tools can be helpful for literature reviews, it's crucial to verify all generated references to avoid including false or inaccurate information in academic work.