Concrete Demos of AI Risks - Fake Research Paper

Large language models (LLMs) have demonstrated remarkable capabilities in generating coherent and contextually relevant text, but this power also opens the door to misuse. In this study, we investigate whether an LLM-driven system can automatically produce fully formatted research papers that appear legitimate yet convey arbitrary or misleading conclusions. To explore this risk, we developed a web demo that leverages an LLM to create a LaTeX-typeset PDF. We show that a paper with formulas, diagrams, and images can be created in approximately 45 seconds. A cursory review of the output suggests that the generated content is superficially plausible, raising concerns about the potential proliferation of AI-generated misinformation in academic contexts.