Science journalists find ChatGPT is bad at summarizing scientific papers

ChatGPT’s Science Summary Struggles: A Wake-Up Call for Tech-Savvy SMBs Science journalists from AAAS recently put ChatGPT through its paces on summarizing complex scientific papers, and the results were underwhelming—think more fumble than feature. They rated the AI’s outputs poorly for blending with human content and overall appeal, highlighting issues like mixing up cause and effect, skipping crucial context, and hyping findings without merit. While ChatGPT handled straightforward facts decently, it bombed on deeper tasks like explaining methods, limitations, or broader impacts, especially with multifaceted or paired papers. For SMBs and MSPs leaning on AI for quick content creation or tech briefs, this underscores the risks of sloppy automation in professional comms, potentially leading to misinformation that erodes trust or demands extra manual fixes—making it a false economy in time-strapped operations. Takeaways: - AI summaries can spread errors, harming SMB credibility.

    **Key takeaways**
    - Fact-checking AI outputs often costs more time than starting fresh. - Over-relying on ChatGPT risks legal or reputational pitfalls in tech fields. - SMBs should pair AI with human review for accurate content.

    **Source:** [https://arstechnica.com/ai/2025/09/science-journalists-find-chatgpt-is-bad-at-summarizing-scientific-papers/](https://arstechnica.com/ai/2025/09/science-journalists-find-chatgpt-is-bad-at-summarizing-scientific-papers/)

• 2025 4
• 2024 20