Increasing Use of AI Chatbots to Fact-Check X Posts

Integrating AI into fact-checking does not necessarily signal the replacement of human fact-checkers.

Mar 28, 2025

Dear Reader,

While many industries have been grappling with the fear of artificial intelligence (AI) replacing human jobs, fact-checkers have largely felt insulated from this threat, given the nuanced analysis and critical thinking their roles demand. That, however, seems to be changing with the increased use of AI chatbot handles on X to “fact-check” posts. We looked at the increasing use of two such handles, @AskPerplexity — the X handle of AI-powered search engine Perplexity AI — and @grok — developed by Elon Musk's company xAI.

If you are into doom scrolling on X, you must have already seen the handles of @AskPerplexity and @grok being tagged under most posts, either asking the bots to fact-check something or providing additional context.

SUPPORT OUR WORK*

But Are the Bots Able to Fact-Check?

On 11 March, Padmaja Joshi, a Senior Executive Editor at English news organisation TV9, shared a segment on her X platform comparing the education levels of Uttar Pradesh and Tamil Nadu. She used the 2024 Annual Status of Education Report (ASER) to help her argument. The report assesses schooling and learning outcomes among children aged 3 to 16 in rural India. As is the nature of the platform, people started questioning and supporting her. However, she then replied with the screenshot of one such account where the user tagged Grok to fact-check her original post.

The response from Grok seemed to validate the original post, declaring that there was “no agenda, just numbers talking”.

Now, for the purpose of this newsletter, we will refrain from doing any fact-checking. To read our fact-checks, check out WebQoof.

Coming back to the case study, now an AI bot had declared the claim to be true, but another user felt the need to ask Perplexity as well.

Now, a second AI tool has also fact-checked the information and found it to be true.

But wait, here comes the twist. Another user changed the question for Perplexity, which changed the response.

Another detailed prompt by yet another user also gave more context and showed how a direct comparison between UP and TN using the ASER survey might be misleading.

This raises two questions:

Can we trust AI bots to fact-check information on important issues?
Can we remove the human from the loop and expect 100% accurate information?

Should LLMs Be Deployed in Fact-Checking?

If you have followed any discourse on fact-checking or mis/disinformation, you might have heard how fact-checking is almost always reactive. The false claims and “fake news” travel way more than a piece of fact-checked information. Keeping that in mind, we believe leveraging advanced Large Language Models (LLM) to interact with users in real-time could be a massive step in the right direction.

There are other benefits of using AI bots for fact-checking, such as:

Scalability: AI chatbots can process vast amounts of data swiftly, enabling them to identify and address misinformation more efficiently than human fact-checkers alone.
Consistency: LLMs can apply uniform criteria when evaluating information, reducing the variability that might arise from human biases.
Accessibility: By providing real-time responses, AI chatbots make fact-checking resources readily available to a broader audience.

However, despite these advantages, deploying LLMs in fact-checking is not without challenges. As established, accuracy is a huge concern in real-time fact-checking on X. Experts argue that LLMs, despite their advanced capabilities, do not truly understand the content they process. Emily M Bender, a professor at the University of Washington, likened chatbots to "parrots" that repeat information without genuine comprehension, highlighting the risk of disseminating information without contextual understanding.

There is also the concern that AI chatbots could inadvertently amplify misinformation. A report revealed that a Russian network utilised AI chatbots to spread disinformation, illustrating how malicious actors might exploit these technologies to disseminate false narratives.

Best Use Case of AI in Fact-Check: The Human-AI Collaboration

Integrating AI into fact-checking does not necessarily signal the replacement of human fact-checkers. Instead, a collaborative approach is emerging:

Augmenting Human Efforts: AI can handle repetitive tasks such as scanning large datasets for potential misinformation, allowing human fact-checkers to focus on nuanced analysis and contextual evaluation.
Tool Development: Organisations like Newtral have developed AI tools to assist in monitoring media and identifying claims worth fact-checking, thereby enhancing the efficiency of human fact-checkers.
Source Verification: Cross-reference AI-generated information with credible sources. Relying solely on AI outputs without verification can lead to the spread of inaccuracies. Independent fact-checking sites and tools can assist in this process.
Editorial Oversight: Human oversight remains crucial in interpreting AI-generated outputs, ensuring that fact-checking maintains accuracy, ethical standards, and contextual sensitivity.

Integrating AI chatbot handles like @AskPerplexity and @Grok into the fact-checking ecosystem signifies a pivotal shift in how information is verified and disseminated. While these tools offer remarkable efficiency and scalability, they are not without limitations and must be regularly updated and monitored. Fact-checkers will have to evolve and adapt to the changing technology while ensuring transparency by declaring the use of AI in their processes. Organisations must also develop and adhere to ethical guidelines for AI usage in fact-checking.

This is not the end.

We studied the AI chatbot handles in greater detail and covered three other things:

Their effectiveness in responding to posts that independent fact-checkers have fact-checked.
Their differing responses to the same or similar queries using a second case study.
Good use cases of the bots that were helpful.

But to read that story, you will have to become a member* of The Quint because we will need your support to continue our fight against misinformation. Consider contributing to our work.

BECOME A MEMBER*