Evaluating sources and validating AI output both involve critical thinking. However, the focus and process of the evaluation are different.
Evaluating Traditional Sources (e.g., books, journal articles, websites)
This is a proactive process focused on judging the credibility of the container before you use the information inside it. You are asking, "Can I trust this source?"
You evaluate a source by investigating its origins:
- Author: Who wrote this? What are their credentials and expertise on this topic?
- Publisher: Who published this? Is it a reputable academic press, a scholarly journal, or a popular magazine? Is there a potential for bias?
- Date: When was it published? Is the information current enough for your topic?
- Citations: Does the author cite their sources? This shows they have done their own research and allows you to check their work.
- Process: Was it peer-reviewed? This means other experts in the field have vetted the information for accuracy and quality.
Evaluating AI-Generated Output (e.g., from ChatGPT, Gemini, Claude)
This is a reactive process focused on verifying the accuracy of the content itself after you have received it. You are asking, "Is this information correct and unbiased?"
An AI is a tool, not a source. It generates new text based on patterns in its training data, but it doesn't "know" where each piece of information came from. Therefore, you must act as the fact-checker.
You evaluate AI output by:
- Fact-Checking: Verifying specific facts, dates, names, and statistics against reliable sources (like the ones you would find using the evaluation method above).
- Identifying "Hallucinations": Watching for plausible-sounding but fabricated information, including fake citations or sources.
- Assessing for Bias: Looking for biases in tone, word choice, or the perspectives that are included or excluded. AI training data contains human biases which can be reflected in the output.
- Checking for Logic: Ensuring the argument is coherent, logical, and doesn't contain contradictions.
