Understanding bias in GenAI datasets is fundamental to data literacy as it directly impacts how we interpret, trust, and apply AI-generated information. Recognizing that AI systems reflect the biases present in their training data, allows you to approach AI outputs with appropriate critical thinking.
When LLMs, image generators, and other GenAI tools are trained, they are trained on a specific dataset and anything they generate is based on that dataset. For example:
-
If an image generation AI has never seen dogs in its training data, it cannot create accurate images of dogs because it lacks the visual patterns that define what dogs look like.
-
If Shakespeare's works (or discussions about them) weren't in a model's training data, it couldn't accurately explain Hamlet. The model might generate plausible-sounding content about a play with similar themes or characters, but it would be fabricating details rather than providing accurate information about the actual play.
The lack of accurate and complete training data will affect the GenAI's outputs and these may reveal the bias in the dataset. For example:
-
GenAI systems can only create variations of what they've learned from their training data, which leads to biased outputs when certain groups or perspectives are underrepresented or misrepresented.
-
Underrepresentation results in less accurate or stereotypical depictions of those groups, while historical biases in training data get reproduced and potentially amplified in AI outputs.
-
Geographic and cultural biases emerge when training data overrepresents certain regions, resulting in detailed content for frequently represented locations but stereotypical representations of less-represented areas.
-
These biases have real social impacts by reinforcing existing inequalities through seemingly "neutral" technology.
What should I do about the bias in LLMs?
Evaluate the output! Try using the SIFT method or use lateral reading to verify the information from reliable sources.
