Designing for Algorithmic Justice: How Children of Color Define "Fair" AI

TL;DR

In a world where algorithms increasingly act as gatekeepers and authority figures, this paper takes a radical "equity-first" approach. By conducting co-design sessions with children of color in Baltimore, researchers moved beyond the "black box" of AI to understand the human expectations of fairness—finding that for these stakeholders, fairness is less about mathematical parity and more about kindness, privacy, and systemic safety.

The "Blind Spot" in AI Design

Most current discourse on "Fair AI" happens in ivory towers or high-tech labs, focusing on statistical definitions of parity. However, the groups most vulnerable to algorithmic harm—children of color from non-affluent backgrounds—are rarely in the room.

The authors argue that algorithms have neither "molecules nor pixels," making them difficult for children to grasp through traditional design. Furthermore, the standard co-design process itself is often a "privileged activity" that requires transportation, time, and a level of comfort with authority that many marginalized families lack.

Methodology: The KidsTeam Evolution

To bridge this gap, the researchers took the KidsTeam methodology directly into Baltimore City public library branches. They simplified the process into four intense, one-hour sessions:

1. Visualizing the AIL: Sketching the physical and emotional traits of an AI Librarian.
2. Role-Playing: Acting out scenarios involving resumes, resource discovery, and discipline.
3. Table-Top Polling: Using a binary voting system to identify the "tipping point" where a behavior becomes unfair.
4. Storyboarding: Finalizing design recommendations based on synthesized themes.

Figure 1: Adults and children engaging as equal design partners in a Baltimore library.

Key Insights: What does "Fair" actually mean?

1. Kindness as a Technical Requirement

The children were adamant: a "fair" AI is a kind AI. They listed traits like "not being a bully" or "not being a bigot" as functional requirements. This suggests that for children, the delivery of a decision is as important as the accuracy of the decision itself. If an AI is rude, it is perceived as inherently unfair.

2. The PII Paradox

Privacy was a high-stakes topic. While the children wanted the AI to remember them (personalization), they were deeply concerned about surveillance.

• 71% felt it was unfair for the police to access AI data.
• 86% believed the library itself shouldn't even store their specific questions.

This reflects a sophisticated understanding of how data can be weaponized against their communities by other authority figures.

Figure 2: Children using the Table-Top Polling technique to judge the fairness of PII usage.

3. Best Interests vs. Personal Freedom

A fascinating shift occurred during the sessions. Initially, some children thought it was "fair" for the AI to limit their game time based on their school grades (promoting academic success). However, by Week 3, the majority reversed this, viewing the connection to school systems as an unfair breach of their personal space.

Critical Analysis & Conclusion

This work highlights that fairness is contextual and sociopolitical. For these children, an AI is not just a tool; it is an extension of an existing social fabric that has historically been used to monitor or exclude them.

Limitations: The study was localized to one library and a small sample size. However, its value lies not in universal generalization, but in proving that children are capable of complex ethical reasoning regarding algorithms.

Future Outlook: Designers building "Personalized AI" or "AI Tutors" must consider the Inductive Bias built into their systems. If a system requires PII to function, it may inherently alienate marginalized users who prioritize safety over convenience. Truly "inclusive" AI must be designed with these users to avoid replicating the systemic biases of the molecules-and-pixels world.