Decoding the Digital Eye: An Audit of Police Social-Media Monitoring

TL;DR

Researchers at Oregon State University performed a rare "under the hood" audit of DigitalStakeout, a social media surveillance tool used by the police. By analyzing leaked log files, they discovered the tool is surprisingly primitive—relying on simple keyword searches that frequently confuse weather reports and pizza parties with drug deals—while potentially magnifying racial disparities in law enforcement.

The "Black Box" of Public Safety

We live in an era where 89% of US police agencies use social media for investigations. However, the software they use is often proprietary and shielded from public accountability. Tools like Geofeedia and MediaSonar have previously been caught targeting hashtags like #BlackLivesMatter, yet we rarely see the raw data of what these tools actually flag on a day-to-day basis.

The authors of this study bypassed corporate secrecy by using public records requests to obtain 13 months of log files from the Corvallis Police Department. This provided a unique dataset: 7,240 social media posts that the "algorithm" deemed suspicious enough to show to the police.

Methodology: Reverse-Engineering Surveillance

The research team employed two main technical thrusts: Demographic Auditing and Keyword Reverse-Engineering.

1. The Demographic Audit

To determine if the tool was biased, the team had to identify the race and gender of users. Since automated AI (like Face++) often carries its own biases and discards profiles without clear face shots, the team used a rigorous manual coding protocol. Three researchers cross-verified profile names, bios, and photos to classify users, achieving "almost perfect" inter-rater reliability (Krippendorff’s $\alpha =0.94$).

2. Cracking the Keyword Code

How does DigitalStakeout decide a tweet is about "Narcotics"? The researchers compared the "Output" (flagged tweets) against a "Control" (all local tweets from the same period collected via Twitter API).

Using a hitting set logic, they identified "Necessary" and "Likely" keywords. If a word only appears in the flagged data and never in the general pool, it’s a high-probability trigger.

Table: The identified roots of keywords used by the monitoring software.

Key Findings: High Noise, Low Intelligence

The results paint a picture of a tool that is technologically shallow yet potentially harmful:

• The Benign Noise Problem: The "Narcotics" filter was triggered by the word "broken." Consequently, the police logs were flooded with updates from a weather bot reporting "broken clouds." Similarly, "hop" flagged tweets from local craft breweries, and "party" flagged "pizzaparty" and "birthdayparty."
• Demographic Skew: The data suggests that Twitter users in Corvallis are more diverse than the city's general population (higher Black and Hispanic representation). While the software didn't statistically "target" minorities more than their general presence on Twitter would suggest, it still resulted in these groups being surveilled at rates higher than their representation in local residents or even arrest records.
• The Utility Gap: Most flagged posts were irrelevant to actual crime. In one instance, a campus shooting threat was caught by an anonymous tip, not by the expensive monitoring software, which was busy flagging tweets about bowling (keyword: "bowl").

Table: Comparison of demographic distributions between general geotagging users and those flagged by surveillance.

Critical Insight: The Risk of "Digital Over-Policing"

The danger of these tools isn't just that they are "dumb"—it's that they create a technological veneer of suspicion. When a police officer sees a "hit" on a dashboard, they may not realize it was triggered by a slang term used in a benign context. If certain demographics use slang or geotag their posts more frequently, they naturally fall into the "surveillance net" more often.

Conclusion

This study serves as a proof-of-concept for Algorithmic Accountability. It proves that we don't need the source code to audit a "black box"—we just need the logs. As law enforcement moves toward AI-driven "predictive" policing, high-fidelity audits like this are the only way to ensure that "public safety" doesn't become a euphemism for automated discrimination.

Future Outlook: The authors suggest that law enforcement agencies should be legally required to maintain accessible log files for independent researchers to ensure that the tools they buy with taxpayer money actually work—and work fairly.