*Postponed* NICEP Seminar: Melissa Sands (LSE)

Title: Generative Artificial Intelligence reflects national, not local, perceptions (with Paige Bollen and Joe Higton)

Abstract:  Researchers across disciplines increasingly use Generative Artificial Intelligence (GenAI) to label text and images or as pseudo-respondents in surveys. But of which populations are GenAI models most representative? We use an image classification task -- assessing crowd-sourced street view images of urban neighbourhoods in an American city -- to compare assessments generated by GenAI models with those from a nationally representative survey and a locally representative survey of city residents. While GenAI responses, on average, correlate strongly with the perceptions of a nationally representative survey sample, the models poorly approximate the perceptions of those actually living in the city. Examining perceptions of neighbourhood safety, wealth, and disorder reveals a clear bias in GenAI toward national averages over local perspectives. GenAI is also better at recovering relative distributions of ratings, rather than mimicking absolute human assessments. Our results provide evidence consistent with the idea that GenAI performs particularly poorly in reflecting the opinions of hard-to-reach populations.

Host: Valeria Rueda