New Knowledge

Synthetic microdata:  A threat to knowledge.  Each week at City Observatory, we usually profile an interesting or provocative research study.  This week, we're spending a minute to highlight a potential threat to a key source of data that helps us better understand our world, and especially the nation's cities: the public use microsample of the American Community Survey (ACS).  The ACS is the nation's largest and most valuable source of data on population, housing, social and economic characteristics.  While the Census Bureau produces many tabulations of these data, its impossible to slice and dice data in a way that bears on every question.  So Census Bureau makes available what is called a "public use microsample" which allows researchers to craft their own customized tabulations of these data to answer specific questions.  At City Observatory, for example, we've used these data to estimate the income, race and ethnicity of peak hour drive alone suburban commuters traveling from suburban Washington State to jobs in Oregon--a question that would be essentially impossible to answer from either published Census tabulations or other publicly available data.

Microdata are valuable because they link answers to different ACS questions--linking a persons age, gender or race to their income, occupation or housing type, and on.  But because the microdata are individual survey responses, some are concerned that there's a potential violation of privacy:  that someone could use answers to a series of questions to deduce the identity of an individual survey respondent.  While that may technically be a possibility, there's no evidence it occurs in practice.  Still, Census Bureau is hypersensitive about privacy concerns, and has proposed replacing actual microdata with "synthetic" microdata, in order to make it even more difficult to identify an individual.  Essentially, synthetic data would replace actual patterns of responses with statistically modeled responses.  The trouble is, this modeled, synthetic data actually subtracts information, and makes it impossible for researchers to know whether the answers to any particular question are a product of actual variation, or just a quirk of Census Bureau's model.  As University of Minnesota data expert Stephen Ruggles puts it, "synthetic data will be useless for research."

The privacy threat from ACS microdata is a phantom menace.  Ruggles and a colleague at the University of Minnesota have just published a paper showing that attempting to use Census microdata to create individually identifiable records via database reconstruction would produce vastly more random (i.e. false) matches that real ones.   This undercuts the idea that microdata is an actual threat to privacy.

But a proposal to replace PUMS data with synthetic data is a real threat to our ability to better understand our world.  It is like requiring piano players to wear mittens when playing Beethoven sonatas:  the piano will still produce sound, but the result will be noise, not music.

Mike Schneider, Census Bureau's use of 'synthetic data' worries researchers,  Some researchers are up in arms about a U.S. Census Bureau proposal to add privacy protections by manipulating numbers in the data most widely used for economic and demographic research, ABC News, May 27, 2021

Steven Ruggles and David VAn Piper, "The Role of Chance in the Census Bureau Database Reconstruction Experiment," University of Minnesota, May 2021 Working Paper No. 2021-01 DOI: https://doi.org/10.18128/MPC2021-01