Error margins for 5-year ACS estimates

Hello:

I work for a news organization that wants to write about some of the 5-year ACS demographics, especially poverty and income, for local towns in our area. There are 340 municipalities that we cover. If we look at the main estimates for 2013-17, and compare the numbers with a non-overlapping time period such as 2006-10, we see some striking changes in income and poverty, but the margins of error give us pause.

We believe our readers would be interested in changes in income and poverty in their towns, and we can say that the top-line numbers are estimates, and are subject to margins of error. But let’s take a town in our region, Bellmawr, N.J., which has about 11,500 people. Poverty rose from 10 percent in 2006-10 to 20 percent in 2013-17, or an increase of 10 percentage points. With the error margins, however, that change could be as small as +2 percentage points to +19 on the high end.

So would you stay away from these numbers, publish them with the margins of error, aggregate towns into large sub-county geographies, or do something else? Of course, we would be doing on-the-ground reporting as well.


Thanks,

John

Parents
  • John,

    I would include margins of error (MoEs), for the following reasons:

    1. One thing you don't want communities to do is to say, well, we are doing better than is community X, because their poverty rate is 23.3% and ours is 21.2%. If the MoEs are largish and they overlap by quite a lot between the two communities, then neither can really say one is doing "better" or "worse" than is the other. If you can give them a bit of education, then hopefully they can better understand what the numbers mean and how they can and cannot use them.

    2. I see quite a lot of surveys reported in the news, and often if it's a major news organization and they are reporting from Roper or some main survey group, the story includes margin of error, and sometimes even a brief explanation of what the MoE means. I hope that means that MoE is now enough in the public so that many people understand it to some degree.

    Gene
    (from home, so I am not representing or speaking for any organization)
  • Reporting MOEs and other elements of methodology are requirements of the AAPOR (American Association for Public Opinion Research) Transparency Initiative. See www.aapor.org/Transparency_Initiative.htm. Organizations self-select into doing this, and being a member of the TI has non-tangible benefits such as having a prior for smaller polling error by Nate Silver (he uses TI membership as a predictor in his models, and it moves the needle in noticeable ways).
Reply Children