Has anyone compared non-overlapping 5-year estimates from 2006 - 2010 and 2011 - 2015 at the tract level? What has been your experience and how do you feel about the reliability of the results?
I looked at ‘Drove Alone to Work’ comparing ACS tables B08006. I calculated the percent change between the two time periods. Next, I applied the Census Bureau’s statistical testing tool (with the confidence interval parameter set at 90%) to the data to determine which census tracts are significantly different from each other for the two periods. Tracts that are not significantly different are considered to have no change. (This testing takes into account the margin of error associated with the survey estimates, based on responses from only a sample of the full population.)
8% of the tracts I am looking at tested significantly different and are considered to have changed. For ‘Drove Alone to Work’ I think 8%, with significant change over 10 years seems reasonable. Has anyone else done this test on a different ACS characteristic? What do you think of your results?
I then calculated the Coefficient of Variation to get reliability estimates on the tracts that tested significantly different. I found 83% (2006-2010) and 92% (2011- 2015) of the data to be highly reliable and eliminated just 5 tracts having low reliability.
The magnitude of change I am seeing in the tracts for ‘Drove Alone to Work’ is a bit disconcerting. I am seeing only 1 of 116 tracts with an increase/decrease under 15%. 56 of 116 tracts with an increase/decrease of 15%-30%, 26 of 116 tracts with an increase/decrease of 31%- 45% and 18 tracts with increase/decrease over 46%. Has anyone else had results where they have found very few tracts with slight change while the majority of tracts (tested statistically significant and reliable) show medium to large change?
What do you think about comparing non-overlapping 5-year estimates in general? Do you find this method to be sound? Would you take a different approach to comparing the data?