Details
  • Views 5852 views
  • Replies 5 replies
  • Subscribers 540 subscribers
  • Users 0 members are here
Labels
Related Census Bureau Resources
Related Forum Threads

Rationale for making comparisons using non-overlapping periods

Diana Lavery
over 3 years ago

Hi ACS friends, 

I'm trying to explain to some brilliant but non-ACS-knowledgeable coworkers and others why the recommended approach is to use *non-overlapping* periods when making comparisons over time.  I've looked at the Census doc (https://www.census.gov/programs-surveys/acs/guidance/comparing-acs-data.html) and all I can find is the guidance that says Do use non-overlapping datasets, Do not use overlapping datasets.  I'm having a hard time finding the why.  All I can come up with is this:

4-5ths of the sample is the same in adjacent datasets, e.g., 2014-2018 vs. 2015-2019. You can think of Census removing respondents from 2014, and adding respondents from 2019, but the respondents from 2015, 2016, 2017, and 2018 are the same. We should compare two completely different datasets, which means two non-overlapping periods.

I'm curious how others answer this question as well!  -Diana

Parents
  • over 3 years ago

    Diana and Todd Z--

    If you were comparing two separate time-periods, two datapoints at different times, then you would want to account for the statistical inference "margin of error" around the two estimate points. There's a formula for this.

    See the "Determining Statistical Significance" section (p 55) in the "Understanding and Using ACS Data" handbook
    https://www.census.gov/content/dam/Census/library/publications/2020/acs/acs_general_handbook_2020.pdf 
    But if the two time-periods are overlapping, this is the statisticians' equivalent of "double-dipping your chip" in the serving bowl. Some people will say "no big deal."  Other people will condemn it as unmannered barbarism.  Not so much for germophobe reasons, but because the "double-dip" (the same survey cases being included in both the 2010-14 and 2014-18 estimates) makes the standard significance test invalid. 
    Bottom line: Do you want to be able to say something about statistical significance of the differences over time? (And does your audience include statisticians?) 
    --TG
Reply
  • over 3 years ago

    Diana and Todd Z--

    If you were comparing two separate time-periods, two datapoints at different times, then you would want to account for the statistical inference "margin of error" around the two estimate points. There's a formula for this.

    See the "Determining Statistical Significance" section (p 55) in the "Understanding and Using ACS Data" handbook
    https://www.census.gov/content/dam/Census/library/publications/2020/acs/acs_general_handbook_2020.pdf 
    But if the two time-periods are overlapping, this is the statisticians' equivalent of "double-dipping your chip" in the serving bowl. Some people will say "no big deal."  Other people will condemn it as unmannered barbarism.  Not so much for germophobe reasons, but because the "double-dip" (the same survey cases being included in both the 2010-14 and 2014-18 estimates) makes the standard significance test invalid. 
    Bottom line: Do you want to be able to say something about statistical significance of the differences over time? (And does your audience include statisticians?) 
    --TG
Children
No Data
For more information about the American Community Survey Census Bureau Website