Hi everyone!
Has anybody looked into this apparent statistically significant difference between the two surveys on educational attainment data?
2020 CPS counts a total of 113381 thousand 18+ civilian noninstitutionalized population with at least an associates degree, out of a total 252117, so ~45% in share.
Estimate for the same scope using 2020 5-years ACS PUMS is 97417 out of 253296, or ~38%.
The discrepancy is even more pronounced for only doctoral degree holders, where CPS estimates 4.7M vs ACS 3.2M, a nearly 50% higher number for the former.
Checking both survey's questionnaires, the question for educational attainment is essentially identical and reasonably unambiguous. What would be some factors leading to this rather stark discrepancy?
Really appreciate it if anyone could shed some light for me!
Sincerely,
Michael
Best,
I'm new, but it seems that the default for ACS is 25 years or older (not always specified in field names). I have no idea why they do this and its possible to get 18-plus ed data from the ACS. So check to make sure the ACS field is not 25 Plus. I'd like to do a breakout in ACS with the 18-24 gap and assuming there is a significant gap send an email asking why use the 25 and above. 18 is the mean grad age add 4 years and get 22, lots of AP courses in HS and some finish in 3 years (it's way cheaper).
If your pulling directly (not TidyCensus) let me know the table and field and I'll give you what I've found for Ed.
Here's a factsheet on comparing ACS and CPS educational attainment data: https://www.census.gov/topics/education/educational-attainment/guidance/factsheet-acs-cps.html(By the way, I found this through the ACS Subject Definitions document, which is invaluable and should be pinned to the top of this forum: https://www2.census.gov/programs-surveys/acs/tech_docs/subject_definitions/2021_ACSSubjectDefinitions.pdf)
Read both and I don't think answered the question. I will try tonight or tomorrow to list data and fields related to the gap between 18-25 and 25 plus.
I see a 5% diff with Population 25 years and over!! Bachelor's degree or higher by including 18-24. If your looking at BA only that goes to around 10%. Table S1501.
If of course you actually looking at 18+ then these numbers won't help you.
That's a fair difference although I was expecting it to be higher. So I think using the 25 or higher for Ed is clearly not a good measure. Worth a deeper dive at some point in the future because I think the difference should be larger.
Hey Tom, Bernie,
Thank you both for replying! Particular Tom, for your warm recommendations.
In regards to the PUMS age universe setting, I double checked on the parameters and it shouldn't have caused any issues. The statistically significant difference still exist if we change the scope to 25+ yr olds, where ACS PUMS suggest 41% vs CPS 48% of the population has at least an associate's degree.
The official subject definitions document is very helpful and points to an article by a Census Bureau staff member N. Scanniello covering the exact subject matter. The unfortunate thing is it is fairly dated (2004), and it appears the discrepancy back then between the two surveys were considerably smaller than contemporary estimates. The paper mentions an approximate 1.3% difference in share of 25 yrs+ population with an associate degree (2.0% in 2020), 0.9% difference in bachelor's degree (3.3% in 2020), and not-statistically-significant difference in advanced degree holders --- very different from current day situation where CPS ASEC has a nearly 50% higher estimate for doctoral holders (4.7M vs 3.2M). The explanations offered in the paper are fairly conventional surveying issues such as nonresponses or wording differences, and in my humble opinion is becoming inadequate for justifying the differences exhibiting today. It would be very helpful if the Bureau someday updates this report.
After some personal considerations, I have decided to settle on seeing this difference being simply due to unaccounted-for-errors likely in selection bias. If one were to blindly guess which of ACS or CPS has the "truer" value I would undoubtedly lean to ACS for its clearly stronger coverage rate and sampling size. However, in less academic environment such as citations in business presentations I would probably grudgingly use CPS figures. One reason being the apparent higher familiarity most people nowadays have with CPS data over ACS. Common newspaper headline figures such as non-farm payrolls and unemployment rates are all based on the more frequent CPS survey, allowing it to gain much greater public attention in general. It would be quite the pain to suggest an alternative dataset that has rather significant differences and try explaining the details. Another reason being people generally prefer to see more "favorable" estimates, for a variety of reasons. And insisting to use a "bad" number sometimes cause unnecessary suspicions questioning your motives, which can be annoying as well.
Due to these complications I will go on to suggest US having 4.7M doctoral degree on the report I'm working on despite being against best judgement. And I will seek solace from knowing the 1.5M phantom PhD holders out there wouldn't care less.