MoE and unreliable

Hi Gene - 

Hi Gene - 

I don't have a direct answer to your question.  However, there was a pretty good discussion about interpreting coefficient of variation in this forum about a year ago

Different coefficients of variation with same information - Forum - Discussion Forum - American Community Survey Data Users Group (prb.org)

From what I remember, Esri tends to apply their reliability standards pretty indiscriminately - often to proportions / percentages.  This has the problem that Matt and Jonathan describe in the above post.  

-- Dave

Thanks, a very interesting discussion.

David Wong and Min Sun wrote about this set of issues in a paper published back in 2013 in Spatial Demography. This may be useful to some

Handling Data Quality Information of Survey data in GIS: A Case of Using the American Community Survey Data.

https://link.springer.com/content/pdf/10.1007/BF03354884.pdf?pdf=button

Best

- Stephen

I am afraid that the Bureau actually miss handled there computation of MOE, if one is a frequentist, if one is a Bayesian that approach is even worse.  On the frequentist, I had a debate with Bureau a long time ago, where they indicated that they realized that they had made a mistake/  (Some of their margins of error go negative, in other words they are saying that their is chance that there will be negative cases.)  This is obviously absurd.  They used the wrong approach to computing error altogether.  That has been corrected for some data using Random Replicates, which are also available in the PUMS data.  Here is a link to my correspondence with the Bureau and their response.

https://www.dropbox.com/s/lfjo11wm5ma1axx/Memo_Regarding_ACS-With_Response.pdf?dl=0

Here is a link to the Random Replicate Explanation. 

www.census.gov/.../variance-tables.html

For small numbers or when any of the percentages are less than 30% or more than 70% one should use random replicates.  The MOE assumes a normal distribution which is not true in such cases.  As another example their estimate of the MOE of median income assumes a normal and symmetric distribution of income.  There are analyses that show that the MOE of median income approaches zero as it becomes more skewed.  Guess what Income is highly skewed.  They use a multiple well above one for MOE of median income based upon the standard error of the average,  These more accurate approaches to standard error on covered in all of the major statistical packages.

Andy

Interesting discussion happy to see Andy refer to the frequentist vs Bayesian issue I read with great delight in The Signal and the Noise. Weighting is another issue you can't address this way, right? Couldn't weighting be happening in the background to adjust some of the sampling issues showing up in these MOEs? 

Thanks very much for the link to this paper.

Thanks very much!