MoE and unreliable

Hi Gene - 

I don't have a direct answer to your question.  However, there was a pretty good discussion about interpreting coefficient of variation in this forum about a year ago

Different coefficients of variation with same information - Forum - Discussion Forum - American Community Survey Data Users Group (prb.org)

From what I remember, Esri tends to apply their reliability standards pretty indiscriminately - often to proportions / percentages.  This has the problem that Matt and Jonathan describe in the above post.  

-- Dave

I am afraid that the Bureau actually miss handled there computation of MOE, if one is a frequentist, if one is a Bayesian that approach is even worse.  On the frequentist, I had a debate with Bureau a long time ago, where they indicated that they realized that they had made a mistake/  (Some of their margins of error go negative, in other words they are saying that their is chance that there will be negative cases.)  This is obviously absurd.  They used the wrong approach to computing error altogether.  That has been corrected for some data using Random Replicates, which are also available in the PUMS data.  Here is a link to my correspondence with the Bureau and their response.

https://www.dropbox.com/s/lfjo11wm5ma1axx/Memo_Regarding_ACS-With_Response.pdf?dl=0

I am afraid that the Bureau actually miss handled there computation of MOE, if one is a frequentist, if one is a Bayesian that approach is even worse.  On the frequentist, I had a debate with Bureau a long time ago, where they indicated that they realized that they had made a mistake/  (Some of their margins of error go negative, in other words they are saying that their is chance that there will be negative cases.)  This is obviously absurd.  They used the wrong approach to computing error altogether.  That has been corrected for some data using Random Replicates, which are also available in the PUMS data.  Here is a link to my correspondence with the Bureau and their response.

https://www.dropbox.com/s/lfjo11wm5ma1axx/Memo_Regarding_ACS-With_Response.pdf?dl=0

Here is a link to the Random Replicate Explanation. 

www.census.gov/.../variance-tables.html

For small numbers or when any of the percentages are less than 30% or more than 70% one should use random replicates.  The MOE assumes a normal distribution which is not true in such cases.  As another example their estimate of the MOE of median income assumes a normal and symmetric distribution of income.  There are analyses that show that the MOE of median income approaches zero as it becomes more skewed.  Guess what Income is highly skewed.  They use a multiple well above one for MOE of median income based upon the standard error of the average,  These more accurate approaches to standard error on covered in all of the major statistical packages.

Andy

Thanks very much!