Conversation
Develop documentation about weights. Add docstrings for `vcov` and `stderror`. Move implementation and comparison with R to separate page and mention weights. Show `using` lines in examples instead of running them in a hidden block so that users can reproduce them. Also improve various details.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #632 +/- ##
=======================================
Coverage 96.94% 96.94%
=======================================
Files 8 8
Lines 1213 1213
=======================================
Hits 1176 1176
Misses 37 37 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| When the model is weighted with `ProbabilityWeights`, the sandwich estimator is used, | ||
| scaling by `nobs(x)/(nobs(x) - 1)` degrees of freedom. |
There was a problem hiding this comment.
@gragusa It's not clear to me what justifies these degrees of freedom, nor why Stata uses a different approach. Would you have references about this in R's survey and Stata, and/or would you suggest wording improvements? These would be useful for users.
Am I right that this is completely different from the df.resid argument that can be passed to summary.svyglm?
There was a problem hiding this comment.
Unfortunately, there's no unique principled choice. svyglm's n/(n-1) follows the survey-statistics tradition (PSU-count-based, analogous to the (n-1) in the sample variance). Stata's pweights uses n/(n-k), which matches its standard HC1 robust SE in the equally-weighted limit. Asymptotically, all of n, n-1, n-k are equivalent, so the choice only matters in small samples.
For the df.resid: these are two completely separate things. df.resid in summary.svyglm is the denominator degrees of freedom for the t/Wald reference distribution used to compute p-values and confidence intervals. The n/(n-1) factor we're discussing here is a finite-sample correction that multiplies the sandwich variance estimate itself. The former affects only the reference distribution; the latter affects the variance estimate.
There was a problem hiding this comment.
OK, thanks. So you think we should keep the formulation I proposed? Any references we could cite to help understand this choice?
There was a problem hiding this comment.
I think so. The classic reference is Lumley's book "Complex Surveys". Also, STATA has two commands (ways) to run regression with pweights. One is my doing regress y x [pweights] (which defaults to n-k corrections). To other (https://www.stata.com/manuals/svyvarianceestimation.pdf) is to use the svy set of commands, which default to n-1 corrections.
There was a problem hiding this comment.
Ah interesting, I'll see whether I can mention these somewhere in the docs. The fact that we follow at least one of Stata's definitions is good as it means both R and Stata users can replicate our results.
| Using analytic weights corresponds to weighted least squares. | ||
| This gives the same results as R and Stata. | ||
|
|
||
| Probability weights give the same point estimates as analytic weights, but standard errors | ||
| and p-values are based on a sandwich (heteroskedasticity-robust) estimator. | ||
| This gives the same results as the R `survey` package with a simple survey design | ||
| without strata nor clustering, but differs from Stata with the `pweights` option, which | ||
| adopts the same approach but with a different assumption regarding degrees of freedom. |
2 participants
Develop documentation about weights. Add docstrings for
vcovandstderror. Move implementation and comparison with R to separate page and mention weights. Showusinglines in examples instead of running them in a hidden block so that users can reproduce them. Also improve various details.