Efficient Subsampling Estimating Equations from Massive Data Sets
摘要
The authors study the efficient subsampling estimating equation for massive data sets. A two-step procedure is proposed: (i) Design a subsampling probability (SP) to draw a subsample from the full data set, and (ii) construct an estimating function based on the subsample. To improve estimation efficiency, the SP in Step (i) is allowed to be informative, i.e., to depend on the response. However, informative subsampling typically induces bias in the estimating function used in Step (ii). To correct this bias, three approaches are developed to modify the estimating function: Inverse probability weighting (IPW), generalized IPW (GIPW), and projection (PJT), yielding three subsampling estimators. IPW is widely applicable but may inflate variance. GIPW adds a non-informative weight to mitigate this issue. PJT relies on likelihood information and is often the most efficient. Asymptotic properties are then established, based on which the authors propose data-driven methods for selecting the SP and weights by minimizing the estimated asymptotic variances. Numerical results support our theoretical findings.