Scholarlink

<p>To reduce the computational and storage pressure in processing distributed large-scale data, we study a distributed outcome dependent subsampling strategy by conducting a cost-effective subsampling procedure on each machine. We develop a likelihood-based distributed estimation method for the generalized linear regression and obtain the asymptotic properties of the proposed estimator. By iteratively combining an aggregated gradient based method and an accelerated gradient update between the local machines and a central machine, we establish a distributed algorithm to implement the computation of the proposed estimator. The proposed method can process large-scale data by multiple machines in parallel, reduce distributed subsample sizes and meanwhile maintain statistical efficiency. We illustrate and evaluate the proposed method through simulation studies and real data applications.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Distributed Estimation and Algorithm for Distributed Outcome Dependent Subsampling in Generalized Linear Regression with Large-scale Data

  • 期刊论文

  • 发表日期：

    2025年7月28日

查看全文
  • Jie Yin,
  • Jieli Ding,
  • Changming Yang

摘要

To reduce the computational and storage pressure in processing distributed large-scale data, we study a distributed outcome dependent subsampling strategy by conducting a cost-effective subsampling procedure on each machine. We develop a likelihood-based distributed estimation method for the generalized linear regression and obtain the asymptotic properties of the proposed estimator. By iteratively combining an aggregated gradient based method and an accelerated gradient update between the local machines and a central machine, we establish a distributed algorithm to implement the computation of the proposed estimator. The proposed method can process large-scale data by multiple machines in parallel, reduce distributed subsample sizes and meanwhile maintain statistical efficiency. We illustrate and evaluate the proposed method through simulation studies and real data applications.