<p>Performative prediction refers to scenarios where model predictions influence the underlying data distribution they aim to predict. A desirable property in this context is <i>performative stability</i>, where model predictions are already optimal for the distribution they induce, indicating converged model parameters and no need for further retraining. Achieving performative stability requires characterizing the data distribution map <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\mathcal {D}(\theta )\)</EquationSource> </InlineEquation>, i.e., the relationship between predictions and the resulting distribution shifts. Current studies typically quantify distribution differences using metrics like <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\mathcal {W}_1\)</EquationSource> </InlineEquation> distance or <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\chi ^2\)</EquationSource> </InlineEquation> divergence, which may not provide isometric embeddings or maintain metric equivalence in practical scenarios, limiting their applicability across various data distribution maps. Moreover, the crucial smoothness parameter <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(\beta \)</EquationSource> </InlineEquation> in existing work is often unobtainable in performative scenarios, constraining the real-world utility of current theoretical results and methods. To address these challenges, we develop an algorithm that learns a performatively stable model for arbitrary data distribution maps without requiring the joint smoothness parameter <InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(\beta \)</EquationSource> </InlineEquation>. Specifically, we introduce a new <InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(\hat{\varepsilon }\)</EquationSource> </InlineEquation>-sensitivity measure for <InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(\mathcal {D}(\theta )\)</EquationSource> </InlineEquation>, quantified by the gradient of the loss function, which naturally and directly characterizes how distribution shifts affect the optimization of the objective function. Based on this sensitivity, we formulate a <InlineEquation ID="IEq8"> <EquationSource Format="TEX">\(\gamma \)</EquationSource> </InlineEquation>-strongly convex loss function and optimize the deployed model accordingly, where <InlineEquation ID="IEq9"> <EquationSource Format="TEX">\(\gamma \)</EquationSource> </InlineEquation> is derived from the defined <InlineEquation ID="IEq10"> <EquationSource Format="TEX">\(\hat{\varepsilon }\)</EquationSource> </InlineEquation>, eliminating the need for the <InlineEquation ID="IEq11"> <EquationSource Format="TEX">\(\beta \)</EquationSource> </InlineEquation>-joint smoothness assumption. Our theoretical results guarantee the convergence of the deployed model to performative stability. Extensive experiments on synthetic and real-world datasets with diverse data distribution maps demonstrate the superiority of our method over state-of-the-art techniques in two key aspects: prediction accuracy and performative stability.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Performative Prediction in the Wild: Adapting to Arbitrary Data Distribution Maps

  • Guangzheng Zhong,
  • Yang Liu,
  • Ruichen Liu,
  • Jiming Liu

摘要

Performative prediction refers to scenarios where model predictions influence the underlying data distribution they aim to predict. A desirable property in this context is performative stability, where model predictions are already optimal for the distribution they induce, indicating converged model parameters and no need for further retraining. Achieving performative stability requires characterizing the data distribution map \(\mathcal {D}(\theta )\) , i.e., the relationship between predictions and the resulting distribution shifts. Current studies typically quantify distribution differences using metrics like \(\mathcal {W}_1\) distance or \(\chi ^2\) divergence, which may not provide isometric embeddings or maintain metric equivalence in practical scenarios, limiting their applicability across various data distribution maps. Moreover, the crucial smoothness parameter \(\beta \) in existing work is often unobtainable in performative scenarios, constraining the real-world utility of current theoretical results and methods. To address these challenges, we develop an algorithm that learns a performatively stable model for arbitrary data distribution maps without requiring the joint smoothness parameter \(\beta \) . Specifically, we introduce a new \(\hat{\varepsilon }\) -sensitivity measure for \(\mathcal {D}(\theta )\) , quantified by the gradient of the loss function, which naturally and directly characterizes how distribution shifts affect the optimization of the objective function. Based on this sensitivity, we formulate a \(\gamma \) -strongly convex loss function and optimize the deployed model accordingly, where \(\gamma \) is derived from the defined \(\hat{\varepsilon }\) , eliminating the need for the \(\beta \) -joint smoothness assumption. Our theoretical results guarantee the convergence of the deployed model to performative stability. Extensive experiments on synthetic and real-world datasets with diverse data distribution maps demonstrate the superiority of our method over state-of-the-art techniques in two key aspects: prediction accuracy and performative stability.