Predicting purchase conversion from web-session clickstreams underpins targeting, personalization, and budget-allocation decisions in digital commerce, but published benchmarks frequently report near-perfect discrimination that does not survive a leakage audit. We present a leakage-aware benchmark study that contrasts a full-session task (Task A, an upper-bound diagnostic in which the input may contain post-outcome tokens) with a pre-conversion task (Task B, a deployable formulation in which post-purchase events and aggregates are removed) on the public Yoochoose RecSys 2015 challenge data, complemented by a controlled synthetic stress test. We compare four tabular baselines (logistic regression, histogram gradient boosting, XGBoost, LightGBM) against two sequence-aware deep models (LSTM, Transformer), report 95 % bootstrap confidence intervals on every metric, and use DeLong’s correlated-AUC test for pairwise model comparison. We further evaluate calibration via reliability diagrams, Brier score, and expected calibration error; quantify business value via top-k expected-revenue capture; and triangulate behavioral evidence with pre-purchase n-gram motifs and counterfactual session edits with paired Wilcoxon tests on <InlineEquation ID="IEq1"> <EquationSource Format="TEX">$n{=}1\,000$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>n</mi> <mo>=</mo> <mn>1</mn> <mspace width="0.166667em" /> <mn>000</mn> </mrow> </math></EquationSource> </InlineEquation> sessions per edit. Across both datasets, the gap between Task A and Task B is substantial: Task A AUC reaches <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$\ge 0.9997$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mo>≥</mo> <mn>0.9997</mn> </mrow> </math></EquationSource> </InlineEquation> for every model on Yoochoose—driven entirely by the purchase token—while Task B AUC ranges from 0.685 (logistic regression) to 0.755 (LightGBM), with sequence models clustering near the lower end at this dataset size. The Transformer matches the LSTM in mean discrimination but is roughly 50<InlineEquation ID="IEq3"> <EquationSource Format="TEX">$\times $</EquationSource> <EquationSource Format="MATHML"><math> <mo>×</mo> </math></EquationSource> </InlineEquation> more stable across random seeds. Pre-purchase motifs reveal interpretable conversion drivers without the trivially-purchase-tokenized motifs that contaminate naive analyses. The study positions Task A/B contrast and statistically grounded multi-seed evaluation as a default protocol for clickstream conversion benchmarks.

Sequence-aware models for predicting online purchase conversion from clickstream data: a leakage-aware benchmark study with calibration, value, and interpretability

期刊论文
发表日期：

2026年7月8日

查看全文

Philipp Goetzinger,
Patricija Topić,
Sebastian Noy,
Karsten Huffstadt

摘要

Predicting purchase conversion from web-session clickstreams underpins targeting, personalization, and budget-allocation decisions in digital commerce, but published benchmarks frequently report near-perfect discrimination that does not survive a leakage audit. We present a leakage-aware benchmark study that contrasts a full-session task (Task A, an upper-bound diagnostic in which the input may contain post-outcome tokens) with a pre-conversion task (Task B, a deployable formulation in which post-purchase events and aggregates are removed) on the public Yoochoose RecSys 2015 challenge data, complemented by a controlled synthetic stress test. We compare four tabular baselines (logistic regression, histogram gradient boosting, XGBoost, LightGBM) against two sequence-aware deep models (LSTM, Transformer), report 95 % bootstrap confidence intervals on every metric, and use DeLong’s correlated-AUC test for pairwise model comparison. We further evaluate calibration via reliability diagrams, Brier score, and expected calibration error; quantify business value via top-k expected-revenue capture; and triangulate behavioral evidence with pre-purchase n-gram motifs and counterfactual session edits with paired Wilcoxon tests on $n{=}1\,000$ $n = 1 000$ sessions per edit. Across both datasets, the gap between Task A and Task B is substantial: Task A AUC reaches $\ge 0.9997$ $\geq 0.9997$ for every model on Yoochoose—driven entirely by the purchase token—while Task B AUC ranges from 0.685 (logistic regression) to 0.755 (LightGBM), with sequence models clustering near the lower end at this dataset size. The Transformer matches the LSTM in mean discrimination but is roughly 50 $\times $ $\times$ more stable across random seeds. Pre-purchase motifs reveal interpretable conversion drivers without the trivially-purchase-tokenized motifs that contaminate naive analyses. The study positions Task A/B contrast and statistically grounded multi-seed evaluation as a default protocol for clickstream conversion benchmarks.