Checkout conversion dropped 8% last month
The problem
Between 1–28 Feb, completed-checkout rate on web fell from 4.6% to 4.2% (an 8% relative drop). The decline is concentrated in evening hours; the mobile app is flat. No pricing change or major UI release shipped in the window.
Sub-problems
This is a broad problem, so it's split into the sub-problems below. Open each to see its hypotheses, experiments, and conclusion. A sub-problem that turns out to be broad can split again the same way.
Technical failures in checkout
The sub-problem
Determine whether technical faults — server errors, gateway timeouts, or slow responses in the checkout path — are causing abandonment. The evening concentration of the drop strongly hints at load-related failures during peak traffic.
Hypotheses
Experiments
Conclusion
Experiments confirm load-dependent 504 timeouts originating in the payment-service database connection pool (Exp 1.1–1.2). Technical failure is a confirmed contributor to the evening drop.
A funnel step losing users
The sub-problem
Determine whether a specific checkout step is shedding more users than before — independent of technical errors — such as a form change, a stricter validation, or a layout change introduced in the window.
Hypotheses
Experiments
Conclusion
Funnel analysis shows the extra drop-off concentrated at the payment step, matching the technical findings rather than a UX change; form and validation steps are unchanged month over month (Exp 2.1). A funnel/UX change is unlikely to be the primary cause.
Reproduce checkout timeouts under controlled load
1 · Process
Ran a load test against the staging checkout, ramping concurrent checkout sessions from 50 to 300 in steps of 50 and holding each level for five minutes, with production-like data. Recorded HTTP status, end-to-end latency, and database connection-pool metrics at each step.
2 · Data
Chart 1 · 504 errors vs concurrent checkout sessions.
3 · Analysis
Errors are near-zero below 200 concurrent sessions and rise sharply above it, tracking database connection-pool saturation step for step. The relationship is load-dependent and matches the evening-peak pattern seen in production.
4 · Conclusion
The 504 timeouts are reproducible and load-dependent, beginning at roughly 200 concurrent checkouts as the payment-service DB pool saturates.
Step-by-step latency during peak
1 · Process
Instrumented the checkout path to record p95 latency per step (cart → address → payment → confirm) and sampled across three evening peaks, comparing against the January baseline.
2 · Data
| Step | Jan p95 | Feb p95 |
|---|---|---|
| Cart | 0.3s | 0.3s |
| Address | 0.5s | 0.5s |
| Payment | 1.1s | 6.2s |
| Confirm | 0.4s | 0.4s |
3 · Analysis
Nearly all of the added latency is in the payment step (1.1s → 6.2s); every other step is unchanged. This isolates the degradation to payment-service, consistent with Exp 1.1.
4 · Conclusion
Latency degradation is isolated to the payment step; the rest of the funnel is healthy.
Funnel drop-off by step (Jan vs Feb)
1 · Process
Compared step-to-step conversion in the checkout funnel for January vs February using the analytics export, looking for any step whose pass-through rate fell.
2 · Data
| Step → next | Jan | Feb |
|---|---|---|
| Cart → Address | 88% | 88% |
| Address → Payment | 81% | 80% |
| Payment → Confirm | 74% | 66% |
3 · Analysis
Pass-through fell only at Payment → Confirm (74% → 66%); the form and validation steps are unchanged. The loss coincides with the payment-step latency and timeouts found in Sub-problem 1 — a technical cause, not a funnel redesign.
4 · Conclusion
No abnormal funnel loss outside the payment step; a UX/funnel change is unlikely to explain the drop.