Success metrics that don't lie
Vanity metrics
Signups, logins, clicks. They almost always go up and prove nothing. A good metric goes down when you do something bad — that's the test.
Replace “how many entered” with “how many completed the action they came for,” and “how many signed up” with “how many returned in the second week.”
A pair of metrics, not one
Every primary metric needs a balancing one. Speed versus quality. Automation rate versus satisfaction. Revenue versus cancellations. Without the balance, any team will find a way to improve the primary metric in a way that hurts elsewhere.
Put both numbers on the same board, side by side.
Before and after
Measure two weeks before release. Without a baseline, any post-release number is a story you can tell either way. If possible, release to some users and compare to the group that stayed on the current state — that's the only way to separate impact from seasonality.
Going deeper
Define the metric as a saved query, not a number someone computes by hand once a month. A metric that requires manual work stops being measured exactly when it becomes interesting. Add breakdowns: by user type, by tenure, by channel — a stable average often hides a sharp decline in one segment.