Metrics

A metric is a number computed from a set of events you log. It is the thing you watch while a flag ramps, and the thing an alert rule compares against a threshold. Get the definition right and the rest of the observability story follows.

Aggregation types

Shipeasy ships four aggregation functions. Each one collapses a user's event stream into a single number per analysis window:

Conversion is the simplest and statistically the friendliest — the variance is bounded by p(1-p), so power calculations are cheap and the t-test behaves. Means and sums need more samples and benefit from outlier handling (see below).

When the metric is "no event"

For count_users (the binary "did it happen?" aggregation), a user with zero matching events contributes 0. For avg, the DSL averages across all exposed users — non-purchasers contribute 0 and pull the mean down. To answer "average among buyers only" instead, express it as a ratio (ratio(sum(purchase, revenueCents), count_users(purchase))) or compute the per-buyer cohort metric offline.

Creating a metric

Metrics are defined by a small query DSL — the same one the dashboard generates when you pick aggregation, source event, and filters in the "New metric" form. Run shipeasy metrics grammar for the full spec; the most common shapes:

# binary conversion on `purchase`
shipeasy metrics create purchase_conversion \
  --event-name purchase --query 'count_users(purchase)'

# revenue per user (includes non-buyers as $0)
shipeasy metrics create revenue_per_user \
  --event-name purchase --query 'sum(purchase, value)'

# average order value (averaged across buyers only)
shipeasy metrics create avg_order_value \
  --event-name purchase --query 'avg(purchase, value)'

# sessions per user
shipeasy metrics create sessions \
  --event-name session_start --query 'count(session_start)'

Or in the dashboard: Metrics → New metric.

Filtering events into a metric

You can tighten what counts toward a metric with filters — written inline in the DSL selector as event{attr=value}. Filters run against the event's properties payload before the aggregation:

shipeasy metrics create organic_purchase \
  --event-name purchase \
  --query 'count_users(purchase{channel="organic"})'

Now organic_purchase only counts users with at least one purchase event whose channel property equals "organic". Compose multiple filters inside the same {} with commas — they're ANDed. Operators are = (equal), != (not equal), =~ (regex match), !~ (regex non-match). Common shapes:

# Web purchases only (exclude mobile app)
'count_users(purchase{platform="web"})'

# Orders priced above $10 (string-coerced, server compares numerically
# when the source event declares the label as numeric)
'sum(purchase{value=~"^[1-9][0-9]+$"}, value)'

# Multiple conditions — commas inside {} are ANDed
'count_users(purchase{platform="web", country="US"})'

Outliers

For sum and mean, a single $50,000 enterprise purchase can swing the mean for thousands of users. Shipeasy supports two outlier handlers per metric:

• Winsorise at a configurable percentile (default p99). Anything above is clamped to the p99 value of the combined sample. Default for sum and mean.
• Cap at an absolute value. Use this when there's a domain-specific ceiling (e.g. a maximum plausible session length).

# Winsorise at p99 (this is the CLI default; --winsorize 99 is implicit)
shipeasy metrics create revenue_per_user \
  --event-name purchase --query 'sum(purchase, value)' \
  --winsorize 99

# Cap at p95 instead
shipeasy metrics create revenue_per_user \
  --event-name purchase --query 'sum(purchase, value)' \
  --winsorize 95

Winsorising is the default and is rarely wrong. The trade-off: clamping reduces variance (good — a steadier series, so a threshold alert stops flapping on one big order) at the cost of slightly understating a real move in the tail (rare).

Ratio metrics

Some questions are inherently ratios — "clicks per impression", "conversion per visit". Express them with the over keyword between two selector arms in the DSL:

shipeasy metrics create click_through_rate \
  --event-name click \
  --query 'ratio(count(click), count(impression))'

The two ratio arms must each be count or count_users — the DSL doesn't allow sum/avg in ratio position.

Ratio metrics use the delta method to compute variance correctly (the naive ratio-of-means understates variance, which makes an anomaly rule fire on noise). The dashboard shows the numerator and denominator alongside the ratio so the math is auditable.

API · metrics.create

Where to next