Skip to main content

Measuring Impact

Learn how to interpret metrics and measure the impact of features on your business.

Key Metrics​

Feature Usage​

  • Evaluation Count: How often the feature is evaluated
  • Enable Rate: Percentage of evaluations that returned enabled
  • User Coverage: Percentage of users who saw the feature

Business Metrics​

  • Conversion Rate: Percentage of users who convert
  • Revenue: Total revenue generated
  • Engagement: User engagement metrics
  • Retention: User retention rates

Performance Metrics​

  • Latency: Response times
  • Error Rate: Percentage of failed requests
  • Throughput: Requests per second

Interpreting Metrics​

Look for trends over time:

  • Increasing: Feature adoption is growing
  • Decreasing: Feature usage is declining
  • Stable: Feature usage is consistent

Comparisons​

Compare metrics:

  • Before/After: Compare before and after feature launch
  • Variants: Compare experiment variants
  • Segments: Compare user segments

Anomalies​

Watch for anomalies:

  • Sudden Spikes: Unusual increases
  • Drops: Unexpected decreases
  • Pattern Changes: Shifts in behavior

Measuring Impact​

Before Launch​

Establish baseline metrics:

  • Current State: Measure current metrics
  • Targets: Define success targets
  • Hypothesis: State expected impact

During Rollout​

Monitor continuously:

  • Real-time Metrics: Watch live metrics
  • Error Rates: Monitor for issues
  • User Feedback: Gather feedback

After Launch​

Measure final impact:

  • Comparison: Compare to baseline
  • Statistical Significance: Check if results are significant
  • Business Impact: Assess business value

Best Practices​

  1. Define Metrics Early: Define metrics before launching
  2. Establish Baselines: Measure current state first
  3. Monitor Continuously: Watch metrics throughout
  4. Compare Appropriately: Use proper comparisons
  5. Document Results: Record findings and decisions

Next Steps​