Mondays are often the busiest day of the week for analysts. Not only do we have two additional days of data since Friday, but stakeholders begin reviewing their dashboards and expecting insights. As analysts, we must proactively identify potential data issues and provide actionable next steps. Below, I outline my process for investigating anomalies, for example: what if I observe a decline WoW conversion rate?

Step 1: Check for Pipeline Issues

First, I review the data engineering channel to identify any known pipeline failures that could affect downstream calculations. This ensures the anomaly isn’t due to a technical issue, such as a data ingestion error or ETL process breakdown.

Step 2: Assess Historical Trends

Next, I extend the timeframe by a few weeks or months to determine if the decline falls within normal variation or warrants flagging as an outlier. This involves comparing the current conversion rate to historical patterns to contextualize the drop.

Step 3: Segment the Data

I then drill down into the conversion rate by different dimensions. For example, is the decline specific to paid traffic or desktop users? This helps pinpoint the issue. I collaborate with relevant teams to investigate potential causes, such as campaign cannibalization or increased competitor activity.

Step 4: Decompose the Metric

To understand the root cause, I break down the conversion rate into its components: conversions (numerator) and sessions (denominator). This reveals whether the decline stems from a surge in sessions, a drop in conversions, or both. I further analyze each component at a granular level to isolate the problem.

Step 5: Document and Communicate

Once the issue is identified, I create a JIRA ticket for stakeholders and, if needed, data engineers. The ticket includes the problem context, estimated revenue impact, and an ETA for resolution. This keeps everyone informed, allowing teams to reprioritize their workloads based on the issue’s severity.

Conclusion

Anomaly detection is a critical skill for analysts, requiring a structured approach to identify, investigate, and communicate issues. By systematically checking pipelines, analyzing trends, segmenting data, decomposing metrics, and documenting findings, analysts can drive actionable outcomes and maintain trust with stakeholders.