Seasonality plays a major role in business performance, and without properly accounting for it, marketing experiments can produce misleading results. Seasonal shifts—like Black Friday, holiday shopping, or even weather-driven trends—can impact sales and customer behavior, making it critical to ensure that experiments reflect true marketing impact rather than external fluctuations.

​
How Dema accounts for seasonality

Dema uses a time-based comparison approach in which both treatment and control regions experience the same seasonal conditions. Because both groups are exposed to the same external factors, such as increased demand during peak shopping periods, any observed differences in performance can be attributed to marketing efforts rather than seasonal changes.

​
Synthetic control for seasonality adjustment

Dema’s synthetic control methodology automatically adjusts for seasonal effects by comparing pre-test and post-test performance between treatment and control regions. This means that fluctuations in customer demand, such as holiday-driven sales spikes, are reflected in both groups, ensuring that the test measures incrementality, not seasonality.

​
How external factors impact seasonality adjustments

​
Testing during peak periods (e.g., Black Friday, holiday sales)

If you’re testing during a major promotional period, the impact of seasonality is already baked into both the treatment and control groups, so the test remains valid. However, it’s important to consider whether your marketing effectiveness might change during promotions.
Example: A Meta campaign running during Black Friday might show stronger performance than usual, but that doesn’t mean the same results will apply outside of peak shopping events.
For the best insights, we recommend testing in both promotional and non-promotional periods to understand how marketing impact varies.

​
How regional promotions affect results

If a promotion applies nationwide, it should not interfere with the experiment since both test and control regions are exposed to the same offer.
If the promotion applies only to certain regions, those areas should be excluded from the experiment. Regional price differences or localized discounts can distort results and make it harder to isolate the true impact of marketing.

​
Weather and other unpredictable factors

Dema’s approach naturally adjusts for broad weather-driven effects. If weather affects sales in both test and control regions equally (e.g., an unseasonably warm winter that boosts apparel sales nationwide), the experiment remains valid. However, if extreme weather events affect only some test regions (e.g., hurricanes, snowstorms), additional adjustments may be necessary to prevent skewed results.

​
Running experiments during softer business periods

Seasonality isn’t just about peak sales—it’s also important to consider how testing during low-demand periods can impact results.
  • If budgets are significantly reduced during these periods, it can be difficult to achieve statistical power.
  • Instead of scaling back testing, softer periods can be used for strategy optimization tests on larger channels.
  • Staying active in low-demand periods can help ensure strong algorithmic learning for when demand picks up again.

​
Handling unexpected volatility

Even with seasonality adjustments, unexpected volatility (e.g., supply chain disruptions, major economic shifts) can introduce anomalies in experiments. Dema incorporates outlier detection and cleaning strategies to smooth out localized spikes. If volatility is evenly distributed across test and control regions, it will balance out over time. However, if a major, localized event disrupts only part of the experiment, results may require further evaluation.

​
Summary: Best practices for seasonality in experiments

  • Always test with control groups exposed to the same seasonal conditions.
  • Consider testing in both peak and non-peak periods to understand how marketing effectiveness varies.
  • Exclude regions with unique external factors, such as localized promotions or extreme weather events.
  • Be mindful of low-demand periods, ensuring there is enough budget to maintain statistical power.
  • Use historical data to estimate expected variability, ensuring experiments are set up for accuracy.