Time Series Forecasting: Methods, Pitfalls and How to Evaluate Forecasts

Forecasting is one of the most valuable things data science does for a business: how much inventory to order, how many support agents to schedule, what revenue to expect next quarter, how much server capacity to buy. It's also one of the easiest to get subtly wrong, because time series break assumptions that ordinary machine learning relies on.

#Key takeaways

• Understand your series first: trend, seasonality, cycles, noise, and structural breaks.
• Always start with simple baselines (naive and seasonal naive). They're surprisingly hard to beat.
• Classical methods (exponential smoothing, ARIMA) work well for individual series; ML models shine with many related series and rich features.
• Never randomly shuffle time series for validation. Use rolling-origin backtesting.
• Provide prediction intervals, not just point forecasts. Decisions depend on uncertainty.

#Components of a time series

• Trend: long-term increase or decrease.
• Seasonality: patterns that repeat at fixed periods (daily, weekly, yearly).
• Cycles: longer, irregular fluctuations (business cycles).
• Noise: random variation.
• Events and breaks: holidays, promotions, outages, pricing changes, a pandemic.

Plot the data before modeling anything. Decomposition tools separate trend and seasonality so you can see them clearly.

#Start with baselines

If a sophisticated model can't beat seasonal naive in backtesting, it isn't adding value. That happens more often than you'd expect.

#Classical statistical methods

#Exponential smoothing (ETS / Holt-Winters)

Weighted averages where recent observations count more. Variants handle trend and seasonality (additive or multiplicative). Fast, robust and a strong default for many business series.

#ARIMA / SARIMA

Models a series using its own past values (autoregressive terms), past forecast errors (moving average terms) and differencing to remove trends. SARIMA adds seasonal terms. Powerful, but requires care: stationarity, parameter selection and diagnostics.

#Regression with time features

Linear regression on trend, seasonal dummies, holidays and known drivers (price, marketing spend). Interpretable, and it handles external variables naturally.

pythonfrom statsmodels.tsa.holtwinters import ExponentialSmoothing

model = ExponentialSmoothing(
    train, trend="add", seasonal="add", seasonal_periods=7
).fit()
forecast = model.forecast(28)

#Machine learning approaches

Gradient-boosted trees and neural networks can forecast by turning the problem into supervised learning with lag features:

pythondf["lag_1"]  = df.sales.shift(1)
df["lag_7"]  = df.sales.shift(7)
df["rolling_mean_28"] = df.sales.shift(1).rolling(28).mean()
df["day_of_week"] = df.date.dt.dayofweek
df["is_holiday"] = df.date.isin(holidays)
df["promo"] = df.promo_flag

Note the shift(1) before rolling: features must only use information available before the forecasted time, or you'll leak the future into training (feature engineering).

ML models shine when:

• You forecast many related series (thousands of products or stores) and a global model learns shared patterns.
• There are rich external drivers: promotions, prices, weather, events.
• Relationships are non-linear.

Classical models shine when:

• You have a small number of series with clear structure.
• Data is limited.
• Interpretability and robustness matter more than squeezing out accuracy.

Deep learning and pre-trained time series foundation models are increasingly used, particularly at scale, but they still need to beat simple baselines in your backtests.

#Validation: backtesting, not random splits

Random train/test splits leak the future into training. Use rolling-origin (walk-forward) evaluation:

textFold 1: train [Jan–Jun]  → test [Jul]
Fold 2: train [Jan–Jul]  → test [Aug]
Fold 3: train [Jan–Aug]  → test [Sep]

Evaluate at the forecast horizon you actually use: if you plan inventory four weeks ahead, measure four-week-ahead accuracy, not one-day-ahead.

#Forecast accuracy metrics

Choose metrics aligned with costs. If under-forecasting (stockouts) costs more than over-forecasting (excess inventory), evaluate and optimize for that asymmetry, for example by forecasting a higher quantile.

#Prediction intervals

A forecast of "1,000 units" is less useful than "1,000 units, 80% interval 850–1,180". Intervals drive decisions like safety stock, staffing buffers and budget ranges. Get them from:

• Statistical models, which provide intervals natively.
• Quantile regression with ML models.
• Conformal prediction or empirical residuals from backtests.

Check interval coverage in backtesting: an 80% interval should contain the actual value about 80% of the time.

#Common pitfalls

1. Leakage through features computed with future data.
2. Ignoring holidays and events, so promotions look like random spikes.
3. Structural breaks: training on data from before a major change (a new pricing model, a pandemic period) without adjusting.
4. Forecasting the wrong thing: sales capture only what inventory allowed you to sell. Stockouts hide true demand.
5. Over-aggregation or under-aggregation: forecasting at SKU-store-day level when decisions happen at category-week level, or vice versa. Consider hierarchical reconciliation so forecasts add up across levels.
6. No monitoring: accuracy degrades as conditions change (MLOps explained).

#A practical forecasting workflow

1. Define the decision, the horizon, the granularity and the cost of errors.
2. Clean the data: handle missing periods, outliers and known anomalies (data cleaning with pandas).
3. Plot and decompose. Identify seasonality and events.
4. Build baselines and backtest them.
5. Try exponential smoothing or ARIMA; then ML with lag and calendar features if there are many series or rich drivers.
6. Compare with rolling-origin backtests at the relevant horizon.
7. Produce intervals and validate their coverage.
8. Deploy as a scheduled batch job, monitor accuracy, and retrain on a schedule.

#FAQ

#Which forecasting method is best?

There's no universal winner. Simple methods often perform competitively, especially with limited data. Global ML models tend to win with many related series and strong external drivers. Backtesting against baselines on your own data is the only reliable way to choose.

#How much history do I need?

Enough to capture the seasonality you care about. For yearly seasonality, ideally two or more years. With limited history, simpler models and pooling information across related series help.

#Can I use regular cross-validation for time series?

Not with random splits, which leak future information. Use time-ordered splits, such as rolling-origin evaluation, where each fold trains on the past and tests on the following period.

#How do I forecast the effect of a promotion?

Include promotion features (type, discount, duration) in a regression or ML model trained on past promotions, and validate on held-out promotions. For causal effects of new interventions, experiments give stronger evidence (A/B testing done right).