Demand Forecasting Model
Using machine learning to predict hourly bike demand at Columbia stations for optimized rebalancing and capacity planning.
Research Question
Can we accurately predict hourly bike demand at Columbia stations to optimize rebalancing operations and improve user experience?
Machine Learning Pipeline
Our demand forecasting system follows a standard machine learning pipeline, from raw trip data to actionable hourly predictions.
Pipeline Overview
The pipeline transforms 529,908 raw trip records into hourly demand forecasts through systematic data preparation, feature engineering, model training, and evaluation stages. Each stage builds upon the previous one to create increasingly refined predictions.
1. Data Preparation
The first step transforms raw trip-level data into hourly station-level records suitable for time series forecasting.
Data Processing Steps
Train/Validation/Test Split
Time-based split ensures no data leakage: the model never sees future data during training.
Loading chart...
Why Time-Based Splitting?
Unlike random splitting, time-based splitting mimics real-world deployment: the model trains on historical data and predicts future demand. This prevents data leakage and provides realistic performance estimates. The test set (October 2025) represents genuine future predictions.
2. Feature Engineering
Feature engineering transforms raw hourly departure counts into 27 predictive features that capture multiple dimensions of bike demand patterns. Each feature category addresses a specific aspect of what drives demand at Columbia stations.
Temporal Features (8)
- • Cyclical encoding: hour, day of week, month (sine/cosine)
- • Rush hour indicator (7-9am, 4-6pm weekdays)
- • Weekend flag
Captures daily and weekly usage cycles
Academic Calendar (6)
- • Semester period, holidays, finals weeks
- • Study days, spring/winter breaks
- • Days since semester start
Columbia-specific seasonal patterns
Lag Features (5)
- • Departures 1 hour, 24 hours, 1 week ago
- • Arrivals 1 hour ago
- • Total trips (departures + arrivals) 1 hour ago
Recent demand momentum and weekly patterns
Aggregated Features (8)
- • Rolling averages: 24-hour, 7-day windows
- • System-wide departures (all 7 stations) 1h ago
- • System-wide total trips 1h ago
- • Historical average by station-hour-daytype
- • Interaction features (e.g., hour × is_weekend)
- • Station encoding
Network-wide trends and baseline patterns
Key Design Choice: Cyclical Encoding
For temporal features like hour, day of week, and month, we use cyclical encoding (sine/cosine transformations) instead of raw numeric values. This preserves the circular nature of time: hour 23 (11pm) and hour 0 (midnight) are numerically 23 units apart, but in reality they're only 1 hour apart. This allows the model to correctly learn that late-night and early-morning hours have similar demand patterns, despite being distant in numeric value.
3. Model Training
We compared four modeling approaches to identify the best predictor of hourly bike demand.
Models Compared
Loading chart...
Key Insight
XGBoost achieves the best performance with MAE = 1.63 departures/hour and R² = 0.722. This represents a 50% improvement over the baseline (MAE = 3.26), demonstrating that machine learning patterns significantly outperform simple historical averages. The negative R² for the baseline (-0.259) indicates that historical averages perform worse than simply predicting the mean for all samples.
4. Model Evaluation
Rigorous evaluation on held-out October 2025 test data ensures the model generalizes to unseen future periods.
MAE
1.63
Mean Absolute Error (departures)
Average prediction error magnitude
RMSE
2.363
Root Mean Squared Error (departures)
Penalizes large errors more heavily
R² Score
0.722
Variance Explained (0-1 scale)
Proportion of variance captured by model
Prediction Error Distribution
The distribution of prediction errors (actual - predicted) shows model bias and accuracy. Ideally, errors center near zero with minimal spread.
Loading chart...
Evaluation Results
The error distribution is roughly centered near zero, indicating unbiased predictions. The MAE of 1.63 departures means the model's average error is less than 2 bikes per hour—highly accurate for operational planning. The R² of 0.722 means the model explains 72% of demand variance, significantly better than the baseline's negative R².
5. Feature Importance Analysis
Understanding which features drive predictions reveals the key factors influencing bike demand.
Loading chart...
Top Predictive Features
- 1.System departures lag (1h): Total departures across all Columbia stations 1 hour ago. Recent system-wide activity is the strongest predictor—when the entire network is busy, individual stations will be too.
- 2.System total trips lag (1h): Total system activity (arrivals + departures) from the previous hour. Captures overall bike circulation momentum.
- 3.Historical average: Average departures for this station-hour-daytype combination. Recurring patterns across time provide a stable baseline.
6. Making Predictions
The trained model generates hourly demand forecasts by processing the 27 engineered features through the XGBoost ensemble.
Loading chart...
Prediction Quality
The model successfully captures temporal demand patterns including rush hour peaks (morning and evening) and off-peak valleys. It tracks actual departures closely throughout October 2025 test period. Occasional spikes (likely due to special events or weather) are underestimated, but overall the predictions enable proactive rebalancing 1-24 hours in advance.
Key Findings & Applications
Model Performance
XGBoost achieves R² = 0.722 (explains 72% of demand variance) with MAE = 1.63 departures/hour. This represents a 50% improvement over baseline historical averages, demonstrating that learned patterns from 27 engineered features significantly outperform simple heuristics.
Key Predictive Signals
Recent system-wide activity (1 hour ago) is the strongest predictor. The model also relies heavily on historical patterns and academic calendar information, indicating demand follows both immediate momentum and longer-term seasonal/academic cycles. No single factor dominates—the ensemble leverages diverse information.
Practical Applications
- •Proactive Rebalancing: Predict when stations will run out of bikes or become full 1-24 hours in advance, enabling efficient truck routing
- •Capacity Planning: Identify underutilized periods and high-demand times to optimize dock/bike allocation across the 7-station network
- •User Experience: Power mobile app alerts about expected bike availability at preferred stations, reducing user frustration
- •Infrastructure Planning: Use demand forecasts to guide decisions about future station expansion, relocation, or capacity increases
Future Improvements
Potential enhancements include weather data integration (temperature, precipitation), event calendars (concerts, sports games), real-time API deployment for live predictions, per-station model tuning, and multi-step ahead forecasting (24-72 hours).
Methodology & Technical Details
For comprehensive documentation of data processing pipelines, feature engineering techniques, model hyperparameter tuning, and evaluation methodology, visit our methodology page.
View Detailed Methodology →