Robust Bus Dwell Time Predictive Machine Learning Model by Handling Real Concept Drift
Description
This research focuses on improving the prediction of Bus Dwell Time (BDT) in developing countries like Sri Lanka, where traffic is highly variable. By addressing concept drift—changes in data patterns due to factors like traffic, weather, or incidents—the project aims to enhance prediction accuracy using real-time GPS data and adaptive machine learning models.
Motivation and Research Objective
The goal is to develop a robust BDT prediction model that can detect and adapt to real-time concept drift using an active learning strategy and a novel batch processing method. The research explores various concept drift detection and incremental learning algorithms to identify the best combination that minimizes prediction error and ensures reliable public transport insights.
Result and Impact
Our proposed method for handling real-time concept drift in Bus Dwell Time (BDT) prediction achieved a 17.35% improvement in Mean Absolute Error (MAE) and a 3.48% reduction in Root Mean Square Error (RMSE) compared to the baseline model without drift handling. By introducing a novel Hybrid Batch Processing (HBP) system and combining it with an active, bufferless strategy and incremental learning, we significantly reduced prediction latency (MAPT) by over 55%, making the system more adaptive and resource-efficient.
Compared to other strategies, our method not only performed better in terms of prediction accuracy but also adapted faster to real-world changes such as traffic, weather, and accidents. In contrast, retraining models from scratch delayed drift adaptation and led to poorer performance.
Our work stands out in three ways:
-
It robustly handles real concept drift in BDT prediction.
-
It surpasses existing baseline approaches using XGBoost.
-
It introduces a Hybrid Batch Processing system that better manages data scarcity than traditional periodic/event-based methods.
Technical Contributions
-
Developed a Python package to convert raw GPS data into GTFS format for public transit.
-
Built a modular Python framework supporting incremental learning and concept drift strategies for BDT.
-
Submitted findings to the 27th IEEE International Conference on Intelligent Transportation Systems.
Team Members
Kesavi Aravinthan
kesavi.19@cse.mrt.ac.lk
Department of Computer Science and Engineering,
University of Moratuwa, Sri Lanka.
Gopinath Shanmugavadivel
gopinath.19@cse.mrt.ac.lk
Department of Computer Science and Engineering,
University of Moratuwa, Sri Lanka.
Kajanan Selvanesan
kajanan.18@cse.mrt.ac.lk
Department of Computer Science and Engineering,
University of Moratuwa, Sri Lanka.