Description

Panel time series forecasting presents unique challenges due to the heterogeneity of panel datasets, where existing models fail to perform consistently across diverse panel data characteristics. While the Temporal Fusion Transformer (TFT) has proven effective for univariate and multivariate time series, it exhibits critical limitations when applied to panel data: its standard attention mechanisms struggle to capture localized temporal patterns, it lacks mechanisms for modeling inter-entity dependencies, and it processes all temporal information through a single pathway that may overlook important trend-seasonal distinctions.

Motivation and Research Objectives

The primary motivation behind this project is to address the fundamental limitations of existing transformer architectures in panel time series forecasting, with a focus on creating a unified solution that can handle diverse panel data characteristics. This is addressed through five key research objectives:

1. Enhanced Attention Mechanisms – To implement segment-wise attention for localized temporal pattern extraction and improved computational efficiency, reducing complexity from O(n²) to O(n·s).

2. Inter-Entity Modeling – To develop cross-entity attention mechanisms that capture dependencies between entities, leveraging shared patterns across the entire panel for improved forecasting accuracy.

3. Component Decomposition – To create multi-scale series decomposition using learnable multi-scale moving averages that adaptively separate trend and seasonal components across diverse temporal patterns.

4. Parallel Processing Architecture – To design specialized processing pathways that handle seasonal and trend components independently, with enhanced TFT for seasonal patterns and GRN-based processors for trend components.

5. Adaptive Integration – To implement dynamic component weighting strategies that optimally combine processed components based on forecasting context and panel data characteristics.

Result and Impact

Key Achievements

• Superior Performance: Achieved 7.99% average improvement in Mean Absolute Percentage Error (MAPE) over baseline TFT

• Consistent Results: Demonstrated superior performance on 9 out of 11 diverse datasets

• Computational Efficiency: Reduced attention complexity from O(n²) to O(n·s) through segment-wise attention

• Robust Architecture: Successfully handled eight distinct panel type combinations across balanced/unbalanced, micro/macro, and short/long panel categories

Architectural and Benchmark Contributions

This project introduced two major contributions to the time series forecasting community:

• Novel Architecture Design: The first transformer-based approach specifically designed for panel time series with integrated multi-scale decomposition, featuring five key innovations that work synergistically to address panel data complexity.

• Comprehensive Panel Categorization: A three-dimensional panel data categorization framework across completeness (balanced vs. unbalanced), aggregation level (micro vs. macro entities), and temporal span (short vs. long panels), enabling systematic evaluation across eight distinct panel types.

Broader Impact

The outcomes of this work have significant implications for both academic research and real-world applications:

• For Time Series Researchers: Panelformer provides a scalable baseline for panel time series forecasting and sets a new standard for performance across diverse panel characteristics. It encourages the use of segment-wise attention and cross-entity modeling in temporal forecasting.

For Industry Applications: By supporting diverse panel types including electricity consumption, climate monitoring, financial markets, and economic indicators, Panelformer can be integrated into energy management systems, financial trading platforms, and economic forecasting tools, enhancing their predictive accuracy and reliability.

Team Members

Sathurgini Uthayakumar 
sathurgini.20@cse.mrt.ac.lk
Department of Computer Science and Engineering,
University of Moratuwa

Kajaani Balabavan
kajaani.20@cse.mrt.ac.lk
Department of Computer Science and Engineering,
University of Moratuwa

Shabthana Johnson
shabthana.20@cse.mrt.ac.lk
Department of Computer Science and Engineering,
University of Moratuwa