Performance Discrepancy on PEMS08 Dataset - GraphBranch Results #1
Description
Issue Description
I'm trying to reproduce the results reported in the paper for the GraphBranch model on the PEMS08 dataset, but I'm getting significantly different metrics. My MAE and RMSE are much better than reported, while MAPE is slightly worse.
Results Comparison
| Metric | My Results | Paper (Table E1) | Difference |
|---|---|---|---|
| MAE | 20.37 | 29.12 | -8.75 (better) |
| RMSE | 31.39 | 41.36 | -9.97 (better) |
| MAPE | 14.16% | 13.54% | +0.62% (worse) |
My Setup
Environment
- Python: 3.10
- PyTorch: 1.12 (CUDA 12.1 compatible)
- Dataset: PEMS08.npz (17856 timesteps, 170 nodes, 3 features)
Training Configuration
python main.py \
--dataset PEMS08 \
--expid pems08_graph \
--model_name GraphBranch \
--epochs 200 \
--batch_size 32 \
--learning_rate 0.001 \
--train_ratio 0.1 \
--val_ratio 0.1 \
--device 0