SuperRocket

SuperRocket: Efficiently Selecting Kernel-based Transformations for Fast and Accurate Time Series Classification

Introduction

时间序列分类相关工作

UCR Archive: 85 datasets

TSC领域中的训练测试数据集. 评测指标包括Euclidean distance, DTW(Dynamic Time Warping), 以及基于kernel的方法.

HIVE-COTE 2.0 Machine Learning (2021)

四种SOTA方法：HIVE-COTE 2.0, ROCKET, STC, TSC

• the deep learning approach called InceptionTime (2020)
• the tree based Time Series Combination of Heterogeneous and Integrated Embedding Forest(TS-CHIEF) (2020)
• the Random Convolutional Kernel Transform (ROCKET) (2020)
• the heterogeneous meta-ensemble Hierarchical Vote Collective of Transformation-based Ensembles (HIVE-COTE) (2018)

ROCKET Data Mining and Knowledge Discovery (2020)

随机卷积核变换, 训练开销极小, 但是性能优秀.

Rocket uses random kernel length, dilation, and padding. 使用线性激活函数, 全局最大池化和ppv池化.

使用随机卷积核提取特征, 用得到的值训练一个线性分类器.

岭回归分类器

MiniROCKET KDD (2021)

确定的,更小的卷积核

MultiROCKET Data Mining and Knowledge Discovery (2022)

额外的一阶差分 和 4个池化操作

从大的候选集选择一个合适的kernel set

select a subset of transformations - NP hard

选择子集是有效的, 如下图所示, 随机1000次选择, 找到最优的那个子集.

子集选择的一些相关工作

• S-ROCKET 进化算法探索子集
• POCKET group elastic net, variant of the embedded method LASSO
• Detach-ROCKET wrapper method based on a backward-stepwise

以上方法是裁剪现有模型而不是探索如何选择更高质量的模型.

SuperRocket 高效选择高质量的kernel set.

额外添加indicating vector, 用于表示每个kernel是否被选择.

Contributions:

1. tackle the challenge of efficiently selecting a subset of kernel-based transformations that can produce high-quality features for TSC
2. Method: SuperRocket, a novel efficient kernelbased transformation selection approach that determines all transformations simultaneously by learning a continuous-valued indicating vector. model it bi-level optimization problem
3. Experiments

Related Work

TSC

1. distance-based methods. eg DTW
2. feature-based approaches eg logistic regression, random forest
   1. interval-based
   2. shapelet-based features
   3. dictionary-based features
   4. kernel-based features
3. deep learning methods.
4. hybrid TSC approaches.

Kernel-based Transformation Selection

1. general variable or feature selection methods in the traditional machine learning

the filter, wrapper, and embedded methods

2. the customized transformation selection approaches.

PRELIMINARIES

Time Series Classification

X -> y

preprocessing,convolution,pooling

ridge regression

优化目标