jiaxiang-cheng / pytorch-transformer-for-rul-prediction

Transformer implementation with PyTorch for remaining useful life prediction on turbofan engine with NASA CMAPSS data set. Inspired by Mo, Y., Wu, Q., Li, X., & Huang, B. (2021). Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unit. Journal of Intelligent Manufacturing, 1-10.

PyTorch Transformer for RUL Prediction

An implementation with Transformer encoder and convolution layers with PyTorch for remaining useful life prediction.
Author: Jiaxiang Cheng, Nanyang Technological University, Singapore

Quick Run

Simply run python train.py --dataset FD001. And you will get the training loss and testing result for each epoch where the RMSE is from the test set:

Epoch: 0, loss: 9474.43470, RMSE: 61.11946
Epoch: 1, loss: 5858.27227, RMSE: 46.03318
Epoch: 2, loss: 3208.53410, RMSE: 29.78244
Epoch: 3, loss: 1310.71390, RMSE: 22.94705
...

The testing is conducted for each epoch as the data set is not large so it's no big deal but you may remove them and only do the evaluation after finishing the training epochs.

Environment Details

python==3.8.8
numpy==1.20.1
pandas==1.2.4
matplotlib==3.3.4
pytorch==1.8.1

Credit

This work is inpired by Mo, Y., Wu, Q., Li, X., & Huang, B. (2021). Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unit. Journal of Intelligent Manufacturing, 1-10.

Citation

License

pytorch-transformer-for-rul-prediction's Issues

Paper

Hello, may I ask that the paper you reproduced is that one? Can you send a link?

找不到模型中decoder模块

你好，jiaxiang
最近了解了一下transformer模型，大都由encoder和decoder模块组成，但是在code中没有找到decoder模块，是什么原因呢？

Errors in data processing

There is a problem with the data processing in this code. If the results in the paper are also processed in this way, it is impossible to get the correct results. Can you provide the complete code for me to do the reproduction work?

RMSE

Hello,

firstly, thank you for your code, it's awesome.

I have run the code without any changes, but I haven't got the same RMSE as yours.
is there any change I must do before run ?

CMAPSS的结果

从这篇论文"KDnet-RUL: A Knowledge Distillation Framework to Compress Deep Neural Networks for Machine Remaining Useful Life Prediction"。 FD002 14.47. FD004 15.96. 它的FD002和FD004效果很好，你可以复现类似的结果吗。我看你复现的这个论文结果FD002 22.81 FD004 24.86并不好。想follow一下，感觉follow了一个大坑。

Gated Convolution

In the code, the convolution operations is applied for the future time stamp also during the training. Example: for the second iteration (t=1), the stack of three time steps (0,1,2) is used for local context extractor (i.e. the convolutional block with gating unit). But doesn't the gated convolution be limited to the current and previous position only (i.e. only applying convolution on 0,1 stacked vector)

Miss method to get "example with unit 43 in FD001.png"

Excuse me, is there any way to draw "example with unit 43 in FD001.png"?

请问下面这个图片是如何生成的

Why my rmse is more than 40

Hello, this is the first time I have asked a question on github. If you offend you, please forgive me.Why my rmse is more than 40 .I hope you have time to answer this question, thank you