aws-samples / lookout-for-equipment-demo

Amazon Lookout For Equipment

---

Overview

Amazon Lookout for Equipment uses the data from your sensors to detect abnormal equipment behavior, so you can take action before machine failures occur and avoid unplanned downtime.

Amazon Lookout for Equipment analyzes data such as pressure, flow rate, RPMs, temperature, and power to automatically train a specific ML model based on just your data, for your equipment – with no ML expertise required.

Lookout for Equipment uses your unique ML model to analyze incoming sensor data in real-time and accurately identify early warning signs that could lead to machine failures.

This means you can detect equipment abnormalities with speed and precision, quickly diagnose issues, take action to reduce expensive downtime, and reduce false alerts.

Demo notebooks

This folder contains various examples covering Amazon Lookout for Equipment best practices. Open the getting_started folder to find all the ressources you need to train your first anomaly detection model. The notebooks provided can also serve as a template to build your own models with your own data.

In the getting_started folder, you will learn to:

1. Prepare your data for use with Amazon Lookout for Equipment
2. Create your own dataset
3. Train a model based on this dataset
4. Evaluate a model performance and get some diagnostics based on historical data
5. Build an inference scheduler and post-process the predictions

Blogs

In this folder, you will find technical content associated to blog posts AWS writes about Amazon Lookout for Equipment.

Preprocessing

Multivariate industrial time series data can be challenging to deal with: these samples will show you how to explore your data, improve data quality, label your anomalies (manually or automatically), etc.

Security

See CONTRIBUTING for more information.

License

This collection of notebooks is licensed under the MIT-0 License. See the LICENSE file.

lookout-for-equipment-demo's Issues

Incorrect model name in file getting_started/notebooks/5_inference_scheduling.py

Hi,
This macro should be MODEL_NAME_FOR_CREATING_INFERENCE_SCHEDULER = 'lookout-demo-model-v1' , instead of 'lookout-demo-model-v4'

getting_started/notebooks/2_data_creation.ipynb need additional code below

getting_started/notebooks/2_data_creation.ipynb need additional code below

lookout_client = boto3.client('lookoutequipment')
data_ingestion_job_id = response['JobId']

before the cell of

describe_data_ingestion_job_response = lookout_client.describe_data_ingestion_job(JobId=data_ingestion_job_id)
describe_data_ingestion_job_response

A problem with scheduler's local time zone adjustment

Hello. I'm working as a Data Scientist at AWS ProServe Korea, and I've tested Lookout for Equipment to try it out on an enterprise project. Then I found a point in need of improvement.

When the scheduler finds the timestamp in the data file name for the current time, I can adjust the local time zone by specifying the 'INPUT_TIMEZONE_OFFSET' parameter. For example, if the current is April 3rd at 12:00 AM UTC, in Korea (UTC + 9) I can enter the parameter as '+9:00' and specify the timestamp of the file name as '20210403090000'. This works fine.

However, when the scheduler filters the timestamp column inside the data file, it does not take into account local time zone adjustments. In other words, if the timestamp field is '2021-04-03 09:00:00', I get an error. It should be put in UTC standard.

This difference is confusing to customers and inconvenient because you have to deal with file names and data timestamps in different ways. I hope my feedback is helpful. Thank you.

unzip is unavailable on the python data science kernel

I ran %conda install -y -c conda-forge unzip to install it

timeseries data is not matching with label data

timeseries data only have data for 2015 but in label data I can see records for 2016 as well.

Scheduling inference failed - missing data

I ran the notebooks in order 1 - 2 - 3 - 5.

When scheduling the inference and checking output for the first run I receive this error:

[{'ModelName': 'lookout-demo-model-v1',
  'ModelArn': 'arn:aws:lookoutequipment:eu-west-1:XYZ:model/lookout-demo-model-v1/5f05f951-xyz-xyz-xyz-a5652dfd0e81',
  'InferenceSchedulerName': 'lookout-demo-model-v1-scheduler',
  'InferenceSchedulerArn': 'arn:aws:lookoutequipment:eu-west-1:XYZ:inference-scheduler/lookout-demo-model-v1-scheduler/0c633e86-xyz-xyz-xyz-e5932d6cc3a4',
  'ScheduledStartTime': datetime.datetime(2021, 4, 26, 21, 35, tzinfo=tzlocal()),
  'DataStartTime': datetime.datetime(2021, 4, 26, 21, 30, tzinfo=tzlocal()),
  'DataEndTime': datetime.datetime(2021, 4, 26, 21, 35, tzinfo=tzlocal()),
  'DataInputConfiguration': {'S3InputConfiguration': {'Bucket': 'xyz',
    'Prefix': 'data/scheduled_inference/input/'}},
  'DataOutputConfiguration': {'S3OutputConfiguration': {'Bucket': 'xyz',
    'Prefix': 'data/scheduled_inference/output/'}},
  'Status': 'FAILED',
  'FailedReason': 'Could not find data file for sensor subsystem-06. Please ensure data is present for all sensors of the machine.'}]

I cannot see any issues with data for subsystem-06, it exists in the bucket.

Get actual prediction results

In notebook 5 of inference, for your source code

results_df = scheduler.get_predictions()
results_df.to_csv(os.path.join(INFER_DATA, 'output', 'results.csv'))
results_df

but in S3 output of each inference result, the file is 'results.jsonl', so the error is,

---

ValueError Traceback (most recent call last)
in
----> 1 results_df = scheduler.get_predictions()
2 results_df.to_csv(os.path.join(INFER_DATA, 'output', 'results.jsonl'))
3 results_df

~/SageMaker/lookout-for-equipment-demo/utils/lookout_equipment_utils.py in get_predictions(self)
1359 # Assembles them into a DataFrame:
1360 results_df = pd.concat(results_df, axis='index')
-> 1361 results_df.columns = ['Timestamp', 'Predictions']
1362 results_df['Timestamp'] = pd.to_datetime(results_df['Timestamp'])
1363 results_df = results_df.set_index('Timestamp')

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/generic.py in setattr(self, name, value)
5152 try:
5153 object.getattribute(self, name)
-> 5154 return object.setattr(self, name, value)
5155 except AttributeError:
5156 pass

pandas/_libs/properties.pyx in pandas._libs.properties.AxisProperty.set()

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/generic.py in _set_axis(self, axis, labels)
562 def _set_axis(self, axis: int, labels: Index) -> None:
563 labels = ensure_index(labels)
--> 564 self._mgr.set_axis(axis, labels)
565 self._clear_item_cache()
566

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/internals/managers.py in set_axis(self, axis, new_labels)
225 if new_len != old_len:
226 raise ValueError(
--> 227 f"Length mismatch: Expected axis has {old_len} elements, new "
228 f"values have {new_len} elements"
229 )

ValueError: Length mismatch: Expected axis has 246 elements, new values have 2 elements