[FEATURE] Add Module/Example for Step Function to AWS Batch Workloads about autonomous-driving-data-framework

[srinivasreddych]

Hello @a13zen Can you provide more context around the use-case for EventBridge here in the ask? Are you using EventPattern or CronBased schedule? If CronBased schedule, can you provide some additional understanding on how it fits the AV/ADAS workload?

[a13zen]

Sure. So, workflows run at difference cadence. You might ingest data, extract the signals/images and run object detection at one cadence lets call it Pipeline1. You might want to run a job that gathers all outputs from Pipeline1 and generate new outputs for instance synchronizing the sensor timings, augmenting the signal data with external data (imagine you have GPS points from the car, and you look up the road type from a third party provider) etc. but once a day only, let's call it Pipeline2.

You might want Pipeline1 to run as a reaction to events since the workloads would be spiky and react to new files being ingested.

You might want Pipeline2 to run as time based job to make use of Autoscaling schedules or Spot instances effectively.