K8s run monitor incorrectly flagging in process asset job as failed about dagster

[dpeng817]

Hey there - I think we generally consider running jobs inside other jobs like this bad practice. What are you trying to achieve with this pattern exactly?

[whytem]

Hi @dpeng817, the job is triggered by an external event (a data load) and utilises the parameters passed to that job to dynamically determine the selection of assets required to materalize and the inner job to complete the materialization.

Is there a way to have a job materalize a dynamic subset of assets based on an input parameter within a single job?

Thanks!

[whytem]

I moved this logic from a job to a sensor which addresses the above issue. However, that eliminates the possibility to externally trigger the execution which was possible with the job (through GraphQL). I instead created a microservice which provides a long-polling endpoint for Dagster sensors to access a queue populated by the incoming event sources. This feels like a hacky solution.

What is considering the best practice for triggering asset materializations from external events such as a webhook call?