lifecycle_node causing launch to hang at shutdown about launch_ros

[pbaughman]

I spoke to @wjwwood and we suspect that things are getting hung here

The lifecycle node checks that a service is available and then tries to call it apparently without a timeout. If the service goes away after the check but before the call, we can maybe hang forever.

I'm trying to reproduce my issue again with additional instrumentation to confirm this hypothesis

[pbaughman]

@wjwwood It looks like you were right. I added some print statements to LifecycleNode:

    def _call_change_state(self, request, context: launch.LaunchContext):
        while not self.__rclpy_change_state_client.wait_for_service(timeout_sec=1.0):
            if context.is_shutdown:
                self.___logger.warning(
                    "Abandoning wait for the '{}' service, due to shutdown.".format(
                        self.__rclpy_change_state_client.srv_name),
                )
                return
        import time; time.sleep(0.5)
        print("Calling the service {}".format(request))
        response = self.__rclpy_change_state_client.call(request)
        print("Finished calling service got response {}".format(response))
        if not response.success:
            self.__logger.error(
                "Failed to make transition '{}' for LifecycleNode '{}'".format(
                    ChangeState.valid_transitions[request.transition.id],
                    self.node_name,
                )
            )

And finally got this to hang:

[lifecycle_listener-2] [INFO] [listener]: notify callback: Transition from state inactive to cleaningup
[lifecycle_listener-2] [INFO] [listener]: notify callback: Transition from state cleaningup to unconfigured
Finished calling service got response lifecycle_msgs.srv.ChangeState_Response(success=True)
Calling the service lifecycle_msgs.srv.ChangeState_Request(transition=lifecycle_msgs.msg.Transition(id=5, label=''))
[lifecycle_talker-1] [INFO] [lc_talker]: on shutdown is called from state unconfigured.
[lifecycle_listener-2] [INFO] [listener]: notify callback: Transition from state unconfigured to shuttingdown
[lifecycle_listener-2] [INFO] [listener]: notify callback: Transition from state shuttingdown to finalized
ok

----------------------------------------------------------------------
Ran 1 test in 4.858s

OK
[INFO] [lifecycle_listener-2]: sending signal 'SIGINT' to process[lifecycle_listener-2]
[INFO] [lifecycle_talker-1]: sending signal 'SIGINT' to process[lifecycle_talker-1]
[lifecycle_listener-2] [INFO] [rclcpp]: signal_handler(signal_value=2)
[lifecycle_talker-1] [INFO] [rclcpp]: signal_handler(signal_value=2)
[INFO] [lifecycle_listener-2]: process has finished cleanly [pid 13315]
[INFO] [lifecycle_talker-1]: process has finished cleanly [pid 13314]

As you can see, the final "Calling the service" does not have a matching "Finished calling service" line.