Config validation exception with Shelly devices (possibly due to them being unavailable) about core

[thecode]

Hi @SeLLeRoNe , I fear you have to disable all automation involving a Shelly and then enable them 1 by 1 until you reproduce the issue. Then post the automation code.

No need to, this is a bug in the integration introduced with adding runtime_data, it is not related to user configuration. I will try to make a bugfix for the next release.

[SeLLeRoNe]

All automations are based on "click" events, I can give any of those as they are all the same, switches are all in decoupled mode.

  - platform: device
    device_id: 6052452aa4a45ec9SOME-RANDOMd254325b63fd8286544
    domain: shelly
    type: single_push
    subtype: button2
    id: button_click

Those automations have been there for more than a year and they were working, only difference I can think of is the device not being available, and the beta update, no other changes done recently.

[thecode]

If you get the same error log please just put a 👍 on the first post, if it is another log message please create a new issue. Thanks

[Anto79-ops]

thanks @thecode i tried to connect to it via the webui but could not, so I unpluged and plugged the device and now it working. Sorry, I think the device was just in a bad mood and probably nothing to do with HA. Did you still want me to submit a new issue?

[Anto79-ops]

@thecode submitted. Thank you :)

[bieniu]

single and long are for Gen1 devices and single_push, double_push, triple_push, long_push for Gen2+ devices.

[bieniu]

Automation is unavailable Triggers: required key not provided @ data['domain']

You probably didn't change the trigger type from device to state.

I guess you tried creating a new automation from the device as starting point.

No, I've been using device triggers for Shelly in my dev system for ages. We probably have a race somewhere and the config entry is loaded, but entry.runtime_data is not available yet. I don't know how it's possible and how to simulate such situation in tests.

Please unzip shelly folder from this archive, copy it to /config/custom_components, add this to your configuration.yaml file and restart HA:

logger:
  default: error
  logs:
    custom_components.shelly: debug
    aioshelly: debug

shelly_runtime_data.zip

EDIT:
And of course, please attach the post-test log file here.

[mhoogenbosch]

home-assistant.zip

With this custom component, I also can confirm it is working. Don't know if you still need a full debug log, but I attached it.

[mhoogenbosch]

Opened issues for SSH and Sun2. For Nordpool there already was a bug opened.

Now for this issue. With the fix linked I assume this will be resolved in beta8. Should we just remove the shelly component from the custom_components folder and everything keeps working?

[bieniu]

Yes, the fix should be a part of beta8 and yes, you should remove the custom version and restart HA after upgrading to beta8.

[SeLLeRoNe]

Thanks a lot!
Sorry I gave pieces because the full log also include other components, apologies

[home-assistant]

Hey there @balloob, @bieniu, @thecode, @chemelli74, @bdraco, mind taking a look at this issue as it has been labeled with an integration (shelly) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of shelly can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign shelly Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

shelly documentation
shelly source
_{^{(message by IssueLinks)}}

[chemelli74]

Hi @SeLLeRoNe , I fear you have to disable all automation involving a Shelly and then enable them 1 by 1 until you reproduce the issue.
Then post the automation code.

[Anto79-ops]

in case you need more logs, my Shelly is also offline (looks like it happened in b1 and didn't come back in b2 either). This log is from b1 and there were several of the same:

2024-05-30 02:01:45.389 ERROR (MainThread) [homeassistant.components.shelly] Unexpected error fetching shellyplus1-b8d61a87d8b8 data
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 312, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 590, in _async_update_data
    if not await self._async_device_connect_task():
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 167, in _async_device_connect_task
    await self.device.initialize()
  File "/usr/local/lib/python3.12/site-packages/aioshelly/rpc_device/device.py", line 154, in initialize
    raise RuntimeError("Already initializing")
RuntimeError: Already initializing

[mhoogenbosch]

+1 here, upgraded to b2 from 2024.5.5 and automations won't load.

Logs are pretty much similar but nevertheless:

`Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config.py", line 1541, in async_process_component_config
    await config_validator.async_validate_config(hass, config), []
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/automation/config.py", line 264, in async_validate_config
    await _try_async_validate_config_item(hass, p_config)
  File "/usr/src/homeassistant/homeassistant/components/automation/config.py", line 242, in _try_async_validate_config_item
    return await _async_validate_config_item(hass, config, False, True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/automation/config.py", line 177, in _async_validate_config_item
    automation_config[CONF_TRIGGER] = await async_validate_trigger_config(
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/trigger.py", line 245, in async_validate_trigger_config
    conf = await platform.async_validate_trigger_config(hass, conf)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/device_automation/trigger.py", line 50, in async_validate_trigger_config
    return await async_validate_device_automation_config(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/device_automation/helpers.py", line 119, in async_validate_device_automation_config
    await getattr(platform, DYNAMIC_VALIDATOR[automation_type])(hass, config),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/device_trigger.py", line 94, in async_validate_trigger_config
    block_coordinator = get_block_coordinator_by_device_id(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 741, in get_block_coordinator_by_device_id
    and isinstance(entry.runtime_data, ShellyEntryData)
                   ^^^^^^^^^^^^^^^^^^
AttributeError: 'ConfigEntry' object has no attribute 'runtime_data'`

[thecode]

@Anto79-ops please create a new issue, this is a different failure case and will not be tracked when the first issue is fixed. Thanks

[thecode]

thanks @thecode i tried to connect to it via the webui but could not, so I unpluged and plugged the device and now it working. Sorry, I think the device was just in a bad mood and probably nothing to do with HA. Did you still want me to submit a new issue?

Would be nice if you do. While the reason for your problem might be due to the device disconnecting and reconnecting many times it actually showed we have a race condition we need to protect against (which I assume if it was fixed you would may still see the device going offline if it has a problem but it will partly work)

[mhoogenbosch]

I've upgraded to beta3, but same issue. Non of my shelly devices are offline. Don't know if this is still identified as a possible cause.

[SeLLeRoNe]

No that's probably no longer a possible cause :)

[mhoogenbosch]

Any progress on this? Can't upgrade to beta now. B4 and B5 have the same error. Anything I can do myself, besides removing all shelly's ;-)

[bieniu]

I spent some time yesterday and today trying to reproduce this error with no success so far.
As a temporary workaround, you can migrate your triggers to use the event entity state:
From:

trigger:
  - platform: device
    device_id: cf2cf1d75e5affd98e733e5875baefca
    domain: shelly
    type: single_push
    subtype: button1

To:

trigger:
  - platform: state
    entity_id: event.your_shelly_device_input_0
    attribute: event_type
    to: single_push

[mhoogenbosch]

I spent some time yesterday and today trying to reproduce this error with no success so far. As a temporary workaround, you can migrate your triggers to use the event entity state: From:

trigger:
  - platform: device
    device_id: cf2cf1d75e5affd98e733e5875baefca
    domain: shelly
    type: single_push
    subtype: button1

To:

trigger:
  - platform: state
    entity_id: event.your_shelly_device_input_0
    attribute: event_type
    to: single_push

Thanks, although a lot of work, if this works. It's fine.

My triggers are slightly different:

  - platform: device
    device_id: 281a16122561102e25c4e481efb6e08a
    domain: shelly
    type: single
    subtype: button

So the 'type' doesn't match the event type. Could this be something?

[SeLLeRoNe]

Removing the domains makes so it returns an error it
Automation is unavailable Triggers: required key not provided @ data['domain']

I guess I would also need to remove the subtype but it would make the automation unusable anyway
I'll wait ^^

Thanks

[mhoogenbosch]

And you say you can't reproduce it. I guess you tried creating a new automation from the device as starting point. So that's not it. I have also a blueprint for my Shelly click events. Could this be a possible cause?

[mhoogenbosch]

Unfortunately I'm not able to test during the day. Will do later this evening if still needed by then.

[SeLLeRoNe]

I'll post here all errors I can find:

2024-06-04 10:00:01.116 ERROR (MainThread) [homeassistant.components.shelly] Unexpected error fetching Kitchen Blinds 2 data
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 315, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 599, in _async_update_data
    if not await self._async_device_connect_task():
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 161, in _async_device_connect_task
    await self.device.initialize()
  File "/usr/local/lib/python3.12/site-packages/aioshelly/rpc_device/device.py", line 154, in initialize
    self._shelly = await get_info(
                   ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aioshelly/common.py", line 74, in get_info
    async with aiohttp_session.get(
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 1197, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 428, in _request
    raise RuntimeError("Session is closed")
RuntimeError: Session is closed

This might actually be more related:

2024-06-04 10:00:00.808 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aioshelly/rpc_device/device.py", line 334, in call_rpc
    return await self._wsrpc.call(method, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aioshelly/rpc_device/wsrpc.py", line 421, in call
    resp = await self._rpc_call(method, params, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aioshelly/rpc_device/wsrpc.py", line 454, in _rpc_call
    resp = await future
           ^^^^^^^^^^^^
aioshelly.exceptions.DeviceConnectionError: <aioshelly.rpc_device.wsrpc.RPCCall object at 0x7f0aa712f040>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 717, in _handle_ha_stop
    await self.shutdown()
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 707, in shutdown
    await async_stop_scanner(self.device)
  File "/usr/local/lib/python3.12/site-packages/aioshelly/ble/__init__.py", line 34, in async_stop_scanner
    script_name_to_id = await _async_get_scripts_by_name(device)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aioshelly/ble/__init__.py", line 28, in _async_get_scripts_by_name
    scripts = await device.script_list()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aioshelly/rpc_device/device.py", line 245, in script_list
    data = await self.call_rpc("Script.List")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aioshelly/rpc_device/device.py", line 340, in call_rpc
    raise DeviceConnectionError from err
aioshelly.exceptions.DeviceConnectionError

[SeLLeRoNe]

Not sure if this is expected, but with this custom build my automatons are working without any change and on latest b6 release.

[bieniu]

Not sure if this is expected, but with this custom build my automatons are working without any change and on latest b6 release.

Yes, this is expected. Please attach the entire log, not just small fragments.

[bieniu]

@mhoogenbosch

The sun2 custom component not behaving well

2024-06-04 17:40:35.901 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open inside the event loop by custom integration 'sun2' at custom_components/sun2/helpers.py, line 369: return getattr(loc, event.split("_")[1])(date_or_dttm) (offender: /usr/local/lib/python3.12/site-packages/pytz/__init__.py, line 108: return open(filename, 'rb')), please create a bug report at https://github.com/pnbruckner/ha-sun2/issues

and ssh as well

2024-06-04 17:41:24.576 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open inside the event loop by custom integration 'ssh' at custom_components/ssh/__init__.py, line 123: manager = SSHManager( (offender: /usr/local/lib/python3.12/site-packages/paramiko/hostkeys.py, line 89: with open(filename, "r") as f:), please create a bug report at https://github.com/zhbjsh/homeassistant-ssh/issues

and nordpool

2024-06-04 17:41:11.803 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open inside the event loop by custom integration 'nordpool' at custom_components/nordpool/aio_price.py, line 110: zone = tz.gettz(zone) (offender: /usr/local/lib/python3.12/site-packages/dateutil/tz/tz.py, line 464: fileobj = open(fileobj, 'rb')), please create a bug report at https://github.com/custom-components/nordpool/issues

[bieniu]

@SeLLeRoNe No worries