richibrics / ioturing Goto Github PK

Hi,

Is it possible to add disk usage data for a external drive?

I think the current flow is not really efficient, and not easily maintainable.

The problems:

• You have to be careful which parts of an init goes to Initialize() and PostInitialize(). it's easy to miss it accidentally, and init flow differ between entities, but it could be similar. Sometimes RegisterEntitySensor() is in Initialize() sometimes it's in PostInitialize(). Chaotic.
• It's not possible to create longer dependency chains. What if Entity A depends on B, but B depends on C. Current flow doesn't support this.
• The only entities others depends on (de and os) are actually just providing static information, which usually shouldn't change at all. MyApp.App also provides static data, but it's not an Entity but a separate submodule.
• On the config workflow it's annoying that it tells you that you need another dependency, but it doesn't add it for you automatically.

Possible solutions:

A:
Remove the whole dependency system. Move all the base methods to submodules, just like MyApp. Convert os and de entities similar to AppInfo. If in the future some part of an Entity needed in another one, move that methods to a submodule, or create the new entity as a subclass.

Maybe this new module could be merged with MyApp, to create a single source for all static info. I recommend this solution.

This solution could also help with the config flow, as elements not available could be hidden from the list.

B:
Build correct dependency chain in EntityManager. Evaluate which entity depends on which and initialize them in the correct order.

What do you think @richibrics ? What is your plan with the dependency system?

While setting up IoTuring on a new system (debian bookworm), i cannot add the entity 'UpTime'.
It seems ignored.

Manually adding 'UpTime' to configurations.json works:

    "active_entities": [
        {
            "type": "AppInfo"
        },
        {
            "type": "Uptime"
        },
        {
            "type": "BootTime"
        }
    ],

I don't need the short send-interval of 10s and found out that i can change this in /IoTuring/Warehouse/Warehouse.py

Maybe you can implement an option in the configuration menu to change the value.
I think it is easier within configuration to setup than editing files.

While testing is see that "cpu_freq_current" has often enormous decimal places...
Examples:

Entity.Cpu.cpu_freq_current: 2214.90675
Entity.Cpu.cpu_freq_current: 1447.7362500000002
Entity.Cpu.cpu_freq_current: 1234.55375

System is Proxmox (Debian) on Celeron J4125

How can I set this up with Home Assistant? (I have it as a docker container or RPI)

Goal: Control Shutdown/Hibernation of Windows 10 PC.

Hi. I installed IoTuring on windows 10, connected it to home assistant. My goal is to put computers to sleep mode using voice assistant or when leaving home. But when I press "sleep" button computer saves memory to disk and shuts down completely. Can this be fixed ?

After installing 2024.1.1, PIP installed Paho 2.0 which includes breaking changes, which you can see here: https://eclipse.dev/paho/files/paho.mqtt.python/html/migrations.html

[ 2024-02-19 00:15:37 |   Info   |              App              ]  Name: IoTuring
[ 2024-02-19 00:15:37 |   Info   |              App              ]  Version: 2024.1.1
[ 2024-02-19 00:15:37 |   Info   |              App              ]  Description: Simple and powerful cross-platform script to control your pc and share statistics
                                                                    using communication protocols like MQTT and home control hubs like HomeAssistant.
[ 2024-02-19 00:15:37 |   Info   |          Configurator         ]  Run the script with -c to enter configuration mode
[ 2024-02-19 00:15:37 |   Info   |         Entity.AppInfo        ]  Initialization successfully completed
[ 2024-02-19 00:15:37 |   Info   |         Entity.Battery        ]  Initialization successfully completed
[ 2024-02-19 00:15:37 |   Info   |           Entity.Cpu          ]  Initialization successfully completed
[ 2024-02-19 00:15:37 |   Info   |           Entity.Ram          ]  Initialization successfully completed
[ 2024-02-19 00:15:37 |   Info   |          Entity.Power         ]  Initialization successfully completed
[ 2024-02-19 00:15:37 |   Info   |              MQTT             ]  Preparing MQTT client
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Program Files\Python312\Lib\site-packages\IoTuring\__main__.py", line 6, in <module>
    loop()
  File "C:\Program Files\Python312\Lib\site-packages\IoTuring\__init__.py", line 93, in loop
    warehouse.Start()
  File "C:\Program Files\Python312\Lib\site-packages\IoTuring\ClassManager\..\Warehouse\Deployments\HomeAssistantWarehouse\HomeAssistantWarehouse.py", line 338, in Start
    self.client = MQTTClient(self.GetFromConfigurations(CONFIG_KEY_ADDRESS),
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\site-packages\IoTuring\Protocols\MQTTClient\MQTTClient.py", line 38, in __init__
    self.SetupClient()
  File "C:\Program Files\Python312\Lib\site-packages\IoTuring\Protocols\MQTTClient\MQTTClient.py", line 45, in SetupClient
    self.client = MqttClient.Client(self.name)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\site-packages\paho\mqtt\client.py", line 766, in __init__
    raise ValueError(
ValueError: Unsupported callback API version: version 2.0 added a callback_api_version, see migrations.md for details

How to reproduce ?
Use a wrong MQTT broker address (HomeAssistant warehouse)

Using htop I noticed the script was using 98% of CPU. After a reboot it was the same.
Correct CPU usage when I typed the correct MQTT broker address.

I think we should track missing sensors and commands from PyMonitorMQTT, so we can check when we reached feature parity.

Commands:

• BrightnessCommand
• InboxCommand
• LockCommand
• NotifyCommand #11
• RebootCommand
• ShutdownCommand
• SleepCommand
• TerminalCommand #66
• TurnOffMonitorsCommand
• TurnOnMonitorsCommand

Sensors:

• ActiveWindowSensor
• BatterySensor
• BoottimeSensor
• CpuSensor
• CpuTemperaturesSensor (Linux+Mac #12 #26)
• DesktopEnvironmentSensor
• DiskSensor
• FileReadSensor as TerminalSensor #66
• HostnameSensor
• MessageSensor as TerminalSensor #66, just use echo message as command
• NetworkSensor #89
• OsSensor
• RamSensor
• ScreenshotSensor
• StateSensor
• TimeSensor
• UptimeSensor
• UsernameSensor
• VolumeSensor (Mac #47 Linux #65 )

Additional features:

• Custom entities

I run PyMonitorMQTT on multiple computers and I want to upgrade to IoTuring, but I need some of the commands and some missing sensors.

Feel free to edit this post, cross the ones you don't plan to implement

I wouldn't place them in configurations folders, so I still need to find the correct place.

Logs are currently saved in script folder and this isn't a great way to store them.

I think there are too much sensor for RAM and CPU sensors; it would be better to have most of them as extra attributes.

Hello

On a PC running Linux Mint 21, when trying to start the main.py script after setting the connection variables I get :

With HA Warehouse :

1 - Manage entities
2 - Manage warehouses
C - Start IoTuring
Q - Quit

Select your choice: C

[ 2022-11-22 19:54:57 |   Info   |              App              ]  Name: IoTuring
[ 2022-11-22 19:54:57 |   Info   |              App              ]  Version: 1.0.0
[ 2022-11-22 19:54:57 |   Info   |              App              ]  Description: Simple and powerful script to control your pc and share stats to HomeAssistant.
[ 2022-11-22 19:54:57 |   Info   |          Configurator         ]  Run the script with -c to enter configuration mode
[ 2022-11-22 19:54:57 |   Info   |         Entity.AppInfo        ]  Initialization successfully completed
[ 2022-11-22 19:54:57 |   Info   |         Entity.AppInfo        ]  Post-initialization successfully completed
[ 2022-11-22 19:54:57 |   Info   |              MQTT             ]  Preparing MQTT client
[ 2022-11-22 19:54:57 |   Info   |              MQTT             ]  MQTT Client ready to connect to the broker
Traceback (most recent call last):
  File "/home/user/Temp/IoTuring/main.py", line 43, in <module>
    warehouse.Start()
  File "/home/user/Temp/IoTuring/ClassManager/../Warehouse/Deployments/HomeAssistantWarehouse/HomeAssistantWarehouse.py", line 43, in Start
    self.client.AsyncConnect()
  File "/home/user/Temp/IoTuring/Protocols/MQTTClient/MQTTClient.py", line 59, in AsyncConnect
    self.client.connect_async(self.address, port=self.port)
  File "/home/user/.local/lib/python3.10/site-packages/paho/mqtt/client.py", line 979, in connect_async
    if port <= 0:
TypeError: '<=' not supported between instances of 'str' and 'int'_

With MQTT Warehouse :

1 - Manage entities
2 - Manage warehouses
C - Start IoTuring
Q - Quit

Select your choice: C

[ 2022-11-22 19:57:13 |   Info   |              App              ]  Name: IoTuring
[ 2022-11-22 19:57:13 |   Info   |              App              ]  Version: 1.0.0
[ 2022-11-22 19:57:13 |   Info   |              App              ]  Description: Simple and powerful script to control your pc and share stats to HomeAssistant.
[ 2022-11-22 19:57:13 |   Info   |          Configurator         ]  Run the script with -c to enter configuration mode
[ 2022-11-22 19:57:13 |   Info   |         Entity.AppInfo        ]  Initialization successfully completed
[ 2022-11-22 19:57:13 |   Info   |         Entity.AppInfo        ]  Post-initialization successfully completed
[ 2022-11-22 19:57:13 |   Info   |              MQTT             ]  Preparing MQTT client
[ 2022-11-22 19:57:13 |   Info   |              MQTT             ]  MQTT Client ready to connect to the broker
Traceback (most recent call last):
  File "/home/user/Temp/IoTuring/main.py", line 43, in <module>
    warehouse.Start()
  File "/home/user/Temp/IoTuring/ClassManager/../Warehouse/Deployments/MQTTWarehouse/MQTTWarehouse.py", line 30, in Start
    self.client.AsyncConnect()
  File "/home/user/Temp/IoTuring/Protocols/MQTTClient/MQTTClient.py", line 59, in AsyncConnect
    self.client.connect_async(self.address, port=self.port)
  File "/home/user/.local/lib/python3.10/site-packages/paho/mqtt/client.py", line 979, in connect_async
    if port <= 0:
TypeError: '<=' not supported between instances of 'str' and 'int'_

From what I understand, their seems to be a type mismatch for the port number (defined as string but expects an int in client.py )

The connection variables seem properly defined though (ie for HA warehouse) :

❯ cat configurations.json                                                                   
{"active_entities": [{"type": "AppInfo"}], "active_warehouses": [{"address": "192.168.0.XX", "port": "1883", "name": "blahblah", "username": "mqtt-pcname", "password": "somepassword", "add_name": "N", "type": "HomeAssistant"}]}% 

Any idea how to solve this ?

Activated Power Entity without Os Entity, then started the script and Power Entity returned a PostInitialize error because self.os was None

It' not possible to exit the program from terminal, Ctrl+C does nothing, closing the window doesn't kills the process.

It has to be killed from Task Manager. It's hard to find which process is this, as it's just called Python

Hi,
installed 2022.12.2 on an RaspberryPi (RaspberryPi OS) and used the HA-Warehouse.

Looking very good, except for the cpu temperature (cpu_thermal).
HA shows 3 decimal places like...
39,166 °C

I think one decimal place should be enough.
Examples:

39,166 °C --> 39,2 °C
41,736 °C --> 41,7 °C

It looks and feels better in the HA UI, especially if you are using charts and long term statistics.

Some enhancement ideas about AppInfo entity

• Make configurable that AppInfo Name and Appinfo Version show up as one sensor
• For update checking, if IoTuring is installed via git, compare git commit hashes instead of version from pypi.

Idea from #94

Windows command:

%windir%\System32\rundll32.exe powrprof.dll,SetSuspendState Hibernate

Source: https://superuser.com/questions/463646/is-there-a-command-line-tool-to-put-windows-8-to-sleep/463652#463652

Linux command:

systemctl hibernate

Hi,
after upgrading from v2023.2.1 to v2023.2.2 i see high "cpu_used_percentage" spikes in the mqtt data.

I can see this on my 2 systems:
RapberryPi 3B+:

~# uname -r
5.15.84-v8+
~# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian 

Mini-PC (Celeron J4125) with Proxmox:

~# uname -r
6.0.15-edge
~# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian

When i check the systems locally with htop, i can see that IoTuring sometimes consumes up to 100% cpu.

"cpu_used_percentage" mqtt-data from my RaspberryPi:
section A: v2023.2.2
section B: v2023.2.1

All logs in Logs folder contain only this message

[ 2023-02-17 10:38:53 | Message  |         ConfiguratorIO        ]  Loaded "/Users/riccardobriccola/Library/Application Support/IoTuring/configurations.json" 

Please choose a license for this nice project: https://choosealicense.com/

No license means all rights reserved, and noone can contribute legally: https://choosealicense.com/no-permission/

I recommend MIT for a project like this.

Since in #13 we are moving the script in site-package folder, the configuration file isn't easy to access.
The idea is to move it in the home folder of the user, something like "/home/USER/.ioturing/configuration.json" for linux, in Library for macOS and in AppData (I think) for Windows.

Next step will be to convert the json file to a yaml file (with an auto convert method for older version users).

I don't create a branch now since I want to wait for #13 to be merged.

I found your nice Project und used it to push the system data of an proxmox-server into an Homeassistant.
Works without problems since on week.

The only sensor i miss is cpu-temperature.
Is it possible to implement such sensor?

Entities not supported (by OS) should not be available in configurator menu.

This will be done after the addition of new OS get method (#30)

All cpu_frequency sensors have the unit "Hz".
But modern pc's and sbc's working mostly in "MHz" range.

HomeAssistant topics are not sanitized for space characters. Home Assistant doesn't allow topics with spaces.

To reproduce:

1. Create a multi instance entity (e.g. Notify)
2. At the Tag prompt add a space somewhere (e.g.: Notify message)
3. Home Assistant won't discover this entity, and this appears in the log:
   Received message on illegal discovery topic 'homeassistant/button/IoTuring/gydesktop_Entity_Notify_Notify message_notify/config'. The topic contains not allowed characters. For more information see https://www.home-assistant.io/docs/mqtt/discovery/#discovery-topic

In order to show the icons that we can see in readme file through GitHub, I think we need to use absolute urls for the images.
The problem occurs only in pypi IoTuring homepage

Unload entity if errors in Initialize/PostInitialize is missing.

Hi @infeeeee
I hope you can give me an advice on this question:
I found a simple macOS C++ program (smctemp) that returns macOS cpu temperature.
The idea is to compile a x86 and an ARM version that will be included in the IoTuring package.

That project has a GPLv2 license, which prevents me from using that executable in my script.
Do I have any solution ?
What do you think if I downgrade IoTuring license to GPLv2 ?

Hope to hear good news,
Riccardo

I've set up IoTuring on Windows 11. Everything is running smoothly - I see the MQTT device in Home Assistant but I do not see the configurations.json file that's supposed to be found on my local machine in C:\Users\username\AppData\Roaming\IoTuring. The IoTuring folder does not even exist in the Roaming folder. Any advice? Thanks.

IoTuring version is 2023.11.1.

Copy of command prompt output after setting up IoTuring:
[ 2023-12-02 00:23:27 | Message | ConfiguratorIO ] Saved "C:\Users\username\AppData\Roaming\IoTuring\configurations.json"

I found this example to install os-specific dependencies (specifying it in pyproject.toml) which seems better that having user to specify the os in the pip install command.

Source

[project]
dependencies = [
  'cached-property >= 1.2.0, < 2',
  'distro >= 1.5.0, < 2',
  'docker[ssh] >= 4.2.2, < 5',
  'dockerpty >= 0.4.1, < 1',

  # Conditional
  'backports.shutil_get_terminal_size == 1.0.0; python_version < "3.3"',
  'backports.ssl_match_hostname >= 3.5, < 4; python_version < "3.5"',
  'colorama >= 0.4, < 1; sys_platform == "win32"',
  'enum34 >= 1.0.4, < 2; python_version < "3.4"',
  'ipaddress >= 1.0.16, < 2; python_version < "3.3"',
  'subprocess32 >= 3.5.4, < 4; python_version < "3.2"',
]

As you can see, there's a sys_platform condition. What do you think @infeeeee ?

Hi!
I've installed it on Windows 11 via Python3.11.2
There are 2 problems: if I press A inside "Manage Entities" it returns me:
[ 2023-02-26 00:08:46 | Error | Configurator ] Error in Entity select menu: 'NoneType' object has no attribute 'AllowMultiInstance'
Btw I "fixed" this by manually adding the Entities inside configurations.json.

The main problem is: when I start IoTuring, it returns:
[ 2023-02-26 00:11:04 | Message | ConfiguratorIO ] Saved "C:\Users\Giuseppe\AppData\Roaming\IoTuring\configurations.json" [ 2023-02-26 00:11:04 | Info | App ] Name: IoTuring [ 2023-02-26 00:11:04 | Info | App ] Version: 2023.2.5 [ 2023-02-26 00:11:04 | Info | App ] Description: Simple and powerful cross-platform script to control your pc and share statistics using communication protocols like MQTT and home control hubs like HomeAssistant. [ 2023-02-26 00:11:04 | Info | Configurator ] Run the script with -c to enter configuration mode [ 2023-02-26 00:11:04 | Info | Entity.Cpu ] Initialization successfully completed [ 2023-02-26 00:11:04 | Info | Entity.Disk ] Initialization successfully completed [ 2023-02-26 00:11:04 | Info | Entity.Power ] Initialization successfully completed [ 2023-02-26 00:11:04 | Info | Entity.Ram ] Initialization successfully completed [ 2023-02-26 00:11:04 | Info | Entity.Temperature ] Initialization successfully completed [ 2023-02-26 00:11:04 | Info | Entity.Volume ] Initialization successfully completed [ 2023-02-26 00:11:04 | Info | Entity.Cpu ] Post-initialization successfully completed [ 2023-02-26 00:11:04 | Error | Entity.Disk ] Post-initialization interrupted due to an error: [ 2023-02-26 00:11:04 | Error | Entity.Disk ] Some dependencies were not correctly loaded: check if there have been errors in their log and, if not already present in the configuration, try to activate them. [ 2023-02-26 00:11:04 | Error | Entity.Disk ] Missing dependencies: Os [ 2023-02-26 00:11:04 | Info | EntityManager ] Entity.Disk unloaded [ 2023-02-26 00:11:04 | Error | Entity.Power ] Post-initialization interrupted due to an error: [ 2023-02-26 00:11:04 | Error | Entity.Power ] Some dependencies were not correctly loaded: check if there have been errors in their log and, if not already present in the configuration, try to activate them. [ 2023-02-26 00:11:04 | Error | Entity.Power ] Missing dependencies: Os [ 2023-02-26 00:11:04 | Info | EntityManager ] Entity.Power unloaded [ 2023-02-26 00:11:04 | Info | Entity.Ram ] Post-initialization successfully completed [ 2023-02-26 00:11:04 | Error | Entity.Temperature ] Post-initialization interrupted due to an error: [ 2023-02-26 00:11:04 | Error | Entity.Temperature ] Some dependencies were not correctly loaded: check if there have been errors in their log and, if not already present in the configuration, try to activate them. [ 2023-02-26 00:11:04 | Error | Entity.Temperature ] Missing dependencies: Os [ 2023-02-26 00:11:04 | Info | EntityManager ] Entity.Temperature unloaded [ 2023-02-26 00:11:04 | Error | Entity.Volume ] Post-initialization interrupted due to an error: [ 2023-02-26 00:11:04 | Error | Entity.Volume ] Some dependencies were not correctly loaded: check if there have been errors in their log and, if not already present in the configuration, try to activate them. [ 2023-02-26 00:11:04 | Error | Entity.Volume ] Missing dependencies: Os [ 2023-02-26 00:11:04 | Info | EntityManager ] Entity.Volume unloaded [ 2023-02-26 00:11:04 | Info | MQTT ] Preparing MQTT client [ 2023-02-26 00:11:04 | Info | MQTT ] MQTT Client ready to connect to the broker [ 2023-02-26 00:11:05 | Error | MQTT ] Connection error [ 2023-02-26 00:11:05 | Error | MQTT ] Connection lost

Am I dumb or what?
Thanks in advance :)

Hi @infeeeee,
I've always asked myself how much this script is really used out of my home.
I tried looking at pypistats but all the statistics are biased by the pipy repository cloners.

I was thinking if there was a way to make a log of the machines it's installed on, nothing that should worry the user of the app about his privacy obviously.
Something like a log with the OS and the script version that is sent, if the user agrees, on script launch.

I searched on internet and found something like this: https://www.linkedin.com/pulse/track-usage-your-python-scripts-using-gas-measurement-steven-b-peutz

I know that knowing the usage won't make any change for the development but it was a curiosity.
Maybe it would allow to develop more for the main OS it's used on for example.

What do you think ? Is it applicable ? Would this bother the user if he's asked to share analytics data while he is configuring the script ?