christian1980nrw / spotmarket-switcher Goto Github PK

Hi Christian,
Can you help me make more sense of the individual settings in config.txt?

• economic_check
  • If this is set to 0, when will the battery charge?
  • Can you give an example when 1 an 2 should be chosen?
• disable_inverting_while_only_switching
  • Can you explain this? What does "if switching is turned on" mean?
• limit_inverter_power_after_enabling
  • I'm also lost with this setting. Can you explain what it means?
• start_price
  • How does this relate to the settings above. Does it overrule it at any times and if for example set to -5, charging will never start if the price is not below -5?
• use_start_stop_logic and switchablesockets_at_start_stop and charge_at_solar_breakeven_logic and switchablesockets_at_solar_breakeven_logic
  • How do these settings relate to the settings above?
• config_matrix_target_soc_weather
  • Can you provide a good resource to calculate how many megajoule are needed for a full day of power?

Also, regarding the Victron settings, I feel we should add soem more information to the documentation. My guess in Winter most users would be happy to simply fully charge their bateries full at any given time of the day. So the scheduler needs to be set to 0:00 - 23:59 and SOC Limit off, right?
Generally, I believe this would be the right setting now, since SOC limit and best charging time is take care of by the matrix now, right?

Thanks
Daniel

@ckvsoft Maybe we have overseen a bigger issue. I noticed that the economy check is done correctly and charging is cancelled but i also noticed that the switchable sockets will not be turned on anymore. They should not be affected by the economy check because there are no charging losses. The lowest price today will be at 23:00, i will verify it again.

Hallo zusammen,

ich habe vor einigen Tagen die neue Firmware Venus OS 3.1 bzw. 3.11 aufgespielt. Das dynamische ESS verhält sich bei mir leider nicht so intelligent wie ich es gerne hätte. Mit aktivem dynamischen ESS wird unnötig der Akku entladen oder teilweise der Akku in der Nacht voll gehalten, trotz hohen Stromkosten (Tibber). Die Logik vom dynamischen ESS scheint aktuell noch nicht für mich zu passen. Daher würde ich gerne wieder die Funktionen vom Spotmarketswitcher verwenden.
Wenn ich die Einstellung von dynamic ESS wieder auf "off" stelle, erfolgt leider die Ladeplanung über den Spotmarketswitcher nicht mehr. Hat jemand hierfür eine Problemlösung? In den letzten Monaten hat der Spotmarketswitcher super funktioniert. :-)

Danke für eure Hilfe.

Viele Grüße
Alex

From what I have understood, in its current implementation all activities (charging, switches) are switched off whenever the control script starts at every hour. Also, there is a long sleep that turns things off after these were switched on.

I would like to change that behaviour to switch off only when the following hour is not good.

If there is no charging, only switching, ESS should be (optional) turned off that hour so that the battery will not be used.

Ich verwende jetzt ein paar Tage die Start Stop logic.
Aber ich kann nicht feststellen was diese eigentlich genau machen sollte.

Es wird ja nur überprüft ob der startpreis grösser als der currentpreis ist. Was sollte der Stoppreis tun?

I dont know why my changes are not working.
I want to submit the correct branch to the installer.
See da2cd07

Hallo @christian1980nrw,

finde Dein Skript sehr gut! Funktioniert auch gut bei mir, meinst Du man könnte die API von https://www.solarprognose.de dort einbauen? Die ist etwas IMHO etwas granularer einstellbar auf die eigene PV-Anlage?

Viele Grüße

Hello @christian1980nrw,

I am operating 4 Victron Energy instances with your script and Tibber and it is working quite well. Now with the current version installed, I discovered when the system is operating towards a nearly full battery the ESS #7 limit inverter power reduces the power of the DC-DC system as it is not allowed to provide power to the loads in the AC side. I guess the "bug" is kind of happening because of the nature of ESS #7 limit inverter power. Furthermore the energy is not sold to the public grid in case the battery is full and the sun in shining, which is at least better than to not use the energy.

I do understand your motivation on this one and it is working great if the battery is not (nearly) full and the day is not sunny. Maybe there is a way to circumvent this issue.

Here you can see the issue with one of the systems (I was able to reproduce this on another system as well):

The battery became full charged approx 12h and 12:15 I disabled manually the ESS #7. The sun was with no clouds delivering all the time.

Output:
root@raspberrypi4:/data/etc/Spotmarket-Switcher# DEBUG=1 sh controller.sh
I: Bash Version: GNU bash, version 5.0.18(1)-release (arm-ve-linux-gnueabi)
I: Spotmarket-Switcher - Version 2.3.7
[] Loading...Config validation passed.
I: Fetching today-data data from Entsoe.
D: No delay of download of entsoe data since DEBUG variable set.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 941 100 941 0 0 1297 0 --:--:-- --:--:-- --:--:-- 1296
D: No delay of download of entsoe data since DEBUG variable set.
D: No delay of download of entsoe data since DEBUG variable set.
cp: can't stat '/tmp/entsoe_today_prices_sorted.txt': No such file or directory
root@raspberrypi4:/data/etc/Spotmarket-Switcher#

XML-Content of entsoe_today_prices.xml (not our fault) in case of API Error:

<Acknowledgement_MarketDocument
xmlns="urn:iec62325.351:tc57wg16:451-1:acknowledgementdocument:7:0">
3639ea49-2e45-4
2023-10-24T20:35:35Z

<sender_MarketParticipant.mRID codingScheme="A01">10X1001A1001A450</sender_MarketParticipant.mRID>
<sender_MarketParticipant.marketRole.type>A32</sender_MarketParticipant.marketRole.type>

<receiver_MarketParticipant.mRID codingScheme="A01">10X1001A1001A450</receiver_MarketParticipant.mRID>
<receiver_MarketParticipant.marketRole.type>A39</receiver_MarketParticipant.marketRole.type>


<received_MarketDocument.createdDateTime>2023-10-24T20:35:35Z</received_MarketDocument.createdDateTime>


<Reason>
	<code>999</code>
	<text>Unexpected exception, please contact support and provide exception ID.

Exception ID: 18b69966-0e7a-42f5-95a6-593a889981a4

</Acknowledgement_MarketDocument>

Wenn bei Entsoe keine Preise für Morgen gibt wird abgebrochen und nicht wie bei aWATTar ohne diese Preise weiter gemacht.
Tibber kann ich nicht sagen wie sich der verhält, da waren Preise da.

Das schlechte bei so einen Abbruch ist das alles was läuft weiter läuft.

Vielleicht kannst Du das checken. Ich bin zwar schon Munter weil ich MotoGP schau komme aber wahrscheinlich erst wieder am Nachmittag dazu mir was anzuschauen.

Ich habe sicherheitshalber einen älteren code getestet, der macht das auch. Nur um sicher zu gehen das ich da nix vermurkst habe als ich den Code umsortiert habe. (aktuelles dev)

lg Chris

Kann man die Batteriekosten Berücksichtigung irgendwie deaktivieren?

Wenn die Kosten Tagelang über den Wert bleiben wird der Akku wohl nie geladen. Die Sonne reicht nicht immer dafür aus.

Lets discuss the charging conditions in case of feeding in / exporting to the grid. @ckvsoft @alex90alex
If the prices and the total charging costs at night are lower than the export payout in the afternoon,
we should pre-charge the battery (maybe up to 100%).

Also if we are charging we should not forget the lost profit caused of the charging.
It should be part of our calculations or am i wrong?

So that we don't talk past each other, we should carry out the example calculations in a shared spreadsheet first.
https://docs.google.com/spreadsheets/d/1Xkx3vjyCw-b7HOuuWABOCdY-kFDthEFuIPjaHk99Tdo/edit?usp=sharing

Regards
Christian

lowest, second, third, fourth, fifth, sixth should be replaced by loops so that integer values from 1 to 24 can be entered.
1 is activation of the cheapest hour of the day up to 24 (the whole day) and if second day is activated, up to 48.
And if we are doing this, we should maybe use "minutes" or additional minutes 1-59 option instead of full hours only to make this more flexible. The last hour will then no longer be switched on for a whole hour, charging or switching will end a bit earlier depending the minutes.
This can be done by a pause combined with
manage_charging "off" "Turn off charging."
manage_fritz_sockets "off"
manage_shelly_sockets "off"

I think the script (and the config.txt) can be shortened in these parts.
It would also make the script more user friendly for people using heat pumps
who need longer run times and just want to avoid the most expensive hours of the day.
Charging and switching should still be independent of each other.

charge_at_lowest_price=1
charge_at_second_lowest_price=1
charge_at_third_lowest_price=0
charge_at_fourth_lowest_price=0
charge_at_fifth_lowest_price=0
charge_at_sixth_lowest_price=0

switchablesockets_at_lowest_price=1
switchablesockets_at_second_lowest_price=1
switchablesockets_at_third_lowest_price=0
switchablesockets_at_fourth_lowest_price=0
switchablesockets_at_fifth_lowest_price=0
switchablesockets_at_sixth_lowest_price=0

There is a possible bug with Entso-E API if you are using the second day data.
Data processing will fail.
Bug is under investigation, i will update the script as soon as possible.

Can you implement a logic to switch on devices via custom api (RESTfull) like the iobroker simple API. You just need to call a simple URL like this:

http://ipaddress:8087/set/javascript.0.test?value=1

So if you set some variables in the config like...

use_custom_api=1
custom_api_base_url="http://ipaddress:8087/set/javascript.0.test?value="
custom_api_on_value="1"
custom_api_off_value="0"
custom_api_username="user"
custom_api_password="pass"
custom_api_return_code="200"
....

you get the point...

EDIT: There may be more than one value that you want to change on a single host

Hi Christian,
Kannst du sagen ob es Sinn macht Batterylife auszuschalten, bei der Nutzung von Spotmarket Switcher, oder besser anlassen? Bei mir hängt der SOC bei 80% fest, da ich noch Probleme mit den richtigen Einstellungen vom Spotmarket Switcher habe.
Ich werde später auch nochmal nen Thread aufmachen bzgl config.txt Fragen :D

Beste Grüße
Daniel

Hallo,
kurze frage bei mir gings jetzt ein paar wochen und seit kurzem krieg ich immer diesen fehler:
/data/etc/Spotmarket-Switcher/controller.sh: line 468: 31: command not found

mfg

Moin,

ich glaube es liegt ein Fehler im Skript vor. Bei bestimmten Preiskonstellationen kommt als "Current price" eine NULL heraus und er startet das Laden ohne Rücksicht:
'Current price is' ' Cent/kWh netto.'
Parallel gibt es eine Fehlermeldung im Log:
line 223: printf: e15: invalid number

Viele Grüße
Nico

The script will be automatically executed at full hours. To avoid overloading the external API systems, there will be a random pause. If the script is executed at other times, there is no need for delays. Usually this will be manual testing starts.

Known bug. To be fixed at the run script.

ToDo

Hi,

just trying your script and it seems to work fine as it switches on/off the first scheduler.
All I wonder is: does it not configure the dates and time of the scheduler according to what it read by Awattar?

Here my output to explain my question a bit more in detail:
root@einstein:~/scripts# ./controller.sh Maybe we are still charging from last script runtime. Stopping scheduled charging. Battery SOC is at 40 %. 0 Awattar today-data is up to date. Awattar tomorrow-data is up to date. Please verify correct system time and timezone: Sun Feb 12 22:16:17 CET 2023 Current price is 15.19100 Cent/kWh netto. Lowest price will be 10.80200 Cent/kWh netto. The average price will be 14.9902 Cent/kWh netto. Highest price will be 22.18000 Cent/kWh netto. Second lowest price will be 11.61700 Cent/kWh netto. Third lowest price will be 11.84600 Cent/kWh netto. Fourth lowest price will be 11.88900 Cent/kWh netto. Fifth lowest price will be 12.00000 Cent/kWh netto. Sixth lowest price will be 12.00000 Cent/kWh netto. Difference between highest price and current price is greater than 23.3%. Charging makes sense. Executing 1 hour charging. Battery SOC is at 40 %. 0 root@einstein:~/scripts#

This sounds like "starting one hour now", but my scheduler is configured as follows:

So it will load for 3 hours and not now but within a about 1h40m

I don´t see how the script helps to load during low price when the schedule does not change the start-time nor the duration?

Let me extend the question about how to change start-time and duration in therms of whether conditions.
Today my remaining capacity won't be suitable to reach sunrise. So it would be great to load in the remaining time when the price would be lowest within that timeframe. To be more precise, not even sunrise is interesting, but the time to when the sunshine is expected due to weather conditions!

Entso-E API integration is currently not working as expected. Please switch to the aWATTar API (or Tibber API if you are using dev branch for testing) until fixed. Maybe the Entso-E API design changed.
Will be fixed soon with next release.

Entso-E API: sed: /tmp/entsoe_prices_sorted.txt: No such file or directory

Needs to be fixed. Instead of exit 1, charging and switching should be stopped first if there are no prices available.

We are already reading sunrise and sunset.
I: Sunrise today will be 08:10 and sunset will be 18:13. Suntime will be 603 minutes.

If the running time of the script is later as sunset, it makes no sense to stop charging because of solar weather.
There is no need to reserve battery capacity anymore for charging by sun, there will be no sun anymore.
In this case script runtime at 19:00 and later that day should not be canceled anymore.

hello,

did you check https://github.com/tfranssen/dynamic-ess ?

currently testing your script on cerbo Gx with latest VenusOS installed.
API connection for Belgium. awaiting token from provider.
possible to activate via schedule for end-user?

Saw this syntax error last night in var/log/messages
Possibly description the ( parentheses need to be escaped?

local conditions=("${!1}") local conditions=\("${!1}"\)

local descriptions=("${!2}") local descriptions=\("${!2}"\)

Can anyone confirm?

Thanks
Daniel

ToDo

Hi, tolles Skript was du da gemacht hast - ich würde es gerne für Awattar verwenden.

Ich fände auch folgendes Feature nützlich (vielleicht kann dein Skript das ja schon, und ich habe es noch nicht verstanden).

Ich habe nur ein Balkonkraftwerk mit 880 Wp und einen 7.7KWh Speicher dazu .... der wird also nie voll (an den besten Tagen liegt der Brutto-Ertrag bei 5.x KWh). Ich möchte also versuchen den Speicher mit Nachladen zu den billigen Stunden entsprechend zu füllen, dass die teureren Stunden überbrückt werden können .... dabei aber den zur erwartenden Sonnen-Ertrag nicht verschenken. Ich denke, dass das mit einer dynamischen Veränderung des Target-SoC im machbar sein müsste?

in jeder Stunde den Target-SoC so anpassen, dass der zur erwartende PV Ertrag des aktuellen bzw kommenden Tages noch reinpasst ...... und darauf dann die BesteStunde / ZweitbesteStunde / DrittbesteStunde usw Logik noch anwenden.

Hy Christian,

Great project!

I just have die following issues with the date from the solar energy csv:

Maybe we are still charging from last script runtime. Stopping scheduled charging.
Battery SOC is at 29 %.
0
Awattar today-data is up to date.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    51  100    51    0     0     80      0 --:--:-- --:--:-- --:--:--    80
/tmp/expected_solarenergy.csv downloaded
/data/etc/Spotmarket-Switcher/controller.sh: line 282: printf: e15: invalid number
/data/etc/Spotmarket-Switcher/controller.sh: line 283: printf: e15: invalid number
date: invalid date '1970-01-01 '
date: invalid date '1970-01-01 '
/data/etc/Spotmarket-Switcher/controller.sh: line 288: (( - )) / 60: syntax error:
operand expected (error token is ")) / 60")
Please verify correct system time and timezone:
Sat Jul 29 11:39:42 CEST 2023
Current price is 6.37500 Cent/kWh netto.
Lowest price will be 1.82000 Cent/kWh netto.
The average price will be 7.66754 Cent/kWh netto.
Highest price will be 10.30900 Cent/kWh netto.
Second lowest price will be 1.96700 Cent/kWh netto.
Third lowest price will be 4.04800 Cent/kWh netto.
Fourth lowest price will be 4.91800 Cent/kWh netto.
Fifth lowest price will be 5.86600 Cent/kWh netto.
Sixth lowest price will be 5.86600 Cent/kWh netto.
Sunrise today will be  and sunset will be . Suntime will be  minutes.
Solarenergy today will be  megajoule per sqaremeter with  percent clouds.
Solarenergy tomorrow will be  megajoule per squaremeter with  percent clouds.

Ich habe das Problem das mit Heute zb. der highest_price mit 9.80700 angezeigt wird.
Die Preise kommen aus aWattAr. Der tatsächliche highest_price ist aber 18.91300.

Ich konnte das Problem natürlich wieder auf AWK zurückführen. 9.80700 ist der höchste vor den komma einstellige Preis.
Lösche ich alle Preise die vor den Komma nur eine Stelle haben wird die highest_price richtig ermittelt.

Eine Lösung hätte ich mit highest_price=$(grep -E '^[0-9]+.[0-9]{5}$' "$file7" | tail -n1). grep und tail sollte ja auch im venusos installiert sein.

Ich hatte ja schon einige Probleme mit AWK Überläufen und hab deshalb bc verendet. BC gibts aber nicht am VenusOS. Nachinstallieren geht zwar aber viele wollen das nicht.

Mich nervt es aber das ich mich auf AWK nicht verlassen kann und da gibt es doch einiges.

Eine Bitte, kann mir hier jemand posten welche awk version im aktuellen VenusOS installiert ist. Ich habe die nächsten zwei Tage leider keinen Zugriff darauf. Auf meinen Test System ist AWK

mawk 1.3.4 20200120
Copyright 2008-2019,2020, Thomas E. Dickey
Copyright 1991-1996,2014, Michael D. Brennan

random-funcs: srandom/random
regex-funcs: internal
compiled limits:
sprintf buffer 8192
maximum-integer 2147483647

Ich schau mir mal die AWK aufrufe im Script an, kann aber nut die aWattAR sachen checken.

lg Chris

ToDo

To be fixed