weewx / weewx

WeeWX code repository

WeeWX

Open source software for your weather station

Description

The WeeWX weather system is written in Python and runs on Linux, MacOSX, Solaris, and *BSD. It runs exceptionally well on a Raspberry Pi. It collects data from many different types of weather stations and sensors, then generates plots, HTML pages, and monthly and yearly summary reports. It can push plots, pages, and reports to a web server, as well as upload weather-related data to many online weather services. Thousands of users worldwide!

See the WeeWX website for examples of websites generated by WeeWX, and a map of stations using WeeWX.

• Robust and hard-to-crash
• Designed with the enthusiast in mind
• Simple internal design that is easily extended (Python skills recommended)
• Large ecosystem of 3rd party extensions
• Internationalized language support
• Localized date/time support
• Support for US and metric units
• Support for multiple skins
• Support for sqlite and MySQL
• Extensive almanac information
• Uploads to your website via FTP, FTPS, or rsync
• Uploads to online weather services
• Requires Python 3.6 or greater

Support for many online weather services, including:

• The Weather Underground
• CWOP
• PWSweather
• WOW
• AWEKAS
• Windy
• Open Weathermap
• WeatherBug
• Weather Cloud
• Wetter
• Windfinder

Support for many data publishing and aggregation services, including:

• MQTT
• InfluxDB

Support for over 70 types of hardware including, but not limited to:

• Davis Vantage Pro, Pro2, Vue, Envoy;
• Oregon Scientific WMR100, WMR300, WMR9x8, and other variants;
• Oregon Scientific LW300/LW301/LW302;
• Fine Offset WH10xx, WH20xx, and WH30xx series (including Ambient, Elecsa, Maplin, Tycon, Watson, and others);
• Fine Offset WH23xx, WH4000 (including Tycon TP2700, MiSol WH2310);
• Fine Offset WH2600, HP1000 (including Ambient Observer, Aercus WeatherSleuth, XC0422);
• Fine Offset GW1000, GW1100, GW2000 (including Ecowitt)
• LaCrosse WS-23XX and WS-28XX (including TFA);
• LaCrosse GW1000U bridge;
• Hideki TE923, TE831, TE838, DV928 (including TFA, Cresta, Honeywell, and others);
• PeetBros Ultimeter;
• RainWise CC3000 and MKIII;
• AcuRite 5-in-1 via USB console or bridge;
• AcuRite Atlas;
• Argent Data Systems WS1;
• KlimaLogg Pro;
• New Mountain;
• AirMar 150WX;
• Texas Weather Instruments;
• Dyacon;
• Meteostick;
• Ventus W820;
• Si1000 radio receiver;
• Software Defined Radio (SDR);
• One-wire (including Inspeed, ADS, AAG, Hobby-Boards).

See the hardware list for a complete list of supported stations, and for pictures to help identify your hardware! The hardware comparison has specifications for many different types of hardware, including some not yet supported by WeeWX.

Install

https://weewx.com/docs/quickstarts.html

Download

For current and previous releases:

https://weewx.com/downloads

For the latest source code:

https://github.com/weewx/weewx

Documentation and Support

Guides for installation, upgrading, and customization:

https://weewx.com/docs/

The wiki includes user-contributed extensions and suggestions:

https://github.com/weewx/weewx/wiki

Community support can be found at:

https://groups.google.com/group/weewx-user

Licensing

WeeWX is licensed under the GNU Public License v3.

Copyright

© 2009-2024 Thomas Keffer, Matthew Wall, and Gary Roderick

weewx's Issues

Remove max gap fraction from bar charts?

Now that bar charts explicitly use a start and stop time for each bar, max gap fraction may no longer be needed for them.

Consider removing the option.

failures in archive_interval result in no retries

this has happened on ws23xx and fo hardware. it happens regularly with ws23xx when there is a lot of RF interference (even with shielded cables and ferrite cores in place). it could happen to any hardware with a logger.

the StdArchive service tries to get the archive interval from the hardware using the archive_interval property. when archive_interval fails while querying the hardware, weewx dies.

so far the workaround is to disable any logging capability - both catchup and record_generation=hardware.

desired behavior would be for the engine to reload the driver and start over (as it does when other hardware communication failures happen)

vantage driver crash

weewx crashed yesterday with an input/output error. i thought the designed behavior was to retry a few times then reload the driver.

weewx: 3.1.0
driver: vantage
weather station: vantage pro 2 model 6152C w/weatherlink data logger
os: Ubuntu 12.04.5 LTS
uname: Linux sailing 3.2.0-69-generic #103-Ubuntu SMP Tue Sep 2 05:02:14 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
uptime: 22:58:15 up 203 days, 4:59, 1 user, load average: 1.95, 1.73, 1.56

stack trace:

May 14 16:40:26 sailing weewx[30353]: engine: Caught unrecoverable exception in engine:
May 14 16:40:26 sailing weewx[30353]: **** (5, 'Input/output error')
May 14 16:40:26 sailing weewx[30353]: **** Traceback (most recent call last):
May 14 16:40:26 sailing weewx[30353]: **** File "/opt/weewx-3.1.0/bin/weewx/engine.py", line 837, in main
May 14 16:40:26 sailing weewx[30353]: **** engine.run()
May 14 16:40:26 sailing weewx[30353]: **** File "/opt/weewx-3.1.0/bin/weewx/engine.py", line 187, in run
May 14 16:40:26 sailing weewx[30353]: **** for packet in self.console.genLoopPackets():
May 14 16:40:26 sailing weewx[30353]: **** File "/opt/weewx-3.1.0/bin/weewx/drivers/vantage.py", line 431, in genLoopPackets
May 14 16:40:26 sailing weewx[30353]: **** for _loop_packet in self.genDavisLoopPackets(200):
May 14 16:40:26 sailing weewx[30353]: **** File "/opt/weewx-3.1.0/bin/weewx/drivers/vantage.py", line 445, in genDavisLoopPackets
May 14 16:40:26 sailing weewx[30353]: **** self.port.wakeup_console(self.max_tries, self.wait_before_retry)
May 14 16:40:26 sailing weewx[30353]: **** File "/opt/weewx-3.1.0/bin/weewx/drivers/vantage.py", line 66, in wakeup_console
May 14 16:40:26 sailing weewx[30353]: **** self.flush_input()
May 14 16:40:26 sailing weewx[30353]: **** File "/opt/weewx-3.1.0/bin/weewx/drivers/vantage.py", line 200, in flush_input
May 14 16:40:26 sailing weewx[30353]: **** self.serial_port.flushInput()
May 14 16:40:26 sailing weewx[30353]: **** File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line 497, in flushInput
May 14 16:40:26 sailing weewx[30353]: **** termios.tcflush(self.fd, TERMIOS.TCIFLUSH)
May 14 16:40:26 sailing weewx[30353]: **** error: (5, 'Input/output error')
May 14 16:40:26 sailing weewx[30353]: **** Exiting.

JSON export to nosql database (for storing in parse.com, firebase, mongodb, etc.)

Is there a way to export de data to a nosql type database or JSON values. I prefer to store the data into a cloud service instead of a fragile raspberry SD card.

Include Wunderfixer with weewx

It was advertised, but didn't make it into the release. Also, it has to be upgraded to V3.2.X

StdArchive service should extract everything it can from the accumulators

The StdArchive service should extract everything it can from the accumulators, whether or not software record generation is used.

(from the TODO file)

Please change extraSensors numbering in wmr200.py

I have changed my weatherstation from wmr100 to wmr200 and have recognized a little difference/inconsistency.
While at wmr100.py the numbering for extra sensors starts at 1 (extraTemp1, extraHumid1),
the wmr200.py driver starts the numbering at 2 for extra sensor no.1 (at channel 2).
So you have to update your sql-columns with
UPDATE archive SET extraTemp3 = extraTemp2
UPDATE archive SET extraTemp2 = extraTemp1
and so on.

At wmr100.py the channel-number is subtracted with 1 - "channel-1":
...
_record['extraTemp%d' % (channel-1)] = T
_record['extraHumid%d' % (channel-1)] = R
...

So, if you would change at lines 1041,1042
record['extraTemp%d' % sensor_id] = temp
record['extraHumid%d' % sensor_id] = humidity
to
record['extraTemp%d' % (sensor_id-1)] = temp
record['extraHumid%d' % (sensor_id-1)] = humidity
the wmr200 driver would be "compatible" to the wmr100 driver in this way and if you don't have to update the database, if you switch from wmr100 to wmr200.
To update an existing WMR200 database, you could do with the sql commands:
UPDATE archive SET extraTemp1 = extraTemp2
UPDATE archive SET extraTemp2 = extraTemp3
...

rainRate gets added to all LOOP packets, even those without rain (SF: #46 )

look at: https://groups.google.com/forum/#!topic/weewx-user/_W5tW6nw3hs

Note: pulled from sourceforge issue 46

schema extension does not work

i tried to extend the wview schema as described in the customization guide:

import schemas.wview
schema_with_electricity = schemas.wview.schema + [('electricity', 'REAL')]

this fails with an exception:

Jul 18 17:07:28 burgos weewx[3921]:     ****  No module named wview
Jul 18 17:07:28 burgos weewx[3921]:     ****  Traceback (most recent call last):
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 836, in main
Jul 18 17:07:28 burgos weewx[3921]:     ****      engine = EngineClass(config_dict)
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 75, in __init__
Jul 18 17:07:28 burgos weewx[3921]:     ****      self.loadServices(config_dict)
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 136, in loadServices
Jul 18 17:07:28 burgos weewx[3921]:     ****      self.service_obj.append(weeutil.weeutil._get_object(svc)(self, config_dict))
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 504, in __init__
Jul 18 17:07:28 burgos weewx[3921]:     ****      self.setup_database(config_dict)
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 602, in setup_database
Jul 18 17:07:28 burgos weewx[3921]:     ****      dbmanager = self.engine.db_binder.get_manager(self.data_binding, initialize=True)
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/weewx/manager.py", line 822, in get_manager
Jul 18 17:07:28 burgos weewx[3921]:     ****      default_binding_dict=defaults)
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/weewx/manager.py", line 952, in get_manager_dict_from_config
Jul 18 17:07:28 burgos weewx[3921]:     ****      manager_dict['schema'] = weeutil.weeutil._get_object(schema_name)
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/weeutil/weeutil.py", line 944, in _get_object
Jul 18 17:07:28 burgos weewx[3921]:     ****      mod = __import__(module)
Jul 18 17:07:28 burgos weewx[3921]:     ****    File "/opt/weewx-3.2.0/bin/user/schemas.py", line 2, in <module>
Jul 18 17:07:28 burgos weewx[3921]:     ****      import schemas.wview
Jul 18 17:07:28 burgos weewx[3921]:     ****  ImportError: No module named wview
Jul 18 17:07:28 burgos weewx[3921]:     ****  Exiting.

or this:

Jul 18 17:12:45 burgos weewx[3936]:     ****  name 'schemas' is not defined
Jul 18 17:12:45 burgos weewx[3936]:     ****  Traceback (most recent call last):
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 836, in main
Jul 18 17:12:45 burgos weewx[3936]:     ****      engine = EngineClass(config_dict)
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 75, in __init__
Jul 18 17:12:45 burgos weewx[3936]:     ****      self.loadServices(config_dict)
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 136, in loadServices
Jul 18 17:12:45 burgos weewx[3936]:     ****      self.service_obj.append(weeutil.weeutil._get_object(svc)(self, config_dict))
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 504, in __init__
Jul 18 17:12:45 burgos weewx[3936]:     ****      self.setup_database(config_dict)
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/weewx/engine.py", line 602, in setup_database
Jul 18 17:12:45 burgos weewx[3936]:     ****      dbmanager = self.engine.db_binder.get_manager(self.data_binding, initialize=True)
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/weewx/manager.py", line 822, in get_manager
Jul 18 17:12:45 burgos weewx[3936]:     ****      default_binding_dict=defaults)
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/weewx/manager.py", line 952, in get_manager_dict_from_config
Jul 18 17:12:45 burgos weewx[3936]:     ****      manager_dict['schema'] = weeutil.weeutil._get_object(schema_name)
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/weeutil/weeutil.py", line 944, in _get_object
Jul 18 17:12:45 burgos weewx[3936]:     ****      mod = __import__(module)
Jul 18 17:12:45 burgos weewx[3936]:     ****    File "/opt/weewx-3.2.0/bin/user/schemas.py", line 60, in <module>
Jul 18 17:12:45 burgos weewx[3936]:     ****      schema = schemas.wview.schema + [('lightning', 'REAL')]
Jul 18 17:12:45 burgos weewx[3936]:     ****  NameError: name 'schemas' is not defined
Jul 18 17:12:45 burgos weewx[3936]:     ****  Exiting.

my workaround is to copy the wview.py file to user/schema.py then modify it there.

Setting Weathercloud interval

Is it possible to set a specific time interval for uploading data to the weathercloud site?
By the way, i used the plugin, made by Matthew Wall

rsync process lingers after weewx says it has shut down

if an rsync is in progress when weewx shuts down, the weewx engine thread does not wait for the rsync process to complete. the result is misleading log messages and a dangling weewx process. if a regular (non-rsync) report is still in progress when the engine shuts down, the engine logs something to the effect of "unable to terminate std report thread".

this is what happens when weewx terminates while an rsync process is running (there are two rsyncs in this example, the first succeeded in 3.34 seconds, the second is retrying due to network issues):

Jul 24 19:56:12 burgos weewx[16088]: cheetahgenerator: Generated 8 files for report Exfoliation in 33.72 seconds
Jul 24 19:56:20 burgos weewx[16088]: genimages: Generated 24 images for Exfoliation in 7.49 seconds
Jul 24 19:56:23 burgos weewx[16088]: rsyncupload: rsync'd 32 files (303553 bytes) in 3.34 seconds
Jul 24 19:57:34 burgos weewx[16088]: engine: Shutting down StdReport thread
Jul 24 19:57:34 burgos weewx[16088]: engine: Terminating weewx version 3.2.0

but this is what is still running:

16088 ?        Sl   265:52 /usr/bin/python /opt/weewx-3.2.0/bin/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx-3.2.0.conf
18207 ?        S      0:00 rsync --archive --stats -e ssh /var/weewx/www/ [email protected]:/var/www/weather/wls-ts
18208 ?        S      0:00 ssh -l weewx www.bartschi.org rsync --server -logDtpre.iLsf . /var/www/weather/wls-ts

and this is what shows up in the log after killing the ssh process (18208):

Jul 24 19:56:12 burgos weewx[16088]: cheetahgenerator: Generated 8 files for report Exfoliation in 33.72 seconds
Jul 24 19:56:20 burgos weewx[16088]: genimages: Generated 24 images for Exfoliation in 7.49 seconds
Jul 24 19:56:23 burgos weewx[16088]: rsyncupload: rsync'd 32 files (303553 bytes) in 3.34 seconds
Jul 24 19:57:34 burgos weewx[16088]: engine: Shutting down StdReport thread
Jul 24 19:57:34 burgos weewx[16088]: engine: Terminating weewx version 3.2.0
Jul 24 20:00:07 burgos weewx[16088]: rsyncupload: rsync reported errors: rsync: connection unexpectedly closed (0 bytes received so far) [sender]. rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7]
Jul 24 20:00:07 burgos weewx[16088]: rsyncupload: rsync executed in 2600.71 seconds

Implement a spike detector (SF: #17 )

Possible options include:

option 1: maximum delta relative to previous value.

if the difference between candidate value x1 and previous value x0 is greater than d, reject the candidate value and use None instead. if there are no previous points (e.g. at first sensor sampling during initial startup), then the delta qc is not applied. the configuration would look something like this:

[StdQC] ... [[Delta]] outTemp = 10 # 
temperature change over 10 F will be replaced with None 
pressure = 0.2 # pressure change over 0.2 inHg will be replaced with None

option 2: maximum percentage change over previous N values.

if the delta between the candidate value and the average of the N previous samples is greater than some value (measured as a percentage), reject the candidate value and use None instead.

[StdQC] ... [[PercentChange]] number_of_samples = 3 #
use 3 samples for calculations outTemp = 2 # 
change greater than 2% will be replaced by None pressure = 1 # 
pressure change greater than 1% will be replaced by None

Option 3: max outlier delta.

The idea here is that a jump in value is allowed (maybe a front went through?), but not a jump up, then down.

    [StdQC] [[MaxOutlier]] outTemp = 0.5 barometer = 0.2

Note: pulled from sourceforge issue 17

Weighted sums may not be calculated correctly

Looking through the accumulator code, it appears that the weighted sum is not being used. Just the sum. This would be an issue if the archive interval changed.

$day tag should be iterable

$week, $month, and $year tags are iterable (over the daily summaries)

$day tag should be iterable as well, but this will require iteration over data in the archive table

wxformulas not defined lastest acurite driver

commit 77481e0 causes the following due to a small typo:

Apr 19 07:49:53 pidora weewx[4071]: engine: Caught unrecoverable exception in engine:
Apr 19 07:49:53 pidora weewx[4071]: ****  global name 'wxformulas' is not defined
Apr 19 07:49:53 pidora weewx[4071]: ****  Traceback (most recent call last):
Apr 19 07:49:53 pidora weewx[4071]: ****    File "/usr/share/weewx/weewx/engine.py", line 836, in main
Apr 19 07:49:53 pidora weewx[4071]: ****      engine.run()
Apr 19 07:49:53 pidora weewx[4071]: ****    File "/usr/share/weewx/weewx/engine.py", line 187, in run
Apr 19 07:49:53 pidora weewx[4071]: ****      for packet in self.console.genLoopPackets():
Apr 19 07:49:53 pidora weewx[4071]: ****    File "/usr/share/weewx/weewx/drivers/acurite.py", line 403, in genLoopPackets
Apr 19 07:49:53 pidora weewx[4071]: ****      self._augment_packet(packet)
Apr 19 07:49:53 pidora weewx[4071]: ****    File "/usr/share/weewx/weewx/drivers/acurite.py", line 428, in _augment_packet
Apr 19 07:49:53 pidora weewx[4071]: ****      packet['rain'] = wxformulas.calculate_rain(total, self.last_rain)
Apr 19 07:49:53 pidora weewx[4071]: ****  NameError: global name 'wxformulas' is not defined
Apr 19 07:49:53 pidora weewx[4071]: ****  Exiting.

Fix is incoming...

Make setup.py do nicer formatting to weewx.conf

• spacing before when injecting skin stanzas
• spacing before when injecting database and binding stanzas
• spacing before when injecting restful stanzas
• rows of hashmarks when injecting service stanzas such as [Forecast]
• inject service stanzas somewhere other than end of file
• inject skin stanzas before ftp and rsync

Document how to add a new unit group

consider a StdCache service

this service would retain the value for observations so that full packets would be available even if the hardware produces only partial packets.

the service would have a 'max_age' setting to specify how long it would retain values. it might need a configuration option to indicate which observations it should retain and which it should not. for example, gauge measurements (e.g., outTemp, inHumidity) could be safely cached, cumulative measurements (e.g., rainTotal) could be safely cached, but incremental measurements (e.g., rain) probably should not be cached.

the service would be inserted just before the StdWXCalculate service.

when would this service be useful? for any hardware that produces partial packets, this service would make services that expect a full packet work properly, e.g., the wu rapidfire uploads. of course, for that example the wu rapidfire uploader could be extended to do caching.

would this service be useful anywhere else?

probably do not want this service installed by default for all hardware?

Allow formatting of $almanac.sunrise and $almanac.sunset

Right now, they are hardwired in. They should return a ValueHelper.

implement cron-like report generation

Look at Gary's (?) changes to allow cron-like report generation.

(from the TODO file)

External links to guides don't work

Anchors generated by the Javascript TOC utility work internally within a document, but do not work externally. For example, the link:

http://www.weewx.com/docs/usersguide.htm#Required_skills

does not jump to the relevant section.

Add ability scale y-axis (SF #10)

For measurements that can come in very large quantities, such as watts or bytes, there should either be separate units (MW or MB), or an ability to scale plots.

Note: pulled from sourceforge issue 10

Should multiple option search_list_extensions concatenate?

A user (reasonably) wanted to use the forecast search list extensions in the exfoliation skin. What was unusual is how he did it --- by overriding in weewx.conf:

[[Exfoliation]]
        skin = exfoliation
        HTML_ROOT = public_html/exfoliation
        [[[Extras]]]
            footer = /home/weewx/skins/exfoliation/footer.inc
            header = /home/weewx/skins/exfoliation/header.inc
            hilo = /home/weewx/skins/exfoliation/hilo.inc

        # MT Addition for forecast
        [[[CheetahGenerator]]]
            search_list_extensions = user.forecast.ForecastVariables

He also declared an additional SLE (the "alltime" example) in the exfoliation skin.conf, the conventional way

[CheetahGenerator]
    encoding = html_entities

    # MT added to provide alltime and seven_day tags
    search_list_extensions = examples.xsearch.MyXSearch

Unfortunately, the former masks the latter, so the tag $alltime did not work.

Here's the thread: https://groups.google.com/d/msg/weewx-user/MlD1mTUakIA/Ai8ua22g6oUJ

I must admit, I had not thought of a user overriding search_list_extensions from weewx.conf, but it seems like a reasonable thing to do.

A solution would be that the final SLE should be the concatenation of the two.
Alternatively,we could warn in the log that one is replacing the other.

Document setup.cfg better (SF: #28)

Nowhere in the docs does it describe the role of setup.cfg.

Note: pulled from sourceforge issue 28

Implement a $time_ago tag

Implement a $time_ago, or $timespan tag (see email with Gary Roderick).

(from the TODO file)

Allow lat, lon, and altitude to change

Right now, a station is assumed to be static, but we should support mobile stations.

My preference would be a service that intercepts LOOP and archive records. If it's missing lat/lon data, it would inject it into the record.

Then other services could treat it like any other data. Of course it's not, because it's not stored in the wview schema, but we might be able to treat it as one. Any query to the database would return lat/lon/altitude. Think of it as an optimization.

Another issue: class Station presently acts as a static global data structure. It would have to be changed into yet another weewx service, which continually updates its data on the basis of the stream of LOOP and archive records.

That feels a little artificial. If lat, lon, and altitude are just another bit of data, why aren't other data types cached in Station? It might be useful to know its, ummm... temperature.

missing dewpoint in loop data

sometimes the dewpoint is not calculated:

https://groups.google.com/forum/#!topic/weewx-user/jwggCJJFQ94

implement mechanism for units scaling

for example, in cmon we need kilo, mega, giga, tera

scaling should be an option in the display layer

does it need to be in the units definitions as well?

figure out how to reset usb from software to fix acurite console flakiness

acurite 01036 console. was running fine with the 0.17 acurite driver. then got a bunch of these messages in the log:

Jun 1 00:01:54 saga weewx[6844]: acurite: R1: bogus signal strength (09): 01 17 f5 e3 01 81 21 c9 09 00 Jun 1 00:02:12 saga weewx[6844]: acurite: R1: bogus message flavor (e3): 01 17 f5 e3 01 81 21 c9 09 00 Jun 1 00:02:12 saga weewx[6844]: acurite: R1: bogus signal strength (09): 01 17 f5 e3 01 81 21 c9 09 00 Jun 1 00:02:30 saga weewx[6844]: acurite: R1: bogus message flavor (e3): 01 17 f5 e3 01 81 21 c9 09 00 Jun 1 00:02:30 saga weewx[6844]: acurite: R1: bogus signal strength (09): 01 17 f5 e3 01 81 21 c9 09 00 Jun 1 00:02:30 saga weewx[6844]: acurite: R2: unknown calibration constants: 02 00 00 83 59 1b 39 01 a1 05 2d 1f d1 2b cf 13 89 0f 2d 0d 0f 1d 6d 5d d9 Jun 1 00:02:48 saga weewx[6844]: acurite: R1: bogus message flavor (e3): 01 17 f5 e3 01 81 21 c9 09 00 Jun 1 00:02:48 saga weewx[6844]: acurite: R1: bogus signal strength (09): 01 17 f5 e3 01 81 21 c9 09 00 Jun 1 00:03:07 saga weewx[6844]: acurite: R1: bogus message flavor (e3): 01 17 f5 e3 01 81 21 c9 09 00

unplug the usb then replug the usb and everything ran fine again. no need to power cycle the station. no need to restart weewx.

will a software reset of the usb fix this problem?

or should we detect the bogus state then let the weewx engine do a reload of the driver? (need to do a test to see if that would actually get comms to work again - manual testing seems to indicate the answer is no, at least not without a reset somewhere in the initialization)

doing a usb reset caused problems with 02032 consoles, but seemed ok with 01036 and 01035 consoles (this is anecdotal and not confirmed)

design a mechanism for defining report defaults in StdReport

leaf nodes in StdReport propagate to reports, but there is not yet a mechanism to propagate non-leaf nodes. for example, if i want to apply a [Units] section to every report, i must copy/paste the [Units] to every report; if i define a [[Units]] section just under [StdReport], the Units section is treated as a report.

proposed solution: define a [StdReport][[defaults]] section. anything in the [[defaults]] stanza will be the fallback/default value for every report.

why is this necessary? to make it easier and less error-prone to apply units across all reports. to reduce the amount of copy/paste required for labels when using multiple sensors (e.g., renaming pond temperature). to minimize the size of the StandardReport (and whatever new, adaptive, responsive report we define as the successor to StandardReport).

Setting lower_case_table_names = 1 on MySQL results in AttributeError

Pretty much summarized by this thread

One solution is to introduce an option that would do case-insensitive comparisons.

Sensor failed event (SF: #14 )

Add an event SENSOR_FAILED or something similar, to signal problems with a sensor or station.

Note: pulled from sourceforge issue 14

add caching to weather underground rapidfire uploader

since weather underground will not work with partial packets, make the uploader upload only complete packets, where 'complete' is defined as the set of observations that weather underground accepts.

the uploader should make a debug log entry when it receives a partial packet. that way users can tell when the uploaded data are only partially 'fresh'.

Wind chill value from hardware not invalidated if not enough info available for calculation

This thread pretty much describes the problem.
https://groups.google.com/forum/#!topic/weewx-user/Q_xWRxBFHsU

Document anti-aliasing feature (SF: #43 )

Nick Dajda's anti-aliasing feature is included in V3.1. After testing it a bit, it needs to be documented.

Note: pulled from sourceforge issue 43

3.2.0a4: skin directory not removed when extension uninstalled

1. clean install
2. wee_extension --install exfoliation.tgz
3. wee_extension --uninstall exfoliation
4. the exfoliation skin directory is not removed. it should be.

$current tag should report all observations in a LOOP packet

the $current tag should include all observations, not just those in the archive table schema

3.2.0a4: no empty line before each report stanza in weewx.conf

installing an extension should put an empty line before the extension's report in weewx.conf.

1. clean install
2. wee_extension --install forecast.tgz
3. forecast stanza in StdReport does not have empty line before it

3.2.0a4: dangling commas in service lists

1. clean install
2. check weewx.conf: no comma at end of archive_services
3. wee_extension --install forecast.tgz
4. wee_extension --uninstall forecast
5. archive_services now has a dangling comma

also, data_services starts out with a dangling comma. the code should be smart enough to deal with dangling commas, but we should not put them in weewx.conf in the first place and whatever wee_* utilities process weewx.conf should not leave them.

Guard against database locks

locks are still showing up in weewx v3.1.0. the most repeatable case is on a slower system with forecast reports when the forecast database is configured to retain all forecasts (this is not the default configuration). since the forecast database is not indexed, the queries can take a long time.

i have also seen locks on a 2012-ish dell dual-processor machine with terabytes of spinning disk on the weewx.sdb database (data since 2009 - around 2 million records). in that case the locks caused weewx to die when it tried to write weather data to the database (weewx v2.6.4).

this is from 24 march 2015 with weewx 2.6.4:

Mar 23 04:55:52 sailing weewx[19489]: wxengine: Caught unrecoverable exception in wxengine:
Mar 23 04:55:52 sailing weewx[19489]: **** database is locked
Mar 23 04:55:52 sailing weewx[19489]: **** Traceback (most recent call last):
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weewx/wxengine.py", line 963, in main
Mar 23 04:55:52 sailing weewx[19489]: **** engine.run()
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weewx/wxengine.py", line 189, in run
Mar 23 04:55:52 sailing weewx[19489]: **** self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weewx/wxengine.py", line 210, in dispatchEvent
Mar 23 04:55:52 sailing weewx[19489]: **** callback(event)
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weewx/wxengine.py", line 562, in post_loop
Mar 23 04:55:52 sailing weewx[19489]: **** self._catchup(self.engine.console.genArchiveRecords)
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weewx/wxengine.py", line 625, in _catchup
Mar 23 04:55:52 sailing weewx[19489]: **** origin='hardware'))
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weewx/wxengine.py", line 210, in dispatchEvent
Mar 23 04:55:52 sailing weewx[19489]: **** callback(event)
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weewx/wxengine.py", line 574, in new_archive_record
Mar 23 04:55:52 sailing weewx[19489]: **** self.archive.addRecord(event.record)
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weewx/archive.py", line 192, in addRecord
Mar 23 04:55:52 sailing weewx[19489]: **** e))
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weedb/init.py", line 130, in exit
Mar 23 04:55:52 sailing weewx[19489]: **** self.connection.commit()
Mar 23 04:55:52 sailing weewx[19489]: **** File "/opt/weewx-2.6.4/bin/weedb/init.py", line 97, in commit
Mar 23 04:55:52 sailing weewx[19489]: **** self.connection.commit()
Mar 23 04:55:52 sailing weewx[19489]: **** OperationalError: database is locked
Mar 23 04:55:52 sailing weewx[19489]: **** Exiting.

here is another example from a raspberri pi:

Jul 30 08:55:22 hummingbird weewx[959]: wxengine: Caught unrecoverable exception in wxengine:
Jul 30 08:55:22 hummingbird weewx[959]: **** database is locked
Jul 30 08:55:22 hummingbird weewx[959]: **** Traceback (most recent call last):
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weewx/wxengine.py", line 962, in main
Jul 30 08:55:22 hummingbird weewx[959]: **** engine.run()
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weewx/wxengine.py", line 189, in run
Jul 30 08:55:22 hummingbird weewx[959]: **** self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weewx/wxengine.py", line 210, in dispatchEvent
Jul 30 08:55:22 hummingbird weewx[959]: **** callback(event)
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weewx/wxengine.py", line 562, in post_loop
Jul 30 08:55:22 hummingbird weewx[959]: **** self._catchup(self.engine.console.genArchiveRecords)
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weewx/wxengine.py", line 625, in _catchup
Jul 30 08:55:22 hummingbird weewx[959]: **** origin='hardware'))
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weewx/wxengine.py", line 210, in dispatchEvent
Jul 30 08:55:22 hummingbird weewx[959]: **** callback(event)
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weewx/wxengine.py", line 574, in new_archive_record
Jul 30 08:55:22 hummingbird weewx[959]: **** self.archive.addRecord(event.record)
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weewx/archive.py", line 192, in addRecord
Jul 30 08:55:22 hummingbird weewx[959]: **** e))
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weedb/init.py", line 130, in exit
Jul 30 08:55:22 hummingbird weewx[959]: **** self.connection.commit()
Jul 30 08:55:22 hummingbird weewx[959]: **** File "/home/weewx/bin/weedb/init.py", line 97, in commit
Jul 30 08:55:22 hummingbird weewx[959]: **** self.connection.commit()
Jul 30 08:55:22 hummingbird weewx[959]: **** OperationalError: database is locked
Jul 30 08:55:22 hummingbird weewx[959]: **** Exiting.

Doing 'setup.py install --extension' should prompt for required parameters

in v3.1 the behavior is to inject required parameters
into weewx.conf. for passwords and tokens, inject INSERT_XXX_HERE. when
service starts up, detect missing or default passwords/tokens and bail out
with a log message (as the restful services do).

if the installer prompts for the parameters required by an extension then user would not have to do the extra step of modifying the configuration file.

WU Rapidfire updates may not work for "partial record" stations (SF: #30 )

For stations that do not create full LOOP packets, the WU's RF protocol may not work.
May have to bundle the partial records together for larger, full posts.

Note: pulled from sourceforge issue 30

syntax for StdWXCalculate settings

what syntax should we use for StdWXCalculate settings?

the default settings for StdWXCalculate look like this:

[StdWXCalculate]
    pressure = prefer_hardware
    barometer = prefer_hardware
    altimeter = prefer_hardware
    windchill = prefer_hardware
    heatindex = prefer_hardware
    dewpoint = prefer_hardware
    inDewpoint = prefer_hardware
    rainRate = prefer_hardware

however, there are other parameters that might need to be adjusted. for example, the rainRate period, the algorithm for altimeter calculation, the algorithm and associated parameters for maximum theoretical solar radiation, evapotranspiration parameters.

here are some alternatives:

option 1: separate stanzas for calculations and algorithms

[StdWXCalculate]                                                        
    ignore_zero_wind = True                                             
    rain_period = 900           # for rain rate                         
    et_period = 3600            # for evapotranspiration                
    wind_height = 2.0           # for evapotranspiration                
    atc = 0.8                   # for solar radiation RS                
    nfac = 2                    # for solar radiation Bras              
    max_delta_12h = 1800                                                
    [[Calculations]]                                                    
        windchill = hardware                                            
        heatindex = prefer_hardware                                     
        dewpoint = software                                             
    [[Algorithms]]                                                      
        altimeter = aaASOS                                       
        maxSolarRad = RS                                                

option 2: parameters in a separate stanza

[StdWXCalculate]
    windchill = hardware
    heatindex = prefer_hardware
    dewpoint = software
    [[parameters]]
        ignore_zero_wind = True                                         
        rain_period = 900           # for rain rate                     
        et_period = 3600            # for evapotranspiration            
        et_wind_height = 2.0           # for evapotranspiration         
        rs_atc = 0.8                   # for solar radiation RS         
        bras_nfac = 2                    # for solar radiation Bras     
        max_delta_12h = 1800                                            
        altimeter_algorithm = aaASOS                                    
        maxSolarRad_algorithm = RS                                      

option 3: keep parameters with variables. variable can be scalar (no params) or dict (with params). if 'source' is not specified, then default to 'prefer_hardware'.

[StdWXCalculate]                                                        
    windchill = hardware
    heatindex = prefer_hardware
    dewpoint = software
    [[wind]]
        ignore_zero_wind = True
    [[rainRate]]
        rain_period = 900
    [[ET]]
        et_period = 3600
        et_wind_height = 2.0
    [[pressure]]
        max_delta_12h = 1800                                            
    [[altimeter]]                                                       
        source = prefer_hardware                                        
        algorithm = aaASOS                                              
    [[maxSolarRad]]                                                     
        source = software                                               
        algorithm = RS                                                  
        rs_atc = 0.8                   # for solar radiation RS         
        bras_nfac = 2                    # for solar radiation Bras     

option 4: flat namespace

[StdWXCalculate]
    ignore_zero_wind = True
    rain_period = 900           # for rain rate
    et_period = 3600            # for evapotranspiration
    et_wind_height = 2.0           # for evapotranspiration
    rs_atc = 0.8                   # for solar radiation RS
    bras_nfac = 2                    # for solar radiation Bras
    max_delta_12h = 1800  # for pressure
    windchill = hardware
    heatindex = prefer_hardware
    dewpoint = software
    altimeter_algorithm = aaASOS
    maxSolarRad_algorithm = RS

perhaps something else?

option 1 is currently implemented in 3.2.0, with backward compatibility for the non-grouped calculations. but it is rather haphazard and should be changed, imho.

Add indoor temp and humidity to Ambient protocol (SF: #7 )

Add indoor temperature and humidity to the Ambient protocol. User should be able to opt-out of sending it.

Check to see what WU does with it.

Note: pulled from sourceforge issue 7

make cheetahgenerator not require a default wx_binding

when a weewx installation does not have a wx_binding, cheetahgenerator fails

Temperature differences should be a separate unit (SF #8 )

Taking the difference of two temperatures really results in a new unit, because the function needed to convert it changes. FtoC(x-y) is not equal to FtoC(x) - FtoC(y).

Note: pulled from sourceforge issue 8

Refactoring how to specify ROOT directories.

This is the continuation of a conversation started in commit 493badc

Thinking a bit more about this, moving all ROOT options to the top of weewx.conf may not be such a good idea.

By definition, CONFIG_ROOT is where weewx.conf is located, so specifying it in the config file is redundant and could introduce an inconsistency.

SKIN_ROOT and HTML_ROOT are defined in the context of StdReport, so I don't see any upside to moving them to global scope.

The location of BIN_ROOT could be useful. It would saving having to search the file system for the weewx Python packages. Perhaps it should be renamed "LIB_ROOT"?

That leaves SQLITE_ROOT, which could be defined in the [Databases] stanza. The present option root goes away:

[Databases]
    # This section defines the actual databases

    # Where the SQLite databases reside:
    SQLITE_ROOT = /var/lib/weewx

    [[archive_sqlite]]
        database_name = weewx.sdb
        driver = weedb.sqlite

alternatively, we introduce a [SQLite] (and MySQL) stanza:

[SQLite]
  SQLITE_ROOT = /var/lib/weewx
  driver = weedb.sqlite

[MySQL]
  host = localhost
  user = weewx
  password = weewx
  driver = weedb.mysql

I like the latter approach. It's a more elegant way of introducing new databases.

3.2.0a4: inconsistent use of double quotes for station location

1. clean install with location name 'whatever'
2. wee_config --reconfigure
3. location name no longer has double quotes around it

there is logic in the code for dealing with a location that has a comma in it, so there is no need to put the location in double quotes.

should drivers load when modules they use are not installed?

if you install using setup.py without python-serial or python-usb installed, the driver listing misses some drivers. not much we can do about the ones that use usb, but i could push the serial imports to a lower level for those that use serial.

do we want to make driver modules load properly even if they depend on python modules that are not installed?

WMR200 driver calculates dayRain incorrectly

The type dayRain should be the rain since midnight, but the WMR200 driver actually emits the rain over the last 24 hours, excluding the current hour.