API Version 2, Beta testing notes, issues, and feedback about pirateweather

[alexander0042]

Great final catches, and put them in! Today's the official go live day, so I'm going to (very slowly) start moving the production endpoint over to V2.0.1. Definitely let me know if anything crops up, and since V1 will keep running in the background as a fallback

[cloneofghosts]

I'm not sure how other APIs handle the situation but I think that most show a precipitation icon if there's any precipitation currently or forecasted. As far as I can tell I think that's how Dark Sky did it as their translation module lists "very-light-X" as 0–0.4 mm/hr (they also say snow is about 1/3 of rain). You could always have the summary say something like "Possible Rain" for really low intensities.

For snowfall is the intensity converted from cm/h to mm/h in the precipIntensity? From my experience when the precipitation type changes from snow to rain or sleet I notice no difference. From your suggestion it seems like it would be harder for the snow icon to show compared to rain/sleet. My opinion would be the lower the threshold the better.

EDIT: Sometimes I'm getting version 2.0h and sometimes 2.0i and it seems to be random which one I get. Any idea on why this would be happening?

EDIT2: The subhourly HRRR doesn't have visibility does it? I've noticed times when the NBM visibility seems off compared to observed visibility at the airport. Now is a good example where with NBM enabled visibility is ~8km whereas HRRR is ~16km which is closer to the observed visibility of 24km at the airport.

[cloneofghosts]

• NBM cloud cover was more of a comment on something I noticed than an actual issue as I'm aware you cannot do anything to change it.
• That works for me.
• I'll create an issue for it so it can be tracked.

I haven't encountered any more issues so I think it's ready for release. The V2 changes will need to be documented but there's another issue for that. I will keep this open until release in case anyone else encounters any more issues.

[trevorturk]

(Just wanted to chime in to say that I'm so excited for v2, thank you for all of your hard work!)

[alexander0042]

Thanks for saying that @trevorturk! It was your discussion/ research into the daily max/,min that started me down this (very slow road), so means a lot to hear that

[alexander0042]

As an first note, here are my "known bugs" to get things started, in addition to the list over at #155:

• Rounding is chaos- needs to be aligned with the production endpoint
• Excluding sections is currently broken
• Daily high/low seems to be a day behind where it should be
• Model (plus elevation) index values should be in lat/lon, not just index
• Precipitation error needs review

[cloneofghosts]

• Daily high/low seems to be a day behind where it should be

I think it's actually a day ahead of where it should be. The hourly section for me shows a maximum temperature of 9.79 and the daily high shows 11.08.

Taking a look at the V2 features I don't really see anything which should break DarkSky compatibility as they are all new fields but I guess you would rather play it safe?

I would also hazard a guess that you haven't looked at seeing if you can use the existing translation module to add in translations without doing the summaries?

EDIT: Seems like hrrr_subh and hrrr_0-18 don't have the same sourceTimes and are off by 1 or 2 hours.

[Monstergerm]

I think the daily0 temperature high (Today's High) is actually yesterday's High (as of 10 am local time). The hourly forecast and daily0 temperate max seem to be right.

[alexander0042]

Thanks for consolidating all that! Just pushed Rev C out, which should fix all the daily max/min timing issues (turns out there were several different things that weren't working), along with most of the #155 items!

Couple other points:

• HRRR subhourly and HRRR 0-18 will always be 1-2 hours off now. I'm trying to ingest the subhourly data as fast as possible, so it gets read as soon as four timesteps are published. The hourly data waits until the 18 hour timestep is out, and then has to process the previous 36 hours of data to create a cohesive timeseries.
• The translation module will be easy to add in, but the harder part is writing it in English first. It's just a giant if statement, so very doable, but that's for later.
• Some libraries don't like it when I add additional fields to the json output, hence hiding the new variables behind a v2 flag. I don't know how big of an issue this is though, so maybe doesn't matter.

Here's a question: should precipitation accumulation for day 0 (the current day) be the total for the day, or the total remaining for that day. So if it's 2 pm and it's already snowed 5 cm, with another 2 cm in the forecast, would it make more sense to return 2 cm (how it currently works), or 7 cm (how the temperature max/min will work)?

[cloneofghosts]

Thanks for consolidating all that! Just pushed Rev C out, which should fix all the daily max/min timing issues (turns out there were several different things that weren't working), along with most of the #155 items!

Feel free to mark any items in that issue as complete as you fix them. I think I marked any ones that I noticed as complete after I saw your comment.

• HRRR subhourly and HRRR 0-18 will always be 1-2 hours off now. I'm trying to ingest the subhourly data as fast as possible, so it gets read as soon as four timesteps are published. The hourly data waits until the 18 hour timestep is out, and then has to process the previous 36 hours of data to create a cohesive timeseries.

Makes sense and thanks for clarifying.

• The translation module will be easy to add in, but the harder part is writing it in English first. It's just a giant if statement, so very doable, but that's for later.

I mentioned it would be nice for V2 so totally understandable it's not high on the priority list.

• Some libraries don't like it when I add additional fields to the json output, hence hiding the new variables behind a v2 flag. I don't know how big of an issue this is though, so maybe doesn't matter.

I haven't directly used any libraries myself but I would think it would skip over any fields instead of throwing an error but I understand the caution . What you could always do is initially keep them behind a flag but after a certain amount of time remove the flag and add it to the base API. Would give any libraries time to implement the changes without breaking current compatibility.

Here's a question: should precipitation accumulation for day 0 (the current day) be the total for the day, or the total remaining for that day. So if it's 2 pm and it's already snowed 5 cm, with another 2 cm in the forecast, would it make more sense to return 2 cm (how it currently works), or 7 cm (how the temperature max/min will work)?

To me it would make more sense to keep it how it currently works and as far as I know it's how most APIs handle things.

EDIT: I see the hourly and possibly daily summaries still sometimes show a different icon/summary

"hourly": {
"summary": "Partly Cloudy",
"icon": "cloudy",

I noticed his with cloud cover but it seems like some of the currently data initially pulls in data from the next hour at the start of the hour then interpolates to the current hour values when it should be the other way around.

[cloneofghosts]

@alexander0042 Something broke with the icon/summary calculation on the currently section. Today no matter what the currently cloud cover showed the icon/summary was always clear. When there is any precipitation or when precipitationProbability is over 30% (even if there is no precipitation currently) it shows None.

Everything used to work without issue so probably something in the 2.0c update broke the icon/summary calculation on the currently block.

[alexander0042]

Great catch- it was fixing the cloud cover percentages in the currently section, I forgot to update the icon thresholds from 75 to 0.75! Fixed it now, and should see the changes in the next 15 minutes or so

[cloneofghosts]

Did the precipitation issue also get fixed? If there is any precipitation or if the precipitationProbability is over 30% the currently section displays None. A location to test this with is Winnipeg or (49.900501,-97.139313)

Also this as referenced in my comment above:

I noticed his with cloud cover but it seems like some of the currently data initially pulls in data from the next hour at the start of the hour then interpolates to the current hour values when it should be the other way around.

EDIT: Seems like the icon/summary isn't fully fixed as it keeps switching back to Clear and then back to the proper condition. This is what I see as the summary in HA (I only update the integration every 30m):

EDIT2: Sorry for asking a million questions but I have a few more questions in addition to the above:

1. You mentioned that you process the HRRR subhourly data after 4 data steps so I thought that it would mean that it would mean that the subhourly data would appear in the API faster but it appears that isn't the case.
2. Is the precipitation data still coming from the HRRR hourly or does the subhourly have that data as well? Just curious because it missed a ~2h storm at approx. 7pm but did get the later storm without issue. (To be fair the live API didn't show anything either so it's not just a development endpoint issue)

[alexander0042]

Don't apologize for the question, they're what help the API to improve, so keep em coming! The precipitation type issue is a bug with my GEFS interpolation, fixing it today!

With respect to SubHourly, it runs out four hours, so 16 steps, and should be ingested faster than the hourly HRRR, so let me know if the times aren't showing that. In terms of sourcing, assuming we're in the right coverage areas (writing this up so I have a note for the docs):
Currently:

• Intensity: subhourly
• Type: subhourly
• Probability: NBM

Minutely

• Intensity: subhourly
• Type: subhourly
• Probability: NBM
• Error: GEFS

Hourly/ Daily

• Intensity: NBM
• Type: NBM
• Probability: NBM
• Error: GEFS

The logic behind this is that subhourly should be more accurate below an hour; however, I don't have a probability from it, so fall back to NBM for that. Part of the issue is when NBM shows rain, but HRRR doesn't, since there isn't an intensity or type. Conversely, when HRRR has precipitation, but probability in the NBM,then there's rain with 0 probability. What V1 does is set it to zero if either of those show zero, which is a clean and easy solution, but runs the risk of missing something.

[cloneofghosts]

With respect to SubHourly, it runs out four hours, so 16 steps, and should be ingested faster than the hourly HRRR, so let me know if the times aren't showing that. In terms of sourcing, assuming we're in the right coverage areas (writing this up so I have a note for the docs): Currently:

• Intensity: subhourly
• Type: subhourly
• Probability: NBM

Minutely

• Intensity: subhourly
• Type: subhourly
• Probability: NBM
• Error: GEFS

Hourly/ Daily

• Intensity: NBM
• Type: NBM
• Probability: NBM
• Error: GEFS

Thanks for that information it's very helpful. I haven't noticed the sub-hourly integrating any faster than it was before but I don't pay too much attention to the sourceTimes.

The logic behind this is that subhourly should be more accurate below an hour; however, I don't have a probability from it, so fall back to NBM for that. Part of the issue is when NBM shows rain, but HRRR doesn't, since there isn't an intensity or type. Conversely, when HRRR has precipitation, but probability in the NBM,then there's rain with 0 probability. What V1 does is set it to zero if either of those show zero, which is a clean and easy solution, but runs the risk of missing something.

I'm little bit confused as to what you said there but my takeaway is that sometimes NBM forecasts precipitation with no intensity or type and in that case you set the type to none and the intensity to 0? For the currently/minutely sections if HRRR is showing precipitation but NBM has a 0% chance of precipitation then it also gets set to 0?

In the second case my preference would be that the minutely/currently data show precipitation even if the percent chance is 0. Not sure how other APIs handle things but that would make the most sense to me.

So you are fixing the currently icon/summary points today or is that an unrelated issue? Is the interpolation issue where it starts from the next hour instead of the current one also going to be fixed today or will that be fixed afterwards?

[alexander0042]

Yea, great comments here! I put the changes in at lunch, and so most of these issues should be resolved in version "e". The missing icons was a separate problem, but should also be fixed.

The method I described in the previous comment started getting super complex, so I simplified it along the lines of what you suggested. It means that probability and intensity are not always aligned, but I think that it's actually not that big of a problem, and brings the results closer to the raw model (which is always good). Now what happens:

Currently/ Minutely

• Intensity: SubHourly -> NBM -> GEFS
• Type: SubHourly -> NBM -> GEFS
• Probability: NBM -> GEFS
• Error: GEFS

Hourly/ Daily

• Intensity: NBM -> HRRR -> GEFS
• Type: NBM -> HRRR -> GEFS
• Probability: NBM -> GEFS
• Error: GEFS

So type always comes from the same source as intensity, which should avoid most of that weirdness. Moreover, unless people specifically exclude NBM, then most people will either get NBM or GEFS, both of which give me probability and intensity, so fewer inconsistencies as well. The only remaining weirdness is currently/, minutely, but probability also makes the least sense for those ones, and isn't used as often in applications. i.e. it makes more sense to say there's a 50% chance that it will rain tomorrow than a 50% chance it's currently raining

[cloneofghosts]

I'm not entirely sure why Dark Sky decided to have precipitationProbability on the currently section as it doesn't really make a whole lot of sense (to me having it on minutely doesn't make sense either but maybe it's useful to someone). I'm also kinda surprised that HRRR hasn't been showing any snow as it's been snowing for a while with some big flakes too.

The currently summary/icon doesn't show rain/snow/sleet if there's any precipitation so that'll need to be fixed.

Has the interpolation issue with the currently data been fixed at all? From what I can tell the datapoints which use NBM as their source start the top of the hour using the data from the next hour and then interpolate to the current hours data. An example would be:

hour 0 cloud cover is 0.77
hour 1 cloud cover is 0.88
start of the hour cloud cover is 0.88
end of the hour cloud cover is 0.77

It also might be a good thing to document which variables are coming from which model for the currently conditions. I have my ideas on what is coming from where but I could be wrong.

[alexander0042]

Looking into the cloud cover, but re: the icon, can you elaborate on that? It's looking correct on my end (ex. 48,-84.5), but might be something else

[cloneofghosts]

I think visibility is also affected by the same bug as cloud cover. The other bug may have already been fixed but last night I noticed that Montreal was saying Cloudy when the precipIntensity was 6. I had to search around to find a place where it's raining/snowing. I tested Owen Sound but it seems to work fine.

EDIT: OH I think I figured out what the issue was. I think the precipProbability on the currently section was below the threshold to show the precipitation icon which is why it showed Cloudy instead of Rain.

[alexander0042]

Got it! You're right that it's a linear interpolation between hours 0 and 1 but I had the interpolation factors backwards 😒, hence the sudden jumps every hour. Just pushed out f, which addresses this

[cloneofghosts]

Just tested and the fix didn't fix the issue. Still seeing it jump from the current hour to the next hour at the start of the hour and it's still interpolating backwards.

EDIT: I also added the last few bugs from issue #155 in the comment where I added links to the open issues. Probably helpful if everything is in one place.

I also see that the interpolation issue has been fixed.

[cloneofghosts]

@alexander0042 In addition to the above comment I noticed on the development API that the models are integrating slower than they are on the live API.

Development:

"sourceTimes": {
"hrrr_subh": "2024-03-25 16Z",
"hrrr_0-18": "2024-03-25 15Z",
"nbm": "2024-03-25 15Z",
"hrrr_18-48": "2024-03-25 12Z",
"gfs": "2024-03-25 12Z",
"gefs": "2024-03-25 06Z"
},

Live:

"sourceTimes": {
"hrrr_0-18": "2024-03-25 17:00:00",
"hrrr_subh": "2024-03-25 17:00:00",
"hrrr_18-48": "2024-03-25 12:00:00",
"gfs": "2024-03-25 12:00:00",
"gefs": "2024-03-25 06:00:00"
},

I know that with the change HRRR 0-18 should be slower than on live but the subhourly HRRR is also 1h behind on the development API when compared to live.

[cloneofghosts]

@alexander0042 One thing I've noticed is that the API can show a precipitation icon/summary in the currently icon with precipIntensity being 0.

Not sure if it's related to the issue with precipitation I pointed out earlier (and have an example in #180) or if it's just that the instensity is so small that because of rounding it doesn't show any value though the minutely data isn't showing any precipitation for 28 minutes.

"currently": {
  "time": 1712154480,
  "summary": "Rain",
  "icon": "rain",
  "nearestStormDistance": 0,
  "nearestStormBearing": 0,
  "precipIntensity": 0,
  "precipProbability": 0.25,
  "precipIntensityError": 0.47,
  "precipType": "rain",
  "temperature": 5.42,
  "apparentTemperature": -0.15,
  "dewPoint": -1.55,
  "humidity": 0.65,
  "pressure": 996.45,
  "windSpeed": 24.68,
  "windGust": 56.19,
  "windBearing": 96,
  "cloudCover": 0.9,
  "uvIndex": 0.24,
  "visibility": 13.85,
  "ozone": 396.59
},

Now I'm seeing the None icon/summary issue which I thought was already fixed:

"currently": {
  "time": 1712163660,
  "summary": "None",
  "icon": "none",
  "nearestStormDistance": 0,
  "nearestStormBearing": 0,
  "precipIntensity": 0,
  "precipProbability": 0.5,
  "precipIntensityError": 0.67,
  "precipType": "none",
  "temperature": 7.3,
  "apparentTemperature": -0.29,
  "dewPoint": -2.3,
  "humidity": 0.71,
  "pressure": 995.77,
  "windSpeed": 30.78,
  "windGust": 58.04,
  "windBearing": 97,
  "cloudCover": 0.91,
  "uvIndex": 0.66,
  "visibility": 7.86,
  "ozone": 389.17
},

[alexander0042]

Back at this! To answer your question about ingest times, it likely has something to do with the nuances of the processing triggers, but I'm not entirely sure. If you notice it again can you also write down the current time, since then I can go back and see what's happening here.

With respect to currently precipitation, it's because the logic is:

if probability is greater than 25%:
    show the precipitation type

The issue is that since (for currently) the probably comes from NBM and the type from HRRR, these can sometimes disagree. To fix this, I'm thinking of switching it to a threshold approach, where it's something like:

if precipitation rate is greater than 0.25 mm/h:
    show the precipitation type

That way they'll both be from HRRR (or GFS outside of North America), which should fix it (and probably be more accurate).

[cloneofghosts]

Ingest Times - Outside of that one time I haven't really noticed a difference between the live API and the dev API.

Using a threshold instead of using probabliilty makes sense as long as everything is consistent between the different data points. I didn't know this until recently but the DarkSky translation module actually lists the threshold for the different intensites.

DarkSky listed "very light" precipitation as 0-0.25mm/h so it sounds like they're showing an icon for any intensities but I don't mind if 0.1 is used instead of 0.

[alexander0042]

Just to go over the tasks from #180, I want to make sure I hit them all:

Tasks

• negative Longitudes displaying incorrectly (-75 shows as 285)
  ** Fixed in h
• visibility is not capped
  ** Can you recall what it should be capped at?
• hourly and daily precipitation data points are rounded to two decimal places instead of four
  ** Fixed in h
• minutely block seems to show array positioning
  ** What do you mean by this?
• Precipitation Probability can go over 1
  ** Fixed in h
• UV index can display -0
  ** Fixed in h
• precipIntensityand precipAccumulation can go below zero. (Should check the new ones added as well)
  ** Fixed in h
• nearestStormDistance and/or nearestStormBearing can display -0
  ** Fixed in h
• It's possible to not have a precipitation icon/summary when precipIntensity is over 0.25mm/h. Might be related to the precipProbability change
  ** Fixed in h
• The threshold to show the precipitation icon is inconsistent between currently and hourly/daily blocks. Currently shows it no matter the intensity whereas hourly/daily it needs to be above 0.25mm/h. Should be consistent across all blocks
  ** Fixed in h by keeping 0.25 mm/h
• Daily precipitationAccumulation in metric units is wrong unless only snow is forecast.
  ** This one was interesting, and your example from #180 was perfect to understand, what was happening, and why separate fields make more sense. The API picks a daily precipitation type based on the most common, and applies that factor. So if it's going to rain 4 hours for a total of 1 cm, and 12 hours of snow for a total of 10 cm, it'll show 20 cm. But I agree that it would make more sense to use the total, so switching it to that.

I need to test out 2.0h a bit more before releasing, but you should see it tomorrow!

[cloneofghosts]

Just to go over the tasks from #180, I want to make sure I hit them all:

Tasks

• visibility is not capped
  ** Can you recall what it should be capped at?

16.09 km (or 10 miles)

• minutely block seems to show array positioning
  ** What do you mean by this?

This is what I see in the data section in the minutely block:

"0": {
  "time": 1712176260,
  "precipIntensity": 2.1583,
  "precipProbability": 0.79,
  "precipIntensityError": 0.89,
  "precipType": "rain"
},

What the data section shows on the live API

{
  "time": 1712176380,
  "precipIntensity": 2.1094,
  "precipProbability": 0.91,
  "precipIntensityError": 0.86,
  "precipType": "rain"
},

Notice how on live it doesn't show "0":{

• Daily precipitationAccumulation in metric units is wrong unless only snow is forecast.
  ** This one was interesting, and your example from Remaining Bugs/Issues for the V2 Development API #180 was perfect to understand, what was happening, and why separate fields make more sense. The API picks a daily precipitation type based on the most common, and applies that factor. So if it's going to rain 4 hours for a total of 1 cm, and 12 hours of snow for a total of 10 cm, it'll show 20 cm. But I agree that it would make more sense to use the total, so switching it to that.

I thought on live it was a total and always in cm when using metric units. With the new fields precipAccumulation is always in cm like on the live API and liquid is in mm, snow is in cm, and ice is in mm?

[cloneofghosts]

@alexander0042 Did you forget to release v2.0h or are you still testing/fixing things?

[alexander0042]

Took a bit longer to get to get those last couple points knocked off the list, and I wanted to validate some UV points as well. Pushed it live now though, so you should start seeing "h" it in about 5 minutes or so!

[cloneofghosts]

Yup, I see version 'h' now. I think the main bugs have been fixed but will test over the weekend to make sure nothing else funky is going on.

I closed the other issue I created since you can either look at the milestone or the project to see any remaining issues.

EDIT: Just to clarify the currently and hourly sections should show a precipitation icon if intensity is 0.1mm/h or greater (daily and hourly need to have a probability of 30% and greater)? It still seems to be set to 0,25mm/h.

[alexander0042]

Great clarification question. As it stands now (and I don't know if I like this approach):

Currently

• 0.25 mm/h of any precipitation

Hourly

• = 30% probability and either:

• 0.25 mm/h rain/sleet

• 2.5 mm/h snow

Daily

• Max probability >=30% and either

• 1 mm total rain/sleet

• 10 mm total snow

This feels intuitively more or less correct, and I do like having different thresholds for snow or rain, since 1 mm is a lot of rain, but not a lot of snow. Maybe 0.25 mm/h rain+ 1 mm/h of snow would make more sense?

[alexander0042]

Great idea checking the translation library! I agree with the logic here, so I'll set currently to 0.02 mm/h, of rain or snow which is basically anything.

• precipIntensity will always show mm/h, while precipAccumulation will always show cm, since that's the way Dark Sky did it as far as I could tell.
• Good note about HRRR subhourly visibility. There's no visibility data in the subhourly, but there is in the hourly HRRR. Could switch to pulling visibility from HRRR if it seems more accurate (you can use "exclude=nbm" to test this, since it falls back to HRRR, the GFS), but it's similar to explain if it's all from NBM.
• I forgot to restart one of the instances (V2 uses two servers for reliability), hence the random responses. I'm pushing "j" now with the new currently threshold, so should sync shortly!

Looking at the project board, I think all the V2.0 requirements are in, since everything else isn't really part of spec to release it live! I'm thinking of keeping it isolated to the dev endpoint for another week or so, but then slowly rolling it out to the api one

[cloneofghosts]

The change to 0.02 mm/h makes sense to me. I think precipAccumulation is still using mm instead of cm as when I turn on version2 I see that precipAccumulation has the same data as liquidAccumulation. I thought that liquid and ice would be in mm but maybe everything is in cm?

"precipAccumulation": 1.6274,
"liquidAccumulation": 1.6274,
"snowAccumulation": 0,
"iceAccumulation": 0

It's too bad there is no visibility in the subhourly HRRR since it's the most up to date model and it would align better with the precipitation. Even though the HRRR data is a couple of hours delayed compared to the subhourly data it might still be worth it to switch to HRRR.

Thought of this the other day but you're not integrating NBM in Alaska (AK), Puerto Rico (PR), Hawaii (HI), Guam (GU) are you? Doesn't have to be for V2 but might be worth adding for those areas. There's also HRRR Alaska and HRRR Hawaii but they only update every 3h and may not be as worthwhile to add.

As for the remaining list I added a comment on the fire index task as there is data available in NBM though it's not global. The only other thing on the list was the self-hosting task which is just making the code available in a repo somewhere and adding some documentation on how to run the code.

[alexander0042]

Good catch on the NBM fire index! I didn't see that, but it is exactly what I was looking for

[cloneofghosts]

The Fire Index is available for all NBM regions except for Guam and Oceanic. You'll still want to find a global model/source somewhere as NBM isn't available globally but it's a start.

Did you see my note above about precipAccumulation and visibility? Just making sure it wasn't missed. Also a few more things I noticed:

• precipProbability seems to be rounded to 4 decimal places instead of 2
• If I exclude NBM for my location the currently and minutely block says the precipitation type is snow and the minutely summary and icon say snow even though no precipitation is forecasted.
• Just want to confirm the precipIntensity change also applies to the hourly data as well. I see some rain in the forecast and most times it's not showing the icon but I think it's more to do with the probability being below 30%.
• precipIntensityError can still be negative

{
  "time": 1712840400,
  "icon": "cloudy",
  "summary": "Cloudy",
  "precipIntensity": 0,
  "precipProbability": 0,
  "precipIntensityError": -0.0401,
  "precipAccumulation": 0,
  "precipType": "rain",
  "temperature": 9.79,
  "apparentTemperature": 7.85,
  "dewPoint": 6.73,
  "humidity": 0.81,
  "pressure": 1003.2,
  "windSpeed": 12.96,
  "windGust": 18.72,
  "windBearing": 80,
  "cloudCover": 0.81,
  "uvIndex": 1.36,
  "visibility": 16.09,
  "ozone": 373.24
},

[alexander0042]

The fire index was trickier than it should have been! It only comes out every 6 hours, so had to put it in as a new model (NBM_Fire). You might notice the run time is slightly off, but that's just a quirk for the next 3 hours, and should stabilize by tomorrow. In other 'k' news:

• Fixed the precipitation probability rounding
• Added another icon check for no precipitation (I'm not entirely sure about this, need to do a bit more testing)
• Fixed negative precipitation intensity
• Set the accumulation threshold to 0.02 mm for minutely/ hourly

One other nice part of this change was that it showed the new (more flexible) approach to adding models works! Which will make it much easier to incorporate things in the future... maybe even let people submit PR's to add additional ones!

[cloneofghosts]

I'm getting the same issue as before where I get a mix of j and k versions. Couple of quick things I noticed about the fire index:

• No fire index data point on the currently section
• There is no fireIndexMaxTime field on the daily section

A couple of quick quesrions:

• Did the threshold to show the precipitation icon change for the daily block or is it the same as before? I know the other blocks got changed so I'm not sure if this one did as well.
• Are all the precipitation accumulation data points using cm as their units or is it dependent on the type of precipitation? On the live endpoint the precipitation accumulation is always in cm and at least from what I can tell it's not the same on dev.

[alexander0042]

Good idea about the currently, I forgot to add it there, so I'll put it in, along with a max time field. To answer those questions:

• Daily precipitation threshold have been kept the same (1 m of rain/ 1 cm of snow)
• I wanted to keep accumulation consistent, so it's always in cm, for both liquid and solid precipitation. Do you think it would be better to put liquid in mm?

[cloneofghosts]

Yeah, I think it would make sense to have liquid and probably ice accumulation in mm as it makes sense for those precipitation types.

I can't really think of any other bugs so I think we're good for release. Just need to update the documentation (#185) for the 2.0 changes and the updated roadmap.

EDIT: I know this isn't your fault and there is likely not much you can do. The API tends to forecast many hours in a row of 57% cloud cover. Currently it's showing 57% cloud cover for now until 6pm.

[cloneofghosts]

• No fire index data point on the currently section
• There is no fireIndexMaxTime field on the daily section

@alexander0042 has this been added to the latest development version? I checked this morning and I didn't see it yet.

Yeah, I think it would make sense to have liquid and probably ice accumulation in mm as it makes sense for those precipitation types.

Haven't heard anything on this but are we going to leave it as is or change those to to mm instead of cm.

It's too bad there is no visibility in the subhourly HRRR since it's the most up to date model and it would align better with the precipitation. Even though the HRRR data is a couple of hours delayed compared to the subhourly data it might still be worth it to switch to HRRR.

Again, are we going to leave things as is or change to HRRR. I don't mind either way but using HRRR would sync things up better with the subhourly precipitation.

Thought of this the other day but you're not integrating NBM in Alaska (AK), Puerto Rico (PR), Hawaii (HI), Guam (GU) are you? Doesn't have to be for V2 but might be worth adding for those areas. There's also HRRR Alaska and HRRR Hawaii but they only update every 3h and may not be as worthwhile to add.

I think this was just missed. I checked yesterday and it doesn't look like you're using NBM in those regions. Would definitely improve accuracy as those regions are only using GFS/GEFS which update every 6h.

[alexander0042]

Should see version "l" coming online shortly with these fixes! In particular:

• Added fire index to currently
• Added fireIndexMaxTime to daily
• Fixed a bug where the source time for the HRRR model 0-18 model was incorrect
• Fixed a bug where the fire index wasn't ingesting
• Added model lat lon values

And to answer those questions:

• I like the simplicity of keeping NBM as the primary source for cloud cover, and I think it should broadly be more accurate. Since this determines the icon, it's an unusually important parameter.
• Also a little counter-intuitive, but I think keeping liquidAccumulation in cm is also clearest. That way accumulations are always in cm, and it doesn't vary by type.
• Definitely want to start ingesting those as well, but it'll have to be a future task for now. They're an entirely new set of files, so while most of the code would be the same, it'll be a bit of a production

[cloneofghosts]

@alexander0042 Just noticed a few last minute bugs with the development version:

cloudCover can go below 0 on hourly:

{
  "time": 1713600000,
  "icon": "clear-night",
  "summary": "Clear",
  "cloudCover": -0.03,
},

I also noticed the bug with longitude for nbm and hrrr in the sourceIDX block that happened with the location longitude where -75 is showing as 285.

[cloneofghosts]

I also saw that precipAccumulation had more than 4 decimal places yesterday but I do not see the same issue this morning. Might be when there are more than one type of precipitation for the day?

[cloneofghosts]

The version number in the flags section seems to have disappeared. It may have happened when you removed the processTime from the flags section earlier.

[cloneofghosts]

It's been a few days and everything seems stable so I'll close this issue. Thanks to everyone who tested the development version.