is there a change log for the api? Trying to find when /observations/current got depricated

[edclow]

Seems like observations/current has been either deprecated and/or replaced with observations/latest

If this accurate?

[akirmse]

Seconding this request, as well as a place to go for announcements a few months before breaking changes are made. I just found out my 50k mobile users are broken because the API changed recently.

[scadergit]

This endpoint was deprecated in the API and noted in the specification over a year ago, but then the deploy schedule went different than expected and the notice of removal was lost in the shuffle. We recognize the importance of a service to notify customers of breaking changes.

The official notice would have been an NWS Service Change Notice (SCN), but many developers don't know those exist let alone monitor them for changes to the API. We have debated other solutions to compliment SCN's, but we haven't formally selected one. Suggestions are welcome.

[akirmse]

Thanks for the quick response. Agreed, the SCN mechanism looks... unwieldy. What I've seen other API owners do is send out announcements on a low-volume, opt-in mailing list. Another option, not quite as good IMO, is a blog that could be monitored by RSS or similar.

(BTW, the API I was using was https://api.weather.gov/points/XXX,YYY/forecast. It looks like I now need to do two round-trips to get the same result. A really good API design would continue to do this work once, internally, rather than having thousands of identical implementations in clients. Unless there's a direct replacement that I didn't see.)

[scadergit]

It's been a long-time goal to implement API keys, which would inherently make mailing lists more feasible.

[StephenClouse]

BTW, the API I was using was https://api.weather.gov/points/XXX,YYY/forecast. It looks like I now need to do two round-trips to get the same result. A really good API design would continue to do this work once, internally, rather than having thousands of identical implementations in clients.

This is how we implemented it originally, as evidenced by the fact that the old endpoint existed. But our design concerns tend towards cacheability and edge offload (which in turn have a dramatic effect on performance). The thing with gridpoints is that the NWS operates on a 2.5-km grid, so every lat/lon point in that square would be returning the exact same data. The result of that was terabytes of duplicate cached data that was not offloading (because a tiny lat/lon change would become a cache miss) when it easily could. This was not easily solvable as long as lat/lon was the authoritative mapping. This is why we decided to push the point-to-gridpoint mapping lookup onto the consuming client.

[StephenClouse]

Now that our release schedule is un-stuck, we should be able to be more proactive in announcing these changes. This will certainly be one of the forums where we post them.

[akirmse]

Good news on announcing changes in the future.

On this particular API change, I can appreciate the need for caching for reasonable performance. But an extra network round trip for all clients is such an impact to the user experience that I think it's worth spending considerable effort to avoid it. On mobile clients in particular, the extra round trip is always visible to the end user.

I thought of two ways you could avoid the extra round trip and still preserve the cache hit rate:

• Convert the lat/lng to a grid ID inside an off-the-shelf caching proxy that you run, and use the grid ID as the cache key. This might be hard to install at edge caches however.

• Ask clients to do the lat/lng to grid ID conversion internally without a network request, and publish the conversion algorithm. You could add a new API that accepts grid IDs so that you can preserve the existing API for those who don't want to do the conversion and are willing to accept the latency of the additional round trip.

It would be best if the grid IDs used for such an API were stable and global, rather than depending on NWS office IDs, which I presume are subject to change, and are of little interest to most clients. For example, a 2.5 km grid overlaid over your whole service area, where the grid IDs are the coordinates of the NW corner. But even if the grid had to be more complex than this (Maybe different grids for Hawaii and Puerto Rico? Just guessing at possible wrinkles), I think many clients would still be willing to implement it to save the round trip. I imagine that users could provide reference implementations for most programming languages that you could host on your github site, as long as the conversion function is stable over time.

[scadergit]

500ms was always a goal, right now we average 2-3s (outside of the temporary issues noted). We're a very small team and we've optimized the API for the current performance requirements. There are other customers (like ships at sea) that need improved latency as well. It is something we'll continue to work on, but the list of priorities continually evolves.

As an avid hiker, I expect to have no signal and look ahead at the weather, but as more towers extend the range farther out, it would be nice to have updated weather information.

[jerryyhom]

@StephenClouse, thanks for your nice explanations.

Since grids overlap, as you showed above, then when using the points endpoint, which gridpoint is returned?

From the documentation, precisely one gridpoint is returned. I am guessing that overlapping gridpoints from different NWS offices might have different computed data if the sources are different. On the other hand, the source data may be similar enough that the computed data is close enough, so overlapping gridpoints are roughly equivalent. Even if I had a choice of gridpoints, what selection criteria would I use? Does it matter?

[StephenClouse]

Since grids overlap, as you showed above, then when using the points endpoint, which gridpoint is returned?

Well, the correct one :)

If there is more than one gridpoint within range, then we lookup the WFO actually responsible for that point via the AWIPS basemaps.

Despite the grids having overlap, the data should not necessarily be trusted for areas that are not actually within a WFO's area of responsibility. This is why we strictly map lat/lon to gridpoint one-to-one.

[DemiMarie]

@akirmse: Would adding a bounding box to the points response be helpful? @scadergit: Is this something you could provide?

A human user clicking on a map or moving around in a forest is rarely going to request the same location twice, so the ability to cache location ⇒ gridpoint mappings is not very useful. However, being able to cache grid zones could be useful. The returned bounding box does not need to be exact, so long as it is a subset of the actual bounding box. If the new location is in the bounding box of a previously fetched gridpoint, the client can reuse the cached version.

[scadergit]

The current caching solution is to manage high-traffic requests--locations within populated areas that hit the same gridpoints. Wilderness lookups will unlikely take advantage of cache, but that is okay on our end. I'm unsure how you define a zone--are you saying you want to cache the path a hike would take? There's nothing preventing an application from capturing all the grids within a app-specific defined area to store for future use.

[akirmse]

A bounding box would be better than nothing, as I could cache it for a time, though if this assignment is known in advance, I'd much rather fetch the mapping for the whole country periodically and avoid the /points call entirely.

BTW in my app almost all weather requests are for a fixed set of mountaintop locations, so caching locations (say, on small S2 cells) would be of some use.

For what it's worth, I am still seeing timeout failures on a regular basis. I've started using a commercial API outside the US, and it always returns in under 1 second, using a single HTTPS call.

[scadergit]

@JJKraw yes, but it's just the geom for that one grid. I believe @DemiMarie wants to cache multiple grids to avoid future /point requests in a defined area.

[scadergit]

@DemiMarie possibly, would need the correlation id to investigate, as there are a few bugs we are currently squashing.

[DemiMarie]

Correlation ID is 5346067f

[scadergit]

@DemiMarie I searched our logging system and can't find that id. If you keep getting them, please send an email to the helpdesk.

[DemiMarie]

Thanks!