Troubleshooting FAQ

[autoSteve]

Q: I have a Solcast API limit of 50 calls. Why is the integration now limiting me to 10?

Can't happen now, given Solcast removed an API call. To answer a question with a question, is the API quota set correctly in the integration configuration? If not, then set it to 50.

[autoSteve]

Q: You asked for DEBUG logs to be provided when raising issues. How do I get these?

We can't usually do anything without them, and it's almost always the first question that will be asked: "Could you provide debug logs, please?"

When set for debug you see all kinds of fascinating stuff about what's going on under the covers. In configuration.yaml:

 logger:
   default: warn
   logs:
     custom_components.solcast_solar: debug

(Make the default info/warn/whatever, as we don't care. We just want the debug goodness.)

Reviewing logs is quite simple, and can be done from the UI. Go to Settings | System | Logs, where "condensed" logs are shown by default. Select the three dots at the top right of screen and select "Show full logs".

You can't filter for just the Solcast integration in the UI, so it might be a good idea to download the log and filter it by another method (for example the *nix utility less "&/" command, or Notepad++ with the Linefilter2 plugin.)

[autoSteve]

Q: Is submitting debug logging going to expose my API key or location to the world?

Yeah, nah. Definitely not now. All sensitive information, including API keys, and your home address via latitude/longitude are now redacted. Make sure you're on the latest version.

[autoSteve]

Q: What polls to Solcast happen, when do they happen, and are they important?

When the integration starts for the very first time, several important things happen, and these involve Solcast API calls that are generally metered. Continued use of API calls also occurs.

This FAQ post is way longer than it should be for a mere three API calls, but there be nuances depending on circumstance. The TL>DR? Getting sites data does not use API call quota. Getting a forecast, or a set of estimated actuals does.

1. The rooftop sites information is gathered for each of the solcast API keys specified.

This is super important information, and the integration cannot function with out this. The return includes the rooftop ID(s), which are used in subsequent calls, plus other data that isn't used except for populating sensor attributes, like location, azimuth and panel tilt.

This call happens when you are first setting up the integration, and also on every re-load. The "first set up" call is used to verify that your API key is good, and also that you've got sites configured. It also occurs on each load just in case you've changed settings at solcast.com and then re-load the integration.

For each re-load if the API call does not work for some reason, then this integration utilises a cache that will recall the data from the last successful call. If the cache doesn't exist yet or has been deleted then the integration won't work, and it will continuously restart until this call succeeds. But do note that if this is your first attempt at setting up the integration and the call fails (i.e. Solcast not available), then you'll be hit with a "Do not pass Go" scenario, and must just follow the instruction: Try again later. You've almost certainly hit a busy API time window. It's not us. It's them... So, try again later. 😅 Five or ten minutes should do.

This call will not use up any precious API call quota, no matter how many times it is called.

2. When "estimated actuals" is not yet available.

This call happens ideally once, and only for a new install or if the solcast.json cache file is deleted (which is an action that can be requested via Developer tools, or by directly deleting the file and re-starting).

It can also be called should the integration have been sitting disabled/failed for over a week, where past data gaps would be seen. (The estimated actuals are used to fill gaps where possible.)

This occurs for each rooftop ID, so if you have two Solcast sites defined, then two calls are made.

This will use up API quota for each site defined, then on top of that usage a forecast update will occur using more quota.

3. When a forecast update is requested.

Auto-update is enabled, or an automation is created by you in Home Assistant to trigger how often solar forecasts are gathered, and when this triggers the service solcast_solar.update_forecasts it will update all of the rooftop IDs for all of the accounts.

This will use up API quota, and if you have two sites configured for an API key then it'll use up two calls for that key.

Should this call not be successful, then it will be re-tried ten times. (A failure almost always does not use quota. It may if the failure happens due to a bug, but I can't recall that ever happening.) The retry mechanism is designed using a back-off mechanism that will retry at delays of 15, 30, 45, 60 and so on seconds, plus a random number of seconds between zero and fifteen for each retry. You'll see this activity in the log as warnings if it happens.

If all retries are exhausted, then a 429/Solcast too busy error will be logged.

But don't panic and raise an issue. It's almost certainly them, not you or us, and the next forecast acquisition will likely be successful.

4. When "estimated actuals" are updated just past midnight, or when requested to be updated using an action.

This will use up API quota for each site defined if the option to get estimated actuals is enabled. Updates occur by default within fifteen minutes of the midnight local time roll-over, or when requested by an action call.

That's all the API calls there are!

Sometimes the API gets so swamped with requests that it asks users to retry. This is the well seen HTTP 429 response where quota has not yet been reached. It's not an error as such, but more a "we heard you, but go away we're busy, try later" notification. Paying users generally never hit this. Un-paying hobbyist users sometimes often, and fair enough. I think Solcast are super generous to offer such a brilliant (but limited) service for us for free.

This integration does its level best to cooperate with Solcast, and retry in a sensible manner.

Q: A follow-up question: If I restart the integration will it use API quota?

The integration has a cache of the last successful forecast call response data and the sites data. The sites are loaded on startup if requesting it from Solcast fails. Then the forecast history loads. This does not use API quota. Forecast is only requested, and API usage incremented when auto-update fetches, or you ask the integration to do so, normally with an automation.

So the answer is definitely no. But this becomes a definite maybe as of v4.2.5.

If you have auto-update enabled in v4.2.5+ then some strange things can seem to happen. If you restored Home Assistant from a backup that pre-dated the last auto fetch then the integration will initiate a fetch because stale data. API call(s) used.

And if you re-started HA immediately before an auto-update was scheduled to fire then that update will fire on integration start. API call(s) will be used, but they would have been used anyway, just weren't previously.

So depending on circumstance, the v4.2.5+ answer is possibly, but probably not.

And a final "what the???" API use scenario: If the integration had been in a failed state that has caused forecast history to be aged out beyond one week then "estimated actuals" will be retrieved from Solcast to cover the gap. This is done to support integration scenarios that rely on recent history, so API calls used to get history. (But it will likely not be an issue as the integration had not been running and using quota for forecast updates...)

That is way too many words to describe that lot, but I trust it has explained every scenario.

[autoSteve]

Q: Why does the Solcast integration sometimes take a long time to start?

The only thing that could cause this to happen when using the latest (possibly beta) release is if the forecast cache file solcast.json failed to load, as well as the Solcast API failed to return all site details.

This would be super rare.

The Solcast integration should load super quick every time.

If it doesn't, and you cant work out why, then please raise an issue including much log.

If this is the very first time that the integration is being loaded for a new installation, then it is also possible that initial startup occurs at precisely the same moment that the Solcast API happens to be really busy. If this is the case then the integration will fail to load, but will then repeatedly attempt a start until things work out. You can see this happening either in the logs, or by opening the integration at Settings | Devices & Services | Integrations.

[autoSteve]

Q: Why have my historical forecasts disappeared from the energy dashboard? I now only see 10/14 days!

At some point, your /config/solcast.json file has gone missing, and was recreated. This contains the history.

First ask yourself, what use are historic forecasts to me anyway? A dashed line that extends back as far as since this integration was first installed is really only visually pleasing, and not really of value.

What temperature was forecast on the 3rd of December 2021, and was it right? Who cares? We know the actual answer now. Solcast would care about improving forecast accuracy, but I'm pretty sure they would not use history to do so. They would compare the predictions of present and proposed forecast models against actuals over time.

The "good" news is that these forecasts will be retained for a couple of years from here on, so your dashed line will get longer, even if it is of almost zero value.

Or do you really want to fix it?

I hope you have a backup from the day when the history vanished. You do back up, right?

The fix involves "merging" the contents of two solcast.json files, which is not as simple as just concatenating the files.

Inside the json structure there is a forecasts key for each rooftop site, which holds an array. What you need to do is get the older values from backup for this array, and carefully (making sure there is the required comma between array elements) insert these forecast elements into the current file. Do this for each rooftop ID, then restart the Solcast integration. (Having duplicates of the timestamped entries won't hurt anything, and they will be cleaned up.) Please, take a backup of solcast.json before attempting this...

(A footnote. I believe I know why this happened, so v4.1.4 includes a change that should prevent it in future. It hasn't just been BJReplay versions that have done this, as folks have encountered, but we're trying to stop it from ever happening again!)

[autoSteve]

Q: Why are certain sensors Watt, while others are Watt-hour or kilo-Watt-hour? Shouldn't these be the same? Why?

The power sensors, with a unit of measurement of Watt represent an instantaneous forecast power at a point in time. Given Solcast forecasts in half-hourly increments these can be thought of as an average power that is expected to be generated for a period (or the value expected half way through each interval).

All values received from Solcast are instantaneous power, or Watts.

The values for Watt-hour/kWh are calculated by the integration from the power numbers, and are power over time, or energy. An example of this is expected solar production for the remainder of the day. For these, the period averages are summed, and then divided by two because the unit is for a whole hour, yet intervals are half-hourly. For some of these, like remaining for the day, a portion of the calculated first period is used because some sensors are updated every five minutes.

@ProphetOfDoom drew up a great annotated representation of an actual forecast chart overlaid with the underlying values that had been received from Solcast.

[autoSteve]

Q: I get a timeout connecting to api.solcast.com.au!!! What the heck is happening? (...raises issue...)

Honestly, it's not us, but you.

This kind of thing can cause you to tear your hair out, but we might be able to put it back. A timeout is a timeout. We got nothing, so we timeout. Period. We got nothing, so we can't go on. But...

You have to check your networking. It may seem right, but it's not. It may seem solid but...

Try a curl from your Home Assistant server first. (This will not use up API call quota.)

curl -X GET -H 'content-type: application/json' https://api.solcast.com.au/rooftop_sites?api_key=YOURAPIKEYHERE

Do you get an instant, and pleasing response of your sites data? (Or even a 429/Solcast too busy) Then, great! Move on. Because more networking bear traps can be laid... It is possible for the command line curl to receive an IPv4 address, and not the address that Home Assistant actually gets from making the same DNS query.

WT?

Check for IPv6 weirdness. Is IPv6 enabled in Home Assistant? If yes, then triple check that HA can actually talk IPv6 to the Internet... If in doubt, then disable IPv6 in the HA network config. Or triple check your router config. This IPv6 stuff is new and scary, but please get your head around it, or disable it if you don't get it.

If that's not the problem, and curl works, then I've got nothing without knowing more. But it's a network issue... Try a more generic networking issue Google search relating to Home Assistant.

[autoSteve]

Q: Does the integration cope with daylight savings time / Summer time transitions?

It does now.

If it is logging odd things for you in debug level logs, and you're getting multiple "Sunday" forecasts (if that's your day of transition) then you need to upgrade to at least v4.2.5.

The transition to daylight time results in Solcast varying the number of half-hourly forecast intervals for the day of transition. When transitioning to daylight time there will be only 46 intervals, and not the usual 48. This is because 2AM will no longer exist for that day. When transitioning from daylight time we get a sleep-in, and there are two 2AMs and a total of 50 intervals.

The integration was messing up the UTC time of period start and end, and using a fixed number of 48 intervals. Now it does not.

[autoSteve]

Q: I just restored Home Assistant from backup, and when it started the Solcast integration updated the forecast! Why?

Auto-update is enabled. The backup that you restored from was prior to the last auto update.

The integration records the date and time of the latest update attempt in solcast.json. On start, it calculates the auto-update intervals, plus the date and time of the most recent auto-update. It then compares the most recent auto-update time with that recorded in solcast.json, and if the forecast cache is out of date, or "stale" then it will do a forecast update.

It has no way of knowing that you have restored from backup.

This is an unusual situation to arise, so we have no plans to alter the integration behaviour. At worst, there will be one instance of API quota exhaustion on update for the day before the next UTC midnight reset of the used count happens.

As an aside, the reason that it does this check is to cover a far more likely scenario. Re-starting Home Assistant.

If HA gets restarted just before a scheduled auto-update is going to happen then that update will be cancelled. If the check for stale forecast data were not done on start then that update would be missed entirely.

So short-term pain on restore from backup. Long-term gain on having auto-update operate reliably for you.

Don't like the behaviour? Send feedback in a discussion, and revert to using an HA automation to update the forecast.

Another aside.

There was an alternate implementation considered, but disregarded by the way. An integration can hold up the restart until it has completed everything it needs to do, so our integration could do that should an update be imminent. This was initially disregarded quickly because it is super-annoying when HA takes a long time to restart for whatever reason. The integration queues pending updates up to five minutes ahead of time, and that would be a super-unreasonable time to wait.

And should a super-annoyingly long restart happen right at the time that auto-update should have done its thing (for no reason to do with the integration), if the update hadn't even been queued yet then the stale forecast check will have that eventuality covered too.

But that alternate idea was also well discarded after futher thought.

A stale check will also work for a VM 'reset' event, or a power failure missing the forecast update instead of a clean shutdown.

Stale check for the win.

[autoSteve]

Q: I've just had a shiny new PV string attached to my inverter, and I've gone to Solcast, added the new rooftop site details, but it's not being included in the forecast results. What's up?

There is almost certainly a simple fix.

The integration loads a list of all rooftop sites from Solcast on startup. It does not attempt to load these until another startup, because that would just add API load to the Solcast service. They don't like that, and tend to respond with 429/Too busy responses when heaps of calls come in from hobbyists, so we like to keep the number of calls to just the essential, and not do a 'get sites' call at every forecast update fetch.

Restart the integration, and your new rooftop will almost certainly be found.

But... Potentially further complicating things for you, it is just possible that a restart will not load the new site details. Should a 429/Too busy be hit at precicely the same moment that you restart to load the new site then the integration will move on, preferring to load cached sites instead. The logs are your friend here, because a 429 gets logged as a warning to tell you that this has happened.

Ugh. Be patient, and persistent if needed.

Restarting near an on-the-fifteen-minute boundary of the hour particularly in the European morning could be to blame. So yeah, try, and try again. It will pick up the new site eventually (but only when restarted).

Q: I've just got lots of 429 errors reported, and I'm not getting forecasts. Should I raise an issue, or continue one of the long running discussions?

or

Q: I'm trying to set up (or re-set up) the integration, and I'm getting 429 errors and can't get any further. What's happening?

or

Solcast's API status page at https://status.solcast.com/ says that the Legacy Rooftop Site API is Operational, but I'm getting 429 errors. What should I do?

As the Solcast API Status page says: Don't agree with what's reported here? Contact us at support@solcast.com.

This integration reports 429 errors returned by the Solcast Legacy Rooftop Site API as received. If that's what Solcast is sending us, that's what we report.

The vast majority of times that this has occurred, it is because someone (not necessarily an integration user, and not necessarily a rooftop hobbyist user) is hammering Solcast servers thousands of times an hour with an out-of-control process.

If the Solcast team aren't yet aware of it (outside of normal Australian business hours) they may not have had a chance to respond and block that process, so as the message on their website says, please, in the first instance, politely (since you're using a free service), ask them if there are any issues, and provide them with as much information as possible.

For example:

• Last successful update 1:30pm AEST (GMT+10).
• Three failed attempts since (10 retries per attempt over a fifteen-minute period)
• 429 responses received on each call to https://api.solcast.com.au/rooftop_sites/

By the way, the 10 retries per attempt over a fifteen-minute period is exactly how the integration works.

Q: I would like to add a second array but I have a message that says Your hobbyist account is limited to the creation of -1 Home PV arrays within 1km of each other.

As advised by the super helpful @billy-solcast (please don't tag him on GitHub for support): This is an issue on our backend. For any future users, if you could please just email through to support@solcast.com we'll fix it up, which will allow you to create the second site next time you log in.

What you should see is:

[autoSteve]

Q: My forecast always updates at midday. Why is the forecast for the morning never updated by the integration, and only the afternoon?

Well, it's a forecast.

The past is the past, and is now not a forecast, so it stands to reason that the past will not be updated.

Usually.

If one changes dampening factors then these may update historical forecast records as far back as the start of day, but never beyond. That is as far as the integration goes with messing with the past.

When midnight rolls around then recent history of estimated actuals will be updated (should the option be set to get estimated actuals). These values are modelled by Solcast based on high resolution satellite images and air clarity data, and likely snow cover of panels if that's a thing for you. And if you have the option set to show estimated actuals on the Energy dashboard then that is what you will see, with those values replacing past forecasted values.

Follow-up Q: So why don't you get estimated actuals at every forecast update so that I can be less confused?

We'd love to. However, it takes an API call that is limited in number. You likely have ten calls per day, used by both forecast update and estimated actual update, and updating estimated actuals at every forecast update would be "expensive" from an API usage point of view.

We could add an option to do so (ask for an enhancement with justification if this is what you want), but the vast majority of the user base want more forecast updates, and not to look at the past but the future.