Skip to content

add error handling to weather fetch functions, including cors #3791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from
Open

add error handling to weather fetch functions, including cors #3791

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6ee9a66
add error handling to weather fetch functions, including cors
sdetweil May 26, 2025
b5487c8
fix let vs const
sdetweil May 26, 2025
c0a84c5
Refactor and get tests work again
KristjanESPERANTO Jun 8, 2025
518e986
Merge branch 'develop' into weather_errors
KristjanESPERANTO Jun 9, 2025
96c8853
Merge branch 'develop' into weather_errors
KristjanESPERANTO Jul 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ planned for 2025-07-01
- [calendar] fix fullday event rrule until with timezone offset (#3781)
- [feat] Add rule `no-undef` in config file validation to fix #3785 (#3786)
- [fonts] Fix `roboto.css` to avoid error message `Unknown descriptor 'var(' in @font-face rule.` in firefox console (#3787)
- [weather] add error handling to fetch functions including cors

### Updated

Expand Down
20 changes: 12 additions & 8 deletions js/server_functions.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,25 +41,29 @@ async function cors (req, res) {
url = `invalid url: ${req.url}`;
Log.error(url);
res.send(url);
} else {
url = match[1];

const headersToSend = getHeadersToSend(req.url);
const expectedReceivedHeaders = geExpectedReceivedHeaders(req.url);
return;
}
url = match[1];

Log.log(`cors url: ${url}`);
const response = await fetch(url, { headers: headersToSend });
const headersToSend = getHeadersToSend(req.url);
const expectedReceivedHeaders = geExpectedReceivedHeaders(req.url);
Log.log(`cors url: ${url}`);

const response = await fetch(url, { headers: headersToSend });
if (response.ok) {
for (const header of expectedReceivedHeaders) {
const headerValue = response.headers.get(header);
if (header) res.set(header, headerValue);
}
const data = await response.text();
res.send(data);
} else {
res.status(response.status).json({ message: response.statusText });
}

} catch (error) {
Log.error(error);
res.send(error);
res.status(500).json({ message: error.message });
}
}

Expand Down
24 changes: 15 additions & 9 deletions modules/default/utils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,26 @@ async function performWebRequest (url, type = "json", useCorsProxy = false, requ
requestUrl = url;
request.headers = getHeadersToSend(requestHeaders);
}

const response = await fetch(requestUrl, request);
const data = await response.text();
if (response.ok) {
const data = await response.text();

if (type === "xml") {
return new DOMParser().parseFromString(data, "text/html");
} else {
if (!data || !data.length > 0) return undefined;
if (type === "xml") {
return new DOMParser().parseFromString(data, "text/html");
} else {
if (!data || !data.length > 0) return "null"; //undefined;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return "null" as a string looks unusual. Can you explain why you did this and why there is a comment // undefined?

Copy link
Collaborator Author

@sdetweil sdetweil Jun 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at suggested error recovery, the undefined prior result (comment) is faulty
Leading to a secondary error masking the first (no data)

And null in quotes is the recommended change

I don't understand that (or undefined) either, as none of these are JSON , or error (throw)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and looking at the provider that calls the performWebRequest (here looking at pirateweather)

	fetchWeatherForecast () {
		this.fetchData(this.getUrl())
			.then((data) => {
				if (!data || !data.daily || !data.daily.data.length) {  // undefined or null (no quotes) 
					// No usable data?
					return;  // <----- I think this bypasses the finally, so the provider is waiting forever, hung
				}

				const forecast = this.generateWeatherObjectsFromForecast(data.daily.data);
				this.setWeatherForecast(forecast);
			})
			.catch(function (request) {   // <--- maybe throw is a better choice
				Log.error("Could not load data ... ", request);
			})
			.finally(() => this.updateAvailable());

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the fetchData in the providers layer

	async fetchData (url, type = "json", requestHeaders = undefined, expectedResponseHeaders = undefined) {
		const mockData = this.config.mockData;
		if (mockData) {
			const data = mockData.substring(1, mockData.length - 1);
			return JSON.parse(data);
		}
		const useCorsProxy = typeof this.config.useCorsProxy !== "undefined" && this.config.useCorsProxy;
		return performWebRequest(url, type, useCorsProxy, requestHeaders, expectedResponseHeaders, config.basePath);
	}


const dataResponse = JSON.parse(data);
if (!dataResponse.headers) {
dataResponse.headers = getHeadersFromResponse(expectedResponseHeaders, response);
const dataResponse = JSON.parse(data);
if (!dataResponse.headers) {
dataResponse.headers = getHeadersFromResponse(expectedResponseHeaders, response);
}
return dataResponse;
}
return dataResponse;
} else {
throw new Error(`Response status: ${response.status}`);
}

}

/**
Expand Down
Loading