Module not found when I require custom module from my github

[espruino-discuss]

Posted at 2021-12-26 by Mark_M

Hi
I am following this doc https://www.espruino.com/Modules#espruino-modules
Created my github https://github.com/markmal/BangleApps/tree/master/modules/
I've put this into web IDE Module URL: https://github.com/markmal/BangleApps/tree/master/modules/|http://www.espruino.com/modules/

Also I tried direct link, but still getting the error

>require("https://github.com/markmal/BangleApps/blob/master/modules/AT6558.js");
Uncaught Error: Module https://github.com/markmal/BangleApps/tree/master/modules/AT6558.js not found
 at line 1 col 78
...e/master/modules/AT6558.js");

Also tried link that returns plain text

>require("https://raw.githubusercontent.com/markmal/BangleApps/master/modules/AT6558.js");
Uncaught Error: Module https://raw.githubusercontent.com/markmal/BangleApps/master/modules/AT6558.js not found
 at line 1 col 88
...s/master/modules/AT6558.js");

What am I doing wrong?

[espruino-discuss]

Posted at 2021-12-26 by Poolitzer

Hi. Just to confirm to you, you do nothing wrong \o/.

The issue steams from GitHub not executing a proper CORS response, which in turn stops the browser from executing the request the JS code asks for.

There are a couple things you can do to fix this issue: You can install an extension which enters the proper header everywhere (the link there didn't work, see this one . I did get some errors when loading a module that way, and Im not sure if they are expected or a result of the plugin getting in the way, but at least it loads something.

There is also the possibility of installing and running the IDE "locally" with NPM. See here. This shouldn't be limited by the missing CORS response as the browser is, I havent been able to try this yet though.

In the end, the only way to fix this is the other methods mentioned in the above ReadMe or asking GitHub to implement the proper CORS headers. Maybe they also only have a bug with their http endpoint and it will fix itself at some point? If other people can confirm it used to work, this is a not so unlikely scenario.

[espruino-discuss]

Posted at 2021-12-26 by Mark_M

Thank you Poolitzer.
I tried to install the local IDE. But got issues. I've installed the recommended Nodejs 16. Then nw and espruino-web-ide. It runs but has 2 major issues. 1 - it works weirdly with window manager (xfce), 2 - it does not see bluetooth ports.

I will try other methods too.

[espruino-discuss]

Posted at 2021-12-26 by Poolitzer

For the second issue, did you do sudo setcap cap_net_raw+eip $(eval readlink -f $(which node)), as the instructions state?

[espruino-discuss]

Posted at 2021-12-26 by Mark_M

Yes, I've done everything what was in the manual.

I've made it simpler. I run Apache. Configured it for CORS. Copied modules into /var/www/html/mark/BangleApps/modules/
Added this to IDE Modules path http://localhost/mark/BangleApps/modules/
Now IDE accepts require, but still saying AT6558 not defined.

>require("AT6558");
=function (a) { ... }
>gnss = new AT6558("TST");
Uncaught ReferenceError: "AT6558" is not defined
 at line 1 col 12
gnss = new AT6558("TST");

To fix the above issue I've added to my module.

exports.connect = function(id) {
    return new AT6558(id);
};

and use it from program following way

gnss = require("AT6558").connect("TST");

Adding to this, it required also to set default expiration to 0 in apache settings. Otherwise Web IDE keeps using old file from browser cache despite it was already modified.

[espruino-discuss]

Posted at 2021-12-26 by myownself

The reason it didn't work without the connect function (the not defined error) is you weren't assigning the result of the require anywhere. You wanted

var AT6558 = require("AT6558");

[gfwilliams]

Posted at 2022-01-05 by @gfwilliams

One issue with your initial code might be that you're entering it on the left-hand side of the IDE? That's a direct connection to the device - if you upload from the right-hand side then the code gets parsed and when external URLs are mentioned they are fetched and uploaded automatically.

[espruino-discuss]

Posted at 2022-01-05 by Poolitzer

No the CORS issue happens from the right side for me.

[gfwilliams]

Posted at 2022-01-05 by @gfwilliams

Odd...

r = require("https://github.com/markmal/BangleApps/blob/master/modules/AT6558.js");

Doesn't work, but:

r = require("https://raw.githubusercontent.com/markmal/BangleApps/master/modules/AT6558.js");

is fine for me

[espruino-discuss]

Posted at 2022-01-05 by Poolitzer

That is because raw.githubusercontent.com does send a proper CORS header, access-control-allow-origin: * to be precise. See MDN.

[gfwilliams]

Posted at 2022-01-06 by @gfwilliams

Did you try it? As I said above, it works.

The Web IDE is able to detect github-style URLs and although raw.githubusercontent.com doesn't work, it uses the GitHub API to request the file in a way which DOES work cross origin

[espruino-discuss]

Posted at 2022-01-06 by Poolitzer

Did you try it? As I said above, it works.

Yes, I agree that raw.githubusercontent.com works, while github.com doesn't (just tested again).

The Web IDE is able to detect github-style URLs and although raw.githubusercontent.com doesn't work, it uses the GitHub API to request the file in a way which DOES work cross origin

Hm. I don't think I can agree with either half of that sentence. If I open raw.githubusercontent.com in my browser and check the request headers, access-control-allow-origin: * is passed, which mean it should allow all cross origin requests.

If I check the requests made by the Web IDE, it sends a fetch request to either raw.githubusercontent.com or github.com, depending ofc which you enter in the require call. I don't see a GitHub API request.

I can be totally wrong though, never done any JS work. But from the previous two comments from us, I would have made a change to the web IDE just checking if the require is a GitHub.com URL and change it to a raw one, as a hotfix. And then I would have opened an Issue at GitHub asking why they dont pass a cross origin header in GitHub.com and if they have a good reason not to, I guess the hotfix will be a permanent fix.

[espruino-discuss]

Posted at 2022-01-06 by Mark_M

Thank you guys. Good to know raw.githubusercontent.com can be a workaround.
But for me my local web server works perfectly well, I can control CORS and expiration.

[gfwilliams]

Posted at 2022-01-07 by @gfwilliams

I just looked up the module: https://github.com/espruino/EspruinoTools/blob/master/plugins/getGitHub.js

So actually I misremembered - it should be attempting to rewrite github URLs into a raw.github... form as you suggest... So it looks like for some reason the first option should work too, but something is broken there