Move to OpenFaaS golang-http-template - part 2

[alexellis]

Expected Behaviour

I would like to move Derek to the OpenFaaS golang-http-template so that the process can stay alive for longer without re-forking. This would allow access tokens etc to be cached in memory to reduce API requests and will increase memory under load.

It is also a step towards #62 (break out SDK) so that Derek can be run on other platforms or by other processes other than the OpenFaaS watchdog to enable wider use.

Current Behaviour

Each request runs in a new process meaning that panic and os.Exit() are fair-game to appear anywhere in the code. This prevents the above, so part of the work is refactoring any exiting and error handling.

Possible Solution

Template:

golang-http via: https://github.com/openfaas-incubator/golang-http-template

The initial work will be moving to the new template signature, this should be minimal, then from there testing that we never call panic/os.Exit.

The current Dockerfile is unnecessary at this point, since it could be replaced by the original Golang template. https://github.com/alexellis/derek/blob/master/Dockerfile

[cheikhshift]

Would it be possible to keep panics and catch them by deferring a call to function recover?
I searched through the code, there are os.Exit invocations in vendor packages and 4 within Derek

[alexellis]

It would be possible, but I think we could change the code as part of this instead. Can you link to the instances (with line number) please?

[cheikhshift]

These are the calls to os.Exit within Derek, there are more from vendor packages :

https://github.com/alexellis/derek/blob/master/main.go#L34

https://github.com/alexellis/derek/blob/master/main.go#L40

https://github.com/alexellis/derek/blob/master/main.go#L47

https://github.com/alexellis/derek/blob/master/main.go#L55

[alexellis]

Thanks for the research.

Do you want to try doing this issue first? #97

For the line numbers you can click the line on GitHub's UI and then copy/paste the URL.

[cheikhshift]

Sure, I updated my previous comment with links to the corresponding line.

[alexellis]

@cheikhshift are you still working on this? 😄

[cheikhshift]

Of course, here is the OpenFaaS template I wrote a while back : (if ti looks good we can proceed with it. :D ) provider: name: faas # gateway: http://127.0.0.1:80 # can be a remote server functions: derek: handler: ./fn image: alexellis/derek:0.6.4-rc2 lang: go environment: debug: true customers_url: https://raw.githubusercontent.com/alexellis/derek/master/.CUSTOMERS validate_hmac: false validate_customers: true secret_path: /var/openfaas/secrets/ # use /run/secrets/ for older OpenFaaS versions environment_file: - secrets.yml # See secrets.example.yml secrets: - derek-secret-key - derek-private-key The contents in folder 'fn' has one file. (fn is an abbreviation of function) The content of the file is a modified version of the current main.go file. Let me know if the template looks Ok, I'll submit another pull request.

…

On Mon, Jun 3, 2019 at 11:51 AM Alex Ellis ***@***.***> wrote: @cheikhshift <https://github.com/cheikhshift> are you still working on this? 😄 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#96?email_source=notifications&email_token=ABGGN3TVZP3LTK3QJPFK7LDPYUAUNA5CNFSM4F7VIYMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWZFA5Q#issuecomment-498225270>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABGGN3XBM6N4C2CUJTVOEPDPYUAUNANCNFSM4F7VIYMA> .

[alexellis]

Can you propose a PR?

Alex

[cheikhshift]

Sure. Cheikh.

…

On Jun 3, 2019, at 2:46 PM, Alex Ellis ***@***.***> wrote: Can you propose a PR? Alex — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[cheikhshift]

I reviewed the issue, this is what I'm going to work on :

• isolate most of the main.go file (in package root) in a new sub package called handler.
• Modify the current OpenFaaS deployment file to use the Go template.
• We need to keep invocations to panic, as they can modify response code provided by OpenFaaS. Some dependencies make invocations to os.Exit, but as long as derek is not using any deprecated functions, things should be fine.
  PR to follow soon.

Since I've mentioned HTTP responses, OpenFaas should have a helper package, to provide a sort of translation of how OpenFaaS interfaces with HTTP requests. This may reduce friction for users coming from a RESTful API background.

[alexellis]

Thanks for your take on the changes. We have a REST-compatible template called golang-http and golang-middleware both are available via faas-cli template store list and would have the benefit of keeping Derek's process alive between requests.