I realized that bot config and context are global variables and when the bot runs and multiple webhooks are received at the same time, it will cause for one of them to override the other one. This is especially bad when the two webhooks are from two different repositories because a lot of functions use context to get repository and owner. Now because we are running the bot as Netlify function on production, as far as I know it runs the whole script as a separate process for every request so that avoid the problem. But if we run the bot as one process like during development then this will cause problems.

As I'm writing this I remember one case where an issue was created with 2 labels so Github sent 3 webhooks, one for issue opened and two for labels. The two webhooks for labels were exactly the same, which I thought was weird at first but in the end I thought this was Github's problem. Now it all makes sense!

Conclusion:
We should avoid using global variables, so context and bot config should be passed down by parameters which I know is more verbose but it's the only solution I can think of right now. If anyone has any suggestions, please comment.