Excesive RAM use.

[blogcraft]

Describe the bug
While running httpgenerator using a URL Swagger it runs long with a 100% use of RAM. I have 32 GB of ram and under idle have 50% idle. While running the generator, it used all the rest of the aviable ram and kept working for like 15 minutes.

Support Key: [jxag/3e]

OpenAPI Specifications
Sorry, but it is not public.

[blogcraft]

After 30+ minutes running I had to stop it.

[christianhelle]

@blogcraft thanks for taking the time to report this. I have honestly no idea what's going on but the tool doesn't really do anything fancy. It's pretty much just an advanced text parser that generates more text, asynchronously writes to files on disk in parallel, and once done, the process is killed

I'll play around with a couple of large OpenAPI specifications that I test with on other projects to see if I can reproduce this. It could be the parallel IO operations that is killing your machine. If it takes 30 minutes then it might be because of a dead lock when multiple threads are trying to perform IO operations on the same file.

[blogcraft]

Thanks for the fast reply. I tested with another swagger.json and worked very fast. I will try to run it again avoiding validation in case there is a problem there.

[christianhelle]

@blogcraft I'm working on a timeout feature so it would never run for more than a few minutes. Unless the OpenAPI spec has millions and millions of lines of code (which will most likely introduce an entirely different set of problems...) parsing and generating text based on the input should never take that long

[christianhelle]

I haven't solved the actual cause yet but I did introduce a default timeout of 2 minutes for writing to disk so that users won't have to wait indefinitely

[blogcraft]

I might have found out the issue with the swagger, there seems to be object results that have relationships that go deep down. So, a fix might be limiting how deep an object might go besides the timeout you mentioned.

Example, A GET returns an object A:

ObjectA:
{
  a: string,
  b: string,
  listz: ObjectZ[]
}

ObjectZ:
{
  d: string,
  e: string,
  listy: ObjectY[]
}

ObjectY:
{
  ...
}