Memory leak when creating a new Impit instance on each iteration (Node.js)

[rustairov]

Hi! I'm running into a memory-related issue when using the library.

I have a scenario where each iteration needs its own isolated cookie storage:
– the first request obtains cookies,
– subsequent requests in the same iteration must reuse those cookies,
– iterations must not share cookies with each other.

Here’s a minimal reproducible example:

for (let i = 0; i < 1e3; i++) {
  const cookieJar = new CookieJar();
  const impit = new Impit({ cookieJar });

  try {
    const resp = await impit.fetch('http://localhost:3000'); // succeeds or fails depending on server availability
    console.log(Object.values(process.memoryUsage()).join(','));

    // follow-up requests that rely on cookies from the first request
  } catch (err) {
    console.log(Object.values(process.memoryUsage()).join(','));
  }
}

The problem:
Creating a new Impit instance on every iteration causes memory usage to continuously grow until the process runs out of memory. It seems that Impit instances (or internal state) are not being released by GC even after the loop moves on.

My question:
What is the correct approach to using the library so that each iteration can perform several sequential requests with its own cookie jar, but without creating a memory leak?

Is there a recommended pattern for:

• per-iteration cookie isolation,
• reusing some internal resources instead of reinstantiating Impit thousands of times,
• or explicitly cleaning up an Impit instance?

Thanks in advance!

[aleksandr258]

same problem

[barjin]

Hello and thank you for your interest in this project!

The library shouldn't memleak on this example, so that's definitely something for us to look into.

If I understand your request correctly, you might use a custom CookieJar implementation. Impit accepts any object that implements the getCookieSync and setCookie methods, so you can e.g. create a FreezableCookieJar class like this:

import { Impit } from 'impit';
import { CookieJar } from 'tough-cookie';

class FreezableCookieJar extends CookieJar {
    constructor(...args) {
        super(...args);
        this.frozen = false;
    }

    freeze() {
        this.frozen = true;
    }

    unfreeze() {
        this.frozen = false;
    }

    setCookie(...args) {
        if (this.frozen) return;
        return super.setCookie(...args);
    }
}

const cj = new FreezableCookieJar();

const impit = new Impit({
    cookieJar: cj,
})

await impit.fetch('https://example.com'); // sets cookies

cj.freeze(); // disables the setCookie method

// subsequent requests won't store any new cookies
await impit.fetch('https://example.com/2');
await impit.fetch('https://example.com/3');
await impit.fetch('https://example.com/4');
await impit.fetch('https://example.com/5');

A similar implementation can be used to e.g. only let the clients to set some cookie names, preset some cookies manually etc.

As I mentioned above, this is only a workaround, though. We'll continue looking into the memory leak.

Cheers!

[liteCarma]

Hello. I think I've encountered a similar problem, and it's definitely not related to cookies. Perhaps it's worth adding a method to destroy the instance.

import { Impit } from 'impit'
import { setTimeout } from 'node:timers/promises'

async function main() {
  for (let i = 0; i <= 10000; i++) {
    const impit = new Impit({
      browser: 'chrome',
      proxyUrl: '',
      ignoreTlsErrors: true,
    })
    await impit.fetch('https://mail.com/')
    const impitRef = new WeakRef(impit)
    setTimeout(10000).then(() => {
      const oldImpit = impitRef.deref()
      console.log(`Still alive: ${!!oldImpit ? 'Yes' : 'No'}`)
    })
    await setTimeout(1000)
  }
}

main()

Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: No
Still live: Yes
Still live: Yes
...