How to download and save a file

[GarikFirst]

I'm trying to download a file (tar.gz archive, aproximatly 100 Kb) from https url, and save it. I tried requests and aiohttp, but no matter what I do I get OSError: [Errno 12] ENOMEM

Code snippet how I tried to do this:

    resp = requests.get("https://link_to_my_file")

    with open("targer_file.tar.gz", "wb") as file:
        file.write(resp.content)

What should I do to download (and save) a file?

[GitHubsSilverBullet]

What should I do to download (and save) a file?

You should think about the amount of RAM you have available at that time. ;-)

[GarikFirst]

You are right, but something like:

    url = "https://my_file"
    r = requests.get(url, stream=True)

    with open("file.tar.gz", "wb") as fd:
        for chunk in r.iter_content(1000):
            fd.write(chunk)

Also leads to ENOMEM

[glenn20]

Try something like:

BUFSIZE = 1024

buffer = memoryview(bytearray(BUFSIZE))
resp = requests.get("https://link_to_my_file")
socket = resp.raw
with open("file.tar.gz", "wb") as fd:
    while (n := socket.readinto(buffer)) > 0:
        fd.write(buffer[:n])
resp.close()   # Dont forget to close your connection

This breaks up the download into chunks, allocates a small memory buffer (wrapped in a memoryview object) and reuses that same buffer for every read/write, which dramatically reduces allocated memory and memory fragmentation for large downloads. Adjust the bufsize to tradeoff memory usage vs read/write performance.

[GarikFirst]

It’s working, thank you so much!