incomplete content

[newtester805]

Hello,
I have a program that submit a request as follows.

import asyncio
import aiohttp
import time
import socket

import aiohttp
from aiohttp import ClientTimeout, http

ua = "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.13; ko; rv:1.9.1b2) Gecko/20081201 Firefox/60.0"
url = 'https://www.amazon.co.uk/dp/B0BMB7FRW5/ref=redir_mobile_desktop?_encoding=UTF8&aaxitk=100033181b4989fa0ccd72702c260c84&content-id=amzn1.sym.efc92a24-7584-4cbf-b786-4f3d4cab43a0%3Aamzn1.sym.efc92a24-7584-4cbf-b786-4f3d4cab43a0&hsa_cr_id=0&pd_rd_plhdr=t&pd_rd_r=5128fd46-f526-4e05-be1f-2551afd80fd2&pd_rd_w=GpN3X&pd_rd_wg=nQTCm&qid=1677399934&ref_=sbx_be_s_sparkle_lsi4d_asin_0_title&sr=1-1-e0fa1fdd-d857-4087-adda-5bd576b25987'

async def get():
    async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(ssl=False,
                                                         force_close=True,
                                                         family=socket.AF_INET,
                                                         ttl_dns_cache=300),
                                         timeout=ClientTimeout(total=None, sock_connect=5, sock_read=5),
                                         trust_env=True, version=http.HttpVersion11) as s:
                resp = await s.request('GET', url,
                                raise_for_status=True,
                                headers={'User-Agent': ua},
                                ssl=False) 
    return resp

async def main():
    resp = await get()
    print('resp size=', resp.content.total_bytes)
    with open('test.html', 'wb') as f:
        i=0
        #async for data in resp.content.iter_chunked(1024):
        async for data in resp.content.iter_any():
           print(i, len(data))
           i += 1
           f.write(data)
           print(f.tell())
    print('here...')



asyncio.run(main())

The problem is that the loop that write the content to file (async for data in resp.content.iter_any()) never finishes. I can not imagine what can be the issue and how to fix it.
Any help will be really appreciated!

[Dreamsorcerer]

async with aiohttp.ClientSession

This is a context manager, handling the client session. When you exit that block, the session is closed.

By the second line of your main() function, both the request and the session have been closed. Therefore, it is not expected to be able to read from the response in the rest of the code.

Please see the quickstart and other examples for the correct way to make a request.

[Dreamsorcerer]

Tip: If you really want to wrap it all into a function like that, try turning that function into a context manager itself, yielding the resp object:
https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager

Though, you should really be using a session across your application, rather than creating a new one on every function call.

[newtester805]

Thank you very much @Dreamsorcerer. I was making incorrect assumptions about the context manager.