Skip to content

feat: Edit the requests configuration pool_maxsize variable when initializing client #521

@lautarovc

Description

@lautarovc
  • Algolia Client Version: 2.4.0
  • Language Version: Python 3.9.1

Description

I'm replicating a dataset (60k records) with 100 threads (Python's builtin ThreadExecutor) at a time. And I find that from time to time I keep getting these warnings from the 'requests' library: "Connection pool is full, discarding connection: xxxxxxxx.algolia.net".

The ability to edit the pool_maxsize configuration variable when initializing the search client would solve this kind of issues, perhaps even allowing to edit other requests configurations.

Steps To Reproduce

Initialize a ThreadExecutor of 100 max workers.
Create a function that uses 'save_object' to store a record in a predefined index.
Loop through thousands of records, for each record:
-Add the function that saves the record with the record data to the executor

You should get the warning every ~1000 records.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions