Support context manager usage of clients and apps to implicitly close token storage and transports (#1326)

Author: sirosen

changelog.d/20251003_154153_sirosen_context_manager_clients_and_apps.rst

@@ -0,0 +1,21 @@
+Added
+-----
+
+- ``GlobusApp`` and SDK client classes now support usage as context managers, and
+  feature a new ``close()`` method to close internal resources.
+  ``close()`` is automatically called on exit. (:pr:`NUMBER`)
+
+  - In support of this, token storages now all feature a ``close()`` method,
+    which does nothing in the default implementation.
+    Previously, only storages with underlying resources to manage featured a
+    ``close()`` method.
+
+  - ``GlobusApp`` will close any token storage via ``close()`` if the token storage
+    was created by the app on init. Explicitly created storages will not be closed
+    and must be explicitly closed via their ``close()`` method.
+
+  - Any class inheriting from ``BaseClient`` features ``close()``, which will
+    close any transport object created during client construction.
+
+  - Transports which are created explicitly will not be closed by their clients,
+    and must be explicitly closed.

changelog.d/20251110_151006_sirosen_context_manager_clients_and_apps.rst

@@ -0,0 +1,5 @@
+Fixed
+-----
+
+- Fixed a resource leak in which a ``GlobusApp`` would create internal client
+  objects and never close the associated transports. (:pr:`NUMBER`)

docs/authorization/globus_app/apps.rst

@@ -82,6 +82,60 @@ The following table provides a comparison of these two options:
     a user be the primary data access actor. In these cases, a ``ClientApp``
     will be rejected and a ``UserApp`` must be used instead.
 
+
+Closing Resources via GlobusApps
+--------------------------------
+
+When used as context managers, ``GlobusApp``\s automatically call their
+``close()`` method on exit.
+
+Closing an app closes the token storage attached to it, unless it was created
+explicitly.
+This covers any token storage created by the app on init, but not those which
+are created and passed in via the config.
+
+For most cases, users are recommended to use the context manager form, and to
+allow ``GlobusApp`` to both create and close the token storage.
+For example,
+
+.. code-block:: python
+
+    from globus_sdk import GlobusAppConfig, UserApp
+
+    # create an app configured to create its own sqlite storage
+    config = GlobusAppConfig(token_storage="sqlite")
+    with UserApp("sample-app", client_id="FILL_IN_HERE", config=config) as app:
+        ...  # any clients, usage, etc.
+
+    # after the context manager, any storage is implicitly closed
+
+
+However, when token storage instances are created by the user, they are not automatically
+closed, and the user is responsible for closing them. For example,
+in the following usage, the user must close the token storage:
+
+.. code-block:: python
+
+    from globus_sdk import GlobusAppConfig, UserApp
+    from globus_sdk.token_storage import SQLiteTokenStorage
+
+    # this token storage is created by the user and will not be closed automatically
+    sql_storage = SQLiteTokenStorage("tokens.sqlite")
+
+    # create an app configured to use this storage
+    config = GlobusAppConfig(token_storage=sql_storage)
+    with UserApp("sample-app", client_id="FILL_IN_HERE", config=config) as app:
+        do_stuff(app)
+
+    # a second app uses the same storage, and shares the resource
+    with UserApp("a-different-app", client_id="OTHER_ID", config=config) as app:
+        do_stuff(app)
+
+    # At this stage, the storage will still be open.
+    # It should be explicitly closed:
+    sql_storage.close()
+
+
 Reference
 ---------
 

docs/core/base_client.rst

@@ -9,8 +9,79 @@ requests, encoding data, and handling potential retries. It also may include an
 optional ``authorizer``, an object responsible for handling token
 authentication for requests.
 
+
+Closing Resources via Clients
+-----------------------------
+
+When used as context managers, clients automatically call their ``close()``
+method on exit.
+
+Closing a client closes the transport object attached to it if the transport was
+created implicitly during init.
+This means a transport passed in from the outside will not be closed, but one
+which was created by the client will be.
+
+For most cases, users are recommended to use the context manager form, and to
+allow clients to both create and close the transport:
+
+.. code-block:: python
+
+    from globus_sdk import SearchClient, UserApp
+
+    with UserApp("sample-app", client_id="FILL_IN_HERE") as app:
+        with SearchClient(app=app) as client:
+            ...  # any usage
+
+    # after the context manager, any transport is implicitly closed
+
+However, if transports are created explicitly, they are not automatically closed,
+and the user becomes responsible for closing them.
+For example, in the following usage, the user must close the transport:
+
+.. code-block:: python
+
+    from globus_sdk import SearchClient, UserApp
+    from globus_sdk.transport import RequestsTransport
+
+    my_transport = RequestsTransport(http_timeout=120.0)
+
+    with UserApp("sample-app", client_id="FILL_IN_HERE") as app:
+        with SearchClient(app=app, transport=my_transport) as client:
+            ...  # any usage
+
+    # At this stage, the transport will still be open.
+    # It should be explicitly closed:
+    my_transport.close()
+
+
+.. note::
+
+    The SDK cannot tell whether or not an explicitly passed transport was bound
+    to a name before it was passed. Therefore, in usage like the following,
+    the transport will not automatically be closed:
+
+    .. code-block:: python
+
+        with SearchClient(app=app, transport=RequestsTransport()) as client:
+            ...
+
+    In order to close the transport in such a case, you must explicitly close it.
+    Since transports are bound to ``client.transport``, the following usage would be a valid
+    resolution:
+
+    .. code-block:: python
+
+        with SearchClient(app=app, transport=RequestsTransport()) as client:
+            ...
+
+        client.transport.close()
+
+
+Reference
+---------
+
 BaseClient
-----------
+^^^^^^^^^^
 
 .. autoclass:: globus_sdk.BaseClient
    :members: scopes, resource_server, attach_globus_app, get, put, post, patch, delete, request

docs/user_guide/getting_started/list_groups_improved.py

@@ -0,0 +1,15 @@
+import globus_sdk
+
+# this is the tutorial client ID
+# replace this string with your ID for production use
+CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
+
+with globus_sdk.UserApp("my-user-app", client_id=CLIENT_ID) as my_app:
+    with globus_sdk.GroupsClient(app=my_app) as groups_client:
+        my_groups = groups_client.get_my_groups()
+
+# print in CSV format
+print("ID,Name,Roles")
+for group in my_groups:
+    roles = "|".join({m["role"] for m in group["my_memberships"]})
+    print(",".join([group["id"], f'"{group["name"]}"', roles]))

docs/user_guide/getting_started/list_groups_with_login.py

@@ -4,18 +4,15 @@
 # replace this string with your ID for production use
 CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
 
-# create your app
-my_app = globus_sdk.UserApp("my-user-app", client_id=CLIENT_ID)
+with globus_sdk.UserApp("my-user-app", client_id=CLIENT_ID) as my_app:
+    with globus_sdk.GroupsClient(app=my_app) as groups_client:
 
-# create a client with your app
-groups_client = globus_sdk.GroupsClient(app=my_app)
+        # Important! The login step needs to happen after the `groups_client` is created
+        # so that the app will know that you need credentials for Globus Groups
+        my_app.login()
 
-# Important! The login step needs to happen after the `groups_client` is created
-# so that the app will know that you need credentials for Globus Groups
-my_app.login()
-
-# call out to the Groups service to get a listing
-my_groups = groups_client.get_my_groups()
+        # call out to the Groups service to get a listing
+        my_groups = groups_client.get_my_groups()
 
 # print in CSV format
 print("ID,Name,Roles")

docs/user_guide/getting_started/minimal_script.rst

@@ -92,11 +92,19 @@ will prompt you to login.
 Summary: Complete Examples
 --------------------------
 
-For ease of use, here are a pair of examples.
+For ease of use, here are a set of examples.
 
 One of them is exactly the same as the tutorial steps above, in a single block.
-The other example includes an explicit login step, so you can control when that
+
+The next is a version of the tutorial which leverages the context manager
+interfaces of the app and client to do cleanup.
+This is slightly more verbose, but such usage is recommended because it ensures
+that network and filesystem resources associated with the client and app are
+properly closed.
+
+The final example includes an explicit login step, so you can control when that
 login flow happens!
+Like the previous example, it uses the context manager style to ensure proper cleanup.
 
 *These examples are complete. They should run without errors "as is".*
 
@@ -108,6 +116,14 @@ login flow happens!
             :caption: ``list_groups.py`` [:download:`download <list_groups.py>`]
             :language: python
 
+    ..  tab-item:: With Context Managers
+
+        This example is the same as the tutorial, but safely cleans up resources.
+
+        .. literalinclude:: list_groups_improved.py
+            :caption: ``list_groups_improved.py`` [:download:`download <list_groups_improved.py>`]
+            :language: python
+
     ..  tab-item:: Explicit ``login()`` Step
 
         This example is very similar to the tutorial, but uses a separate login

docs/user_guide/usage_patterns/data_transfer/create_guest_collection/create_guest_collection_client_owned.py

@@ -4,12 +4,6 @@
 # Confidential Client ID/Secret - <replace these with real client values>
 CONFIDENTIAL_CLIENT_ID = "..."
 CONFIDENTIAL_CLIENT_SECRET = "..."
-CLIENT_APP = ClientApp(
-    "my-simple-client-collection",
-    client_id=CONFIDENTIAL_CLIENT_ID,
-    client_secret=CONFIDENTIAL_CLIENT_SECRET,
-)
-
 
 # Globus Tutorial Collection 1
 # https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
@@ -19,8 +13,16 @@
 
 
 def main():
-    gcs_client = globus_sdk.GCSClient(ENDPOINT_HOSTNAME, app=CLIENT_APP)
+    with ClientApp(
+        "my-simple-client-collection",
+        client_id=CONFIDENTIAL_CLIENT_ID,
+        client_secret=CONFIDENTIAL_CLIENT_SECRET,
+    ) as app:
+        with globus_sdk.GCSClient(ENDPOINT_HOSTNAME, app=app) as gcs_client:
+            create_guest_collection(gcs_client)
+
 
+def create_guest_collection(gcs_client: globus_sdk.GCSClient):
     # Comment out this line if the mapped collection is high assurance
     attach_data_access_scope(gcs_client, MAPPED_COLLECTION_ID)
 

docs/user_guide/usage_patterns/data_transfer/create_guest_collection/create_guest_collection_user_owned.py

@@ -3,7 +3,6 @@
 
 # Tutorial Client ID - <replace this with your own client>
 NATIVE_CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
-USER_APP = UserApp("my-simple-user-collection", client_id=NATIVE_CLIENT_ID)
 
 # Globus Tutorial Collection 1
 # https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
@@ -13,8 +12,12 @@
 
 
 def main():
-    gcs_client = globus_sdk.GCSClient(ENDPOINT_HOSTNAME, app=USER_APP)
+    with UserApp("my-simple-user-collection", client_id=NATIVE_CLIENT_ID) as app:
+        with globus_sdk.GCSClient(ENDPOINT_HOSTNAME, app=app) as client:
+            create_guest_collection(client)
 
+
+def create_guest_collection(gcs_client: globus_sdk.GCSClient):
     # Comment out this line if the mapped collection is high assurance
     attach_data_access_scope(gcs_client, MAPPED_COLLECTION_ID)
 

docs/user_guide/usage_patterns/data_transfer/detecting_data_access/index.rst

@@ -36,16 +36,18 @@ the service:
 
     # Tutorial Client ID - <replace this with your own client>
     NATIVE_CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
-    USER_APP = globus_sdk.UserApp("detect-data-access-example", client_id=NATIVE_CLIENT_ID)
 
     # Globus Tutorial Collection 1
     # https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
     # replace with your own COLLECTION_ID
     COLLECTION_ID = "6c54cade-bde5-45c1-bdea-f4bd71dba2cc"
 
-    transfer_client = globus_sdk.TransferClient(app=USER_APP)
 
-    collection_doc = transfer_client.get_endpoint(COLLECTION_ID)
+    with globus_sdk.UserApp(
+        "detect-data-access-example", client_id=NATIVE_CLIENT_ID
+    ) as app:
+        transfer_client = globus_sdk.TransferClient(app=app)
+        collection_doc = transfer_client.get_endpoint(COLLECTION_ID)
 
 .. note::