feat: Add Python 3.11 support (#2414)

* Add Python 3.11 to build scripts

* Update lambda layer docs

* Add Python 3.11 to install docs

* Add Python 3.11 version badge

* Add Python 3.11 to black as well

* Minor - Upgrade to arrow 12

Signed-off-by: Abdel Jaidi <[email protected]>

* Minor - layers.rst is automatically created at release

Signed-off-by: Abdel Jaidi <[email protected]>

---------

Signed-off-by: Abdel Jaidi <[email protected]>
Co-authored-by: Abdel Jaidi <[email protected]>

Author: moralesl

README.md

@@ -12,7 +12,7 @@ Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, Q
 > An [AWS Professional Service](https://aws.amazon.com/professional-services/) open source initiative | [email protected]
 
 [![Release](https://img.shields.io/badge/3.2.1-brightgreen.svg)](https://pypi.org/project/awswrangler/)
-[![Python Version](https://img.shields.io/badge/python-3.8%20%7C%203.8%20%7C%203.9%20%7C%203.10-brightgreen.svg)](https://anaconda.org/conda-forge/awswrangler)
+[![Python Version](https://img.shields.io/badge/python-3.8%20%7C%203.8%20%7C%203.9%20%7C%203.10%20%7C%203.11-brightgreen.svg)](https://anaconda.org/conda-forge/awswrangler)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 

building/build-lambda-layers.sh

@@ -37,3 +37,11 @@ docker run \
   --rm \
   awswrangler-build-py310 \
   build-lambda-layer.sh "${VERSION}-py3.10${ARCH_SUFFIX}" "ninja-build"
+
+# Python 3.11
+docker run \
+  --volume "$DIR_NAME":/aws-sdk-pandas/ \
+  --workdir /aws-sdk-pandas/building/lambda \
+  --rm \
+  awswrangler-build-py311 \
+  build-lambda-layer.sh "${VERSION}-py3.11${ARCH_SUFFIX}" "ninja-build"

building/lambda/build-docker-images.sh

@@ -27,4 +27,11 @@ docker build \
   --build-arg base_image=public.ecr.aws/lambda/python:3.10 \
   .
 
+# Python 3.11
+docker build \
+  --pull \
+  --tag awswrangler-build-py311 \
+  --build-arg base_image=public.ecr.aws/lambda/python:3.11 \
+  .
+
 rm -rf pyproject.toml poetry.lock

building/lambda/build-lambda-layer.sh

@@ -15,7 +15,7 @@ export LD_LIBRARY_PATH=$(pwd)/dist/lib:$LD_LIBRARY_PATH
 
 git clone \
   --depth 1 \
-  --branch apache-arrow-10.0.0 \
+  --branch apache-arrow-12.0.0 \
   --single-branch \
   https://github.com/apache/arrow.git
 

docs/source/install.rst

@@ -1,7 +1,7 @@
 Install
 =======
 
-**AWS SDK for pandas** runs on Python ``3.8``, ``3.9`` and ``3.10``,
+**AWS SDK for pandas** runs on Python ``3.8``, ``3.9``, ``3.10`` and ``3.11``,
 and on several platforms (AWS Lambda, AWS Glue Python Shell, EMR, EC2,
 on-premises, Amazon SageMaker, local, etc).
 
@@ -119,6 +119,10 @@ This option provides the ability to use semantic versions (i.e. library version)
    * - aws-sdk-pandas-layer-py3-10
      - arn:aws:serverlessrepo:us-east-1:336392948345:applications/aws-sdk-pandas-layer-py3-10
      - Layer for ``Python 3.10.x`` runtimes
+   * - aws-sdk-pandas-layer-py3-11
+     - arn:aws:serverlessrepo:us-east-1:336392948345:applications/aws-sdk-pandas-layer-py3-11
+     - Layer for ``Python 3.11.x`` runtimes
+
 
 Here is an example of how to create and use the AWS SDK for pandas Lambda layer in your CDK app:
 

pyproject.toml

@@ -124,7 +124,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.black]
 line-length = 120
-target-version = ["py38", "py39", "py310"]
+target-version = ["py38", "py39", "py310", "py311"]
 extend_exclude = '''
 /(
     \.eggs

tox.ini

@@ -20,7 +20,7 @@ commands =
               --cov-fail-under={env:COV_FAIL_UNDER} \
               --junitxml=test-reports/junit.xml --log-file=test-reports/logs.txt tests/unit
 
-[testenv:py{37,38,39,310}-distributed]
+[testenv:py{37,38,39,310,311}-distributed]
 passenv =
        AWS_PROFILE
        AWS_DEFAULT_REGION