GH-45305: [CI][Python] Temporarily avoid newer boto3 version (#45311)

Until Minio gets fixed.

### Rationale for this change

### What changes are included in this PR?

### Are these changes tested?

### Are there any user-facing changes?

* GitHub Issue: #45305

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>

Author: pitrou

ci/appveyor-cpp-setup.bat

@@ -70,7 +70,6 @@ conda create -n arrow ^
   "ninja" ^
   "nomkl" ^
   "pandas" ^
-  "fsspec" ^
   "python=%PYTHON%" ^
   || exit /B
 conda list -n arrow

ci/conda_env_python.txt

@@ -15,8 +15,11 @@
 # specific language governing permissions and limitations
 # under the License.
 
-# don't add pandas here, because it is not a mandatory test dependency
-boto3  # not a direct dependency of s3fs, but needed for our s3fs fixture
+# Don't add pandas here, because it is not a mandatory test dependency
+
+# Not a direct dependency of s3fs, but needed for our s3fs fixture
+# (temporary upper bound because of GH-45305)
+boto3<1.36
 cffi
 cython>=3
 cloudpickle