Skip to content

Commit 2fa12ad

Browse files
committed
Moved to new folder
1 parent ae772de commit 2fa12ad

15 files changed

+551
-345
lines changed

.doc_gen/metadata/s3-directory-buckets_metadata.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -413,6 +413,17 @@ s3-directory-buckets_Scenario_ExpressBasics:
413413
snippet_tags:
414414
- php.example_code.s3.ExpressBasics
415415
- php.example_code.s3.service.S3Service
416+
Python:
417+
versions:
418+
- sdk_version: 3
419+
github: python/example_code/s3-directory-buckets/
420+
sdkguide:
421+
excerpts:
422+
- description: >
423+
Run a scenario demonstrating the basics of &S3; directory buckets
424+
and S3 Express One Zone.
425+
snippet_tags:
426+
- python.example_code.s3.s3_express_basics
416427
services:
417428
s3-directory-buckets: {CreateBucket, CopyObject, GetObject, PutObject, ListObjects, DeleteObject, DeleteBucket}
418429

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# S3 Directory Buckets code examples for the SDK for Python
2+
3+
## Overview
4+
5+
Shows how to use the AWS SDK for Python (Boto3) to work with Amazon S3 Directory Buckets.
6+
7+
<!--custom.overview.start-->
8+
<!--custom.overview.end-->
9+
10+
_S3 Directory Buckets are designed to store data within a single AWS Zone. Directory buckets organize data hierarchically into directories, providing a structure similar to a file system._
11+
12+
## ⚠ Important
13+
14+
* Running this code might result in charges to your AWS account. For more details, see [AWS Pricing](https://aws.amazon.com/pricing/) and [Free Tier](https://aws.amazon.com/free/).
15+
* Running the tests might result in charges to your AWS account.
16+
* We recommend that you grant your code least privilege. At most, grant only the minimum permissions required to perform the task. For more information, see [Grant least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege).
17+
* This code is not tested in every AWS Region. For more information, see [AWS Regional Services](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services).
18+
19+
<!--custom.important.start-->
20+
<!--custom.important.end-->
21+
22+
## Code examples
23+
24+
### Prerequisites
25+
26+
For prerequisites, see the [README](../../README.md#Prerequisites) in the `python` folder.
27+
28+
Install the packages required by these examples by running the following in a virtual environment:
29+
30+
```
31+
python -m pip install -r requirements.txt
32+
```
33+
34+
<!--custom.prerequisites.start-->
35+
<!--custom.prerequisites.end-->
36+
37+
### Basics
38+
39+
Code examples that show you how to perform the essential operations within a service.
40+
41+
- [Learn the basics](s3_express_getting_started.py)
42+
43+
44+
<!--custom.examples.start-->
45+
<!--custom.examples.end-->
46+
47+
## Run the examples
48+
49+
### Instructions
50+
51+
52+
<!--custom.instructions.start-->
53+
<!--custom.instructions.end-->
54+
55+
56+
#### Learn the basics
57+
58+
This example shows you how to do the following:
59+
60+
- Set up a VPC and VPC Endpoint.
61+
- Set up the Policies, Roles, and User to work with S3 directory buckets and the S3 Express One Zone storage class.
62+
- Create two S3 Clients.
63+
- Create two buckets.
64+
- Create an object and copy it over.
65+
- Demonstrate performance difference.
66+
- Populate the buckets to show the lexicographical difference.
67+
- Prompt the user to see if they want to clean up the resources.
68+
69+
<!--custom.basic_prereqs.s3-directory-buckets_Scenario_ExpressBasics.start-->
70+
<!--custom.basic_prereqs.s3-directory-buckets_Scenario_ExpressBasics.end-->
71+
72+
Start the example by running the following at a command prompt:
73+
74+
```
75+
python s3_express_getting_started.py
76+
```
77+
78+
79+
<!--custom.basics.s3-directory-buckets_Scenario_ExpressBasics.start-->
80+
<!--custom.basics.s3-directory-buckets_Scenario_ExpressBasics.end-->
81+
82+
83+
### Tests
84+
85+
⚠ Running tests might result in charges to your AWS account.
86+
87+
88+
To find instructions for running these tests, see the [README](../../README.md#Tests)
89+
in the `python` folder.
90+
91+
92+
93+
<!--custom.tests.start-->
94+
<!--custom.tests.end-->
95+
96+
## Additional resources
97+
98+
- [S3 Directory Buckets User Guide](https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html)
99+
- [S3 Directory Buckets API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)
100+
- [SDK for Python S3 Directory Buckets reference](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3-directory-buckets.html)
101+
102+
<!--custom.resources.start-->
103+
<!--custom.resources.end-->
104+
105+
---
106+
107+
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
108+
109+
SPDX-License-Identifier: Apache-2.0
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
boto3>=1.35.49
2+
pytest>=7.2.1
3+
requests>=2.28.2

python/example_code/s3/s3_basics/s3_express_getting_started.py renamed to python/example_code/s3-directory-buckets/s3_express_getting_started.py

Lines changed: 48 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
import os
77
import uuid
88
import time
9+
import argparse
910
from boto3.resources.base import ServiceResource
1011
from boto3 import resource
1112
from boto3 import client
@@ -16,10 +17,9 @@
1617
script_dir = os.path.dirname(os.path.abspath(__file__))
1718

1819
# Add relative path to include demo_tools in this code example without need for setup.
19-
sys.path.append(os.path.join(script_dir, "../../.."))
20+
sys.path.append(os.path.join(script_dir, "../.."))
2021
import demo_tools.question as q
2122

22-
no_art = False # 'no_art' suppresses 'art' to improve accessibility.
2323
logger = logging.getLogger(__name__)
2424

2525

@@ -48,12 +48,11 @@ def __init__(
4848
cloud_formation_resource: ServiceResource,
4949
ec2_client: client,
5050
iam_client: client,
51-
region: str = None,
5251
):
5352
self.cloud_formation_resource = cloud_formation_resource
5453
self.ec2_client = ec2_client
5554
self.iam_client = iam_client
56-
self.region = region if region else ec2_client.meta.region_name
55+
self.region = ec2_client.meta.region_name
5756
self.stack = None
5857
self.vpc_id = None
5958
self.vpc_endpoint_id = None
@@ -93,7 +92,7 @@ def s3_express_scenario(self):
9392
self.demonstrate_performance(bucket_object)
9493

9594
# Populate the buckets to show the lexicographical difference between regular and express buckets.
96-
self.show_lexigraphical_differences(bucket_object)
95+
self.show_lexicographical_differences(bucket_object)
9796

9897
print("")
9998
print("That's it for our tour of the basic operations for S3 Express One Zone.")
@@ -104,7 +103,11 @@ def s3_express_scenario(self):
104103
):
105104
self.cleanup()
106105

107-
def create_vpc_and_users(self):
106+
def create_vpc_and_users(self) -> None:
107+
"""
108+
Optionally create a VPC.
109+
Create two IAM users, one with S3 Express One Zone permissions and one without.
110+
"""
108111
# Configure a gateway VPC endpoint. This is the recommended method to allow S3 Express One Zone traffic without
109112
# the need to pass through an internet gateway or NAT device.
110113
print("")
@@ -148,7 +151,14 @@ def create_vpc_and_users(self):
148151
raise ValueError(error_string)
149152
return express_user_name, regular_user_name
150153

151-
def setup_clients_and_buckets(self, express_user_name, regular_user_name):
154+
def setup_clients_and_buckets(
155+
self, express_user_name: str, regular_user_name: str
156+
) -> None:
157+
"""
158+
Set up two S3 clients, one regular and one express, and two buckets, one regular and one express.
159+
:param express_user_name: The name of the user with S3 Express permissions.
160+
:param regular_user_name: The name of the user with regular S3 permissions.
161+
"""
152162
regular_credentials = self.create_access_key(regular_user_name)
153163
express_credentials = self.create_access_key(express_user_name)
154164
# 3. Create an additional client using the credentials with S3 Express permissions.
@@ -220,7 +230,10 @@ def setup_clients_and_buckets(self, express_user_name, regular_user_name):
220230
print("Great! Both buckets were created.")
221231
press_enter_to_continue()
222232

223-
def create_session_and_add_objects(self):
233+
def create_session_and_add_objects(self) -> None:
234+
"""
235+
Create a session for the express S3 client and add objects to the buckets.
236+
"""
224237
print("")
225238
print("5. Create an object and copy it over.")
226239
print(
@@ -260,7 +273,11 @@ def create_session_and_add_objects(self):
260273
press_enter_to_continue()
261274
return bucket_object
262275

263-
def demonstrate_performance(self, bucket_object):
276+
def demonstrate_performance(self, bucket_object: str) -> None:
277+
"""
278+
Demonstrate performance differences between regular and Directory buckets.
279+
:param bucket_object: The name of the object to download from each bucket.
280+
"""
264281
print("")
265282
print("6. Demonstrate performance difference.")
266283
print(
@@ -308,7 +325,12 @@ def demonstrate_performance(self, bucket_object):
308325
)
309326
press_enter_to_continue()
310327

311-
def show_lexigraphical_differences(self, bucket_object):
328+
def show_lexicographical_differences(self, bucket_object: str) -> None:
329+
"""
330+
Show the lexicographical difference between Directory buckets and regular buckets.
331+
This is done by creating a few objects in each bucket and listing them to show the difference.
332+
:param bucket_object: The object to use for the listing operations.
333+
"""
312334
print("")
313335
print("7. Populate the buckets to show the lexicographical difference.")
314336
print(
@@ -376,7 +398,7 @@ def show_lexigraphical_differences(self, bucket_object):
376398
)
377399
press_enter_to_continue()
378400

379-
def cleanup(self):
401+
def cleanup(self) -> None:
380402
"""
381403
Delete resources created by this scenario.
382404
"""
@@ -728,7 +750,7 @@ def setup_vpc(self):
728750
)
729751
raise
730752

731-
def tear_done_vpc(self):
753+
def tear_done_vpc(self) -> None:
732754
if self.vpc_endpoint_id is not None:
733755
try:
734756
self.ec2_client.delete_vpc_endpoints(
@@ -753,16 +775,22 @@ def tear_done_vpc(self):
753775
self.vpc_id,
754776
client_error.response["Error"]["Message"],
755777
)
756-
778+
# snippet-end:[python.example_code.s3.s3_express_basics]
757779

758780
if __name__ == "__main__":
781+
parser = argparse.ArgumentParser(
782+
description="Run S3 Express getting started scenario."
783+
)
784+
parser.add_argument(
785+
"--no-art",
786+
action="store_true",
787+
help="accessibility setting that suppresses art in the console output.",
788+
)
789+
args = parser.parse_args()
790+
no_art = args.no_art
791+
759792
s3_express_scenario = None
760-
my_s3_client = client("s3")
761-
# delete all directory buckets
762-
bucket_list = my_s3_client.list_directory_buckets()
763-
for bucket in bucket_list["Buckets"]:
764-
print(f"Deleting bucket {bucket['Name']}")
765-
S3ExpressScenario.delete_bucket_and_objects(my_s3_client, bucket["Name"])
793+
766794
try:
767795
a_cloud_formation_resource = resource("cloudformation")
768796
an_ec2_client = client("ec2")
@@ -784,4 +812,4 @@ def tear_done_vpc(self):
784812
if s3_express_scenario is not None:
785813
s3_express_scenario.cleanup()
786814

787-
# snippet-end:[python.example_code.s3.s3_express_basics]
815+
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
"""
5+
Contains common test fixtures used to run unit tests.
6+
"""
7+
8+
import sys
9+
10+
import boto3
11+
import pytest
12+
import os
13+
14+
15+
script_dir = os.path.dirname(os.path.abspath(__file__))
16+
17+
# Add relative path to include SchedulerWrapper.
18+
sys.path.append(script_dir)
19+
sys.path.append(os.path.dirname(script_dir))
20+
import s3_express_getting_started
21+
22+
# Add relative path to include demo_tools in this code example without need for setup.
23+
sys.path.append(os.path.join(script_dir, "../.."))
24+
25+
from test_tools.fixtures.common import *
26+
27+
28+
class ScenarioData:
29+
def __init__(
30+
self,
31+
ec2_client,
32+
s3_client,
33+
iam_client,
34+
cloud_formation_resource,
35+
ec2_stubber,
36+
s3_stubber,
37+
iam_stubber,
38+
39+
cloud_formation_stubber,
40+
region,
41+
):
42+
self.ec2_client = ec2_client
43+
self.s3_client = s3_client
44+
self.iam_client = iam_client
45+
self.cloud_formation_resource = cloud_formation_resource
46+
self.ec2_stubber = ec2_stubber
47+
self.s3_stubber = s3_stubber
48+
self.iam_stubber = iam_stubber
49+
self.cloud_formation_stubber = cloud_formation_stubber
50+
self.region = region
51+
self.scenario = s3_express_getting_started.S3ExpressScenario(
52+
cloud_formation_resource=self.cloud_formation_resource,
53+
ec2_client = self.ec2_client,
54+
iam_client = self.iam_client,
55+
)
56+
s3_express_getting_started.use_press_enter_to_continue = False
57+
58+
59+
@pytest.fixture
60+
def scenario_data(make_stubber):
61+
cloud_formation_resource = boto3.resource("cloudformation")
62+
cloud_formation_stubber = make_stubber(cloud_formation_resource.meta.client)
63+
region = "us-east-1"
64+
ec2_client = boto3.client("ec2", region_name=region)
65+
ec2_stubber = make_stubber(ec2_client)
66+
67+
iam_client = boto3.client("iam")
68+
iam_stubber = make_stubber(iam_client)
69+
70+
s3_client = boto3.client("s3")
71+
s3_stubber = make_stubber(s3_client)
72+
return ScenarioData(
73+
ec2_client = ec2_client,
74+
s3_client = s3_client,
75+
iam_client = iam_client,
76+
cloud_formation_resource = cloud_formation_resource,
77+
ec2_stubber = ec2_stubber,
78+
s3_stubber = s3_stubber,
79+
iam_stubber = iam_stubber,
80+
cloud_formation_stubber = cloud_formation_stubber,
81+
region = region
82+
)

0 commit comments

Comments
 (0)