Skip to content

Commit f2352f2

Browse files
committed
revamp mwaa
1 parent 04426f7 commit f2352f2

File tree

1 file changed

+21
-25
lines changed
  • src/content/docs/aws/services

1 file changed

+21
-25
lines changed

src/content/docs/aws/services/mwaa.md

Lines changed: 21 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
11
---
22
title: "Managed Workflows for Apache Airflow (MWAA)"
3-
linkTitle: "Managed Workflows for Apache Airflow (MWAA)"
4-
description: >
5-
Get started with Managed Workflows for Apache Airflow (MWAA) on LocalStack
3+
description: Get started with Managed Workflows for Apache Airflow (MWAA) on LocalStack
64
tags: ["Ultimate"]
75
---
86

@@ -12,7 +10,7 @@ Managed Workflows for Apache Airflow (MWAA) is a fully managed service by AWS th
1210
MWAA leverages the familiar Airflow features and integrations while integrating with S3, Glue, Redshift, Lambda, and other AWS services to build data pipelines and orchestrate data processing workflows in the cloud.
1311

1412
LocalStack allows you to use the MWAA APIs in your local environment to allow the setup and operation of data pipelines.
15-
The supported APIs are available on the [API coverage page]({{< ref "coverage_mwaa" >}}).
13+
The supported APIs are available on the [API coverage page]().
1614

1715
## Getting started
1816

@@ -26,34 +24,34 @@ We will demonstrate how to create an Airflow environment and access the Airflow
2624
Create a S3 bucket that will be used for Airflow resources.
2725
Run the following command to create a bucket using the [`mb`](https://docs.aws.amazon.com/cli/latest/reference/s3/mb.html) command.
2826

29-
{{< command >}}
30-
$ awslocal s3 mb s3://my-mwaa-bucket
31-
{{< /command >}}
27+
```bash
28+
awslocal s3 mb s3://my-mwaa-bucket
29+
```
3230

3331
### Create an Airflow environment
3432

3533
You can now create an Airflow environment, using the [`CreateEnvironment`](https://docs.aws.amazon.com/mwaa/latest/API/API_CreateEnvironment.html) API.
3634
Run the following command, by specifying the bucket ARN we created earlier:
3735

38-
{{< command >}}
39-
$ awslocal mwaa create-environment --dag-s3-path /dags \
36+
```bash
37+
awslocal mwaa create-environment --dag-s3-path /dags \
4038
--execution-role-arn arn:aws:iam::000000000000:role/airflow-role \
4139
--network-configuration {} \
4240
--source-bucket-arn arn:aws:s3:::my-mwaa-bucket \
4341
--airflow-version 2.10.1 \
4442
--airflow-configuration-options agent.code=007,agent.name=bond \
4543
--name my-mwaa-env
46-
{{< /command >}}
44+
```
4745

4846
### Access the Airflow UI
4947

5048
The Airflow UI can be accessed via the URL in the `WebserverUrl` attribute of the response of the `GetEnvironment` operation.
5149
The username and password are always set to `localstack`.
5250

53-
{{< command >}}
54-
$ awslocal mwaa get-environment --name my-mwaa-env --query Environment.WebserverUrl
51+
```bash
52+
awslocal mwaa get-environment --name my-mwaa-env --query Environment.WebserverUrl
5553
"http://localhost.localstack.cloud:4510"
56-
{{< /command >}}
54+
```
5755

5856
LocalStack also prints this information in the logs:
5957

@@ -91,9 +89,9 @@ Just upload your DAGs to the designated S3 bucket path, configured by the `DagS3
9189

9290
For example, the command below uploads a sample DAG named `sample_dag.py` to your S3 bucket named `my-mwaa-bucket`:
9391

94-
{{< command >}}
95-
$ awslocal s3 cp sample_dag.py s3://my-mwaa-bucket/dags
96-
{{< /command >}}
92+
```bash
93+
awslocal s3 cp sample_dag.py s3://my-mwaa-bucket/dags
94+
```
9795

9896
LocalStack syncs new and changed objects in the S3 bucket to the Airflow container every 30 seconds.
9997
The polling interval can be changed using the [`MWAA_S3_POLL_INTERVAL`]({{< ref "configuration#mwaa" >}}) config option.
@@ -105,9 +103,9 @@ LocalStack seamlessly supports plugins packaged according to [AWS specifications
105103

106104
To integrate your custom plugins into the MWAA environment, upload the packaged `plugins.zip` file to the designated S3 bucket path:
107105

108-
{{< command >}}
109-
$ awslocal s3 cp plugins.zip s3://my-mwaa-bucket/plugins.zip
110-
{{< /command >}}
106+
```bash
107+
awslocal s3 cp plugins.zip s3://my-mwaa-bucket/plugins.zip
108+
```
111109

112110
## Installing Python dependencies
113111

@@ -124,9 +122,9 @@ botocore==1.20.54
124122
Once you have your `requirements.txt` file ready, upload it to the designated S3 bucket, configured for use by the MWAA environment.
125123
Make sure to upload the file to `/requirements.txt` in the bucket:
126124

127-
{{< command >}}
128-
$ awslocal s3 cp requirements.txt s3://my-mwaa-bucket/requirements.txt
129-
{{< /command >}}
125+
```bash
126+
awslocal s3 cp requirements.txt s3://my-mwaa-bucket/requirements.txt
127+
```
130128

131129
After the upload, the environment will be automatically updated, and your Apache Airflow setup will be equipped with the new dependencies.
132130
It is important to note that, unlike [AWS](https://docs.aws.amazon.com/mwaa/latest/userguide/connections-packages.html), LocalStack does not install any provider packages by default.
@@ -143,9 +141,7 @@ This information must be explicitly passed in operators, hooks, and sensors.
143141
The LocalStack Web Application provides a Resource Browser for managing MWAA Environments.
144142
You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **MWAA** under the **App Integration** section.
145143

146-
<p>
147-
<img src="mwaa-resource-browser.png" alt="MWAA Resource Browser" title="MWAA Resource Browser" width="900" />
148-
</p>
144+
![MWAA Resource Browser](/images/aws/mwaa-resource-browser.png)
149145

150146
The Resource Browser allows you to perform the following actions:
151147

0 commit comments

Comments
 (0)