Skip to content

Commit 376aba4

Browse files
committed
last pieces
1 parent 6a5326a commit 376aba4

File tree

2 files changed

+34
-38
lines changed

2 files changed

+34
-38
lines changed

src/content/docs/aws/services/firehose.md

Lines changed: 32 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,20 @@
11
---
22
title: "Data Firehose"
3-
linkTitle: "Data Firehose"
4-
description: >
5-
Get started with Data Firehose on LocalStack
3+
description: Get started with Data Firehose on LocalStack
64
tags: ["Free"]
75
---
86

9-
{{< callout >}}
7+
:::note
108
This service was formerly called as 'Kinesis Data Firehose'.
11-
{{< /callout >}}
9+
:::
1210

1311
## Introduction
1412

1513
Data Firehose is a service provided by AWS that allows you to extract, transform and load streaming data into various destinations, such as Amazon S3, Amazon Redshift, and Elasticsearch.
1614
With Data Firehose, you can ingest and deliver real-time data from different sources as it automates data delivery, handles buffering and compression, and scales according to the data volume.
1715

1816
LocalStack allows you to use the Data Firehose APIs in your local environment to load and transform real-time data.
19-
The supported APIs are available on our [API coverage page]({{< ref "coverage_firehose" >}}), which provides information on the extent of Data Firehose's integration with LocalStack.
17+
The supported APIs are available on our [API coverage page](), which provides information on the extent of Data Firehose's integration with LocalStack.
2018

2119
## Getting started
2220

@@ -30,9 +28,9 @@ We will demonstrate how to use Firehose to load Kinesis data into Elasticsearch
3028
You can create an Elasticsearch domain using the [`create-elasticsearch-domain`](https://docs.aws.amazon.com/cli/latest/reference/es/create-elasticsearch-domain.html) command.
3129
Execute the following command to create a domain named `es-local`:
3230

33-
{{< command >}}
34-
$ awslocal es create-elasticsearch-domain --domain-name es-local
35-
{{< / command >}}
31+
```bash
32+
awslocal es create-elasticsearch-domain --domain-name es-local
33+
```
3634

3735
Save the value of the `Endpoint` field from the response, as it will be required further down to confirm the setup.
3836

@@ -43,17 +41,17 @@ Now let us create our target S3 bucket and our source Kinesis stream:
4341
Before creating the stream, we need to create an S3 bucket to store our backup data.
4442
You can do this using the [`mb`](https://docs.aws.amazon.com/cli/latest/reference/s3/mb.html) command:
4543

46-
{{< command >}}
47-
$ awslocal s3 mb s3://kinesis-activity-backup-local
48-
{{< / command >}}
44+
```bash
45+
awslocal s3 mb s3://kinesis-activity-backup-local
46+
```
4947

5048
You can now use the [`CreateStream`](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_CreateStream.html) API to create a Kinesis stream named `kinesis-es-local-stream` with two shards:
5149

52-
{{< command >}}
53-
$ awslocal kinesis create-stream \
50+
```bash
51+
awslocal kinesis create-stream \
5452
--stream-name kinesis-es-local-stream \
5553
--shard-count 2
56-
{{< / command >}}
54+
```
5755

5856
### Create a Firehouse delivery stream
5957

@@ -64,20 +62,20 @@ Within the `kinesis-stream-source-configuration`, it is required to specify the
6462
The `elasticsearch-destination-configuration` sets vital parameters, which includes the access role, `DomainARN` of the Elasticsearch domain where you wish to publish, and the settings including the `IndexName` and `TypeName` for the Elasticsearch setup.
6563
Additionally to backup all documents to S3, the `S3BackupMode` parameter is set to `AllDocuments`, which is accompanied by `S3Configuration`.
6664

67-
{{< callout >}}
65+
:::note
6866
Within LocalStack's default configuration, IAM roles remain unverified and no strict validation is applied on ARNs.
6967
However, when operating within the AWS environment, you need to check the access rights of the specified role for the task.
70-
{{< /callout >}}
68+
:::
7169

7270
You can use the [`CreateDeliveryStream`](https://docs.aws.amazon.com/firehose/latest/APIReference/API_CreateDeliveryStream.html) API to create a Firehose delivery stream named `activity-to-elasticsearch-local`:
7371

74-
{{< command >}}
75-
$ awslocal firehose create-delivery-stream \
72+
```bash
73+
awslocal firehose create-delivery-stream \
7674
--delivery-stream-name activity-to-elasticsearch-local \
7775
--delivery-stream-type KinesisStreamAsSource \
7876
--kinesis-stream-source-configuration "KinesisStreamARN=arn:aws:kinesis:us-east-1:000000000000:stream/kinesis-es-local-stream,RoleARN=arn:aws:iam::000000000000:role/Firehose-Reader-Role" \
7977
--elasticsearch-destination-configuration "RoleARN=arn:aws:iam::000000000000:role/Firehose-Reader-Role,DomainARN=arn:aws:es:us-east-1:000000000000:domain/es-local,IndexName=activity,TypeName=activity,S3BackupMode=AllDocuments,S3Configuration={RoleARN=arn:aws:iam::000000000000:role/Firehose-Reader-Role,BucketARN=arn:aws:s3:::kinesis-activity-backup-local}"
80-
{{< / command >}}
78+
```
8179

8280
On successful execution, the command will return the `DeliveryStreamARN` of the created delivery stream:
8381

@@ -93,43 +91,43 @@ Before testing the integration, it's necessary to confirm if the local Elasticse
9391
You can use the [`describe-elasticsearch-domain`](https://docs.aws.amazon.com/cli/latest/reference/es/describe-elasticsearch-domain.html) command to check the status of the Elasticsearch cluster.
9492
Run the following command:
9593

96-
{{< command >}}
97-
$ awslocal es describe-elasticsearch-domain \
94+
```bash
95+
awslocal es describe-elasticsearch-domain \
9896
--domain-name es-local | jq ".DomainStatus.Processing"
99-
{{< / command >}}
97+
```
10098

10199
Once the command returns `false`, you can move forward with data ingestion.
102100
The data can be added to the source Kinesis stream or directly to the Firehose delivery stream.
103101

104102
You can add data to the Kinesis stream using the [`PutRecord`](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html) API.
105103
The following command adds a record to the stream:
106104

107-
{{< command >}}
108-
$ awslocal kinesis put-record \
105+
```bash
106+
awslocal kinesis put-record \
109107
--stream-name kinesis-es-local-stream \
110108
--data '{ "target": "barry" }' \
111109
--partition-key partition
112-
{{< / command >}}
110+
```
113111

114-
{{< callout "tip" >}}
112+
:::note
115113
For users using AWS CLI v2, consider adding `--cli-binary-format raw-in-base64-out` to the command mentioned above.
116-
{{< /callout >}}
114+
:::
117115

118116
You can use the [`PutRecord`](https://docs.aws.amazon.com/firehose/latest/APIReference/API_PutRecord.html) API to add data to the Firehose delivery stream.
119117
The following command adds a record to the stream:
120118

121-
{{< command >}}
122-
$ awslocal firehose put-record \
119+
```bash
120+
awslocal firehose put-record \
123121
--delivery-stream-name activity-to-elasticsearch-local \
124122
--record '{ "Data": "eyJ0YXJnZXQiOiAiSGVsbG8gd29ybGQifQ==" }'
125-
{{< / command >}}
123+
```
126124

127125
To review the entries in Elasticsearch, you can employ [curl](https://curl.se/) for simplicity.
128126
Remember to replace the URL with the `Endpoint` field from the initial `create-elasticsearch-domain` operation.
129127

130-
{{< command >}}
131-
$ curl -s http://es-local.us-east-1.es.localhost.localstack.cloud:443/activity/_search | jq '.hits.hits'
132-
{{< / command >}}
128+
```bash
129+
curl -s http://es-local.us-east-1.es.localhost.localstack.cloud:443/activity/_search | jq '.hits.hits'
130+
```
133131

134132
You will get an output similar to the following:
135133

src/content/docs/aws/services/neptune.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ if __name__ == '__main__':
142142

143143
Amazon Neptune resources with IAM DB authentication enabled require all requests to use AWS Signature Version 4.
144144

145-
When LocalStack starts with [IAM enforcement enabled]({{< ref "/user-guide/security-testing" >}}), the Neptune database checks user permissions before granting access.
145+
When LocalStack starts with [IAM enforcement enabled](/aws/capabilities/security-testing/iam-policy-enforcement), the Neptune database checks user permissions before granting access.
146146
The following Gremlin query actions are available for database engine versions `1.3.2.0` and higher:
147147

148148
```json
@@ -237,9 +237,7 @@ rm -rf <LocalStack Volume>/lib/tinkerpop
237237
The LocalStack Web Application provides a Resource Browser for managing Neptune databases and clusters.
238238
You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Neptune** under the **Database** section.
239239

240-
<img src="neptune-resource-browser.png" alt="Neptune Resource Browser" title="Neptune Resource Browser" width="900" />
241-
<br>
242-
<br>
240+
![Neptune Resource Browser](/images/aws/neptune-resource-browser.png)
243241

244242
The Resource Browser allows you to perform the following actions:
245243

0 commit comments

Comments
 (0)