Skip to content
This repository was archived by the owner on Feb 12, 2022. It is now read-only.

Commit f2e774c

Browse files
jpeddicordmm318
jpeddicord
authored and
mm318
committed
Initial commit
0 parents  commit f2e774c

26 files changed

+1730
-0
lines changed

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
*Issue #, if available:*
2+
3+
*Description of changes:*
4+
5+
6+
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

‎.travis.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
install:
2+
- git clone https://github.com/ros-industrial/industrial_ci.git .ros_ci
3+
script:
4+
- .ros_ci/travis.sh

‎README.md‎

Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
# lex_node
2+
3+
4+
## Overview
5+
The ROS `lex_node` node enables a robot to comprehend natural language commands by voice or textual input and respond through a set of actions, which an AWS Lex Bot maps to ROS messages. Out of the box this node provides a ROS interface to communicate with a specified Amazon Lex bot (configured via lex_config.yaml) and requires configuration of AWS credentials. The Amazon Lex bot needs to be defined with responses and slots for customer prompts. A set of default slots and mappings are demonstrated in the [sample app] and include actions as “Create <location_name>,” “Go to <location_name>” and “Stop.” Additional guides on configuring bots with are available at [Getting Started with Amazon Lex].
6+
7+
Delivering a voice-enabled customer experience (e.g. “Robot, go to x”) will require dialog facilitation, wake word, and offline processing which are not yet provided by this integration. A wake word would trigger the dialog facilitation node to start recording and send the audio to Amazon Lex, then prompt the user for more information should Amazon Lex require it.
8+
9+
The ROS `lex_node` wraps the [aws-sdk-c++] in a ROS service API.
10+
11+
**Amazon Lex Summary**: Amazon Lex is a service for building conversational interfaces into any application using voice and text. Amazon Lex provides the advanced deep learning functionality of automatic speech recognition (ASR) for converting speech to text, and natural language understanding (NLU) to recognize the intent of the text, to enable you to build applications with highly engaging user experiences and lifelike conversational interactions. With Amazon Lex, the same deep learning technologies that power Amazon Alexa are now available to any developer, enabling you to quickly and easily build sophisticated, natural language, conversational bots (“chatbots”).
12+
13+
### License
14+
The source code is released under an [Apache 2.0].
15+
16+
**Author**: AWS RoboMaker<br/>
17+
**Affiliation**: [Amazon Web Services (AWS)]<br/>
18+
**Maintainer**: AWS RoboMaker, [email protected]
19+
20+
### Supported ROS Distributions
21+
- Kinetic
22+
- Lunar
23+
24+
25+
## Installation
26+
27+
### AWS Credentials
28+
You will need to create an AWS Account and configure the credentials to be able to communicate with AWS services. You may find [AWS Configuration and Credential Files] helpful.
29+
30+
This node requires an IAM User with the following permission policy:
31+
- `AmazonLexRunBotsOnly`
32+
33+
### Building from Source
34+
Create a ROS workspace and a source directory
35+
36+
mkdir -p ~/ros-workspace/src
37+
38+
To build from source, clone the latest version from master branch and compile the package
39+
40+
- Clone the package into the source directory
41+
42+
cd ~/ros-workspace/src
43+
git clone https://github.com/aws-robotics/utils-common.git
44+
git clone https://github.com/aws-robotics/utils-ros1.git
45+
git clone https://github.com/aws-robotics/lex-ros1.git
46+
47+
- Install dependencies
48+
49+
cd ~/ros-workspace && sudo apt-get update
50+
rosdep install --from-paths src --ignore-src -r -y
51+
52+
- Build the packages
53+
54+
cd ~/ros-workspace && colcon build
55+
56+
- Configure ROS library Path
57+
58+
source ~/ros-workspace/install/setup.bash
59+
60+
- Build and run the unit tests
61+
62+
colcon build --packages-select lex_node --cmake-target tests
63+
colcon test --packages-select lex_node && colcon test-result --all
64+
65+
66+
## Launch Files
67+
An example launch file called `sample_application.launch` is provided.
68+
69+
70+
## Usage
71+
72+
### Resource Setup
73+
1. Go to Amazon Lex
74+
2. Create sample bot: BookTrip
75+
3. Select publish, create a new alias
76+
4. Modify the configuration file in `config/sample_configuration.yaml` to reflect the new alias
77+
78+
### Run the node
79+
- **With** launch file using parameters in .yaml format (example provided)
80+
- ROS: `roslaunch lex_node sample_application.launch`
81+
82+
### Send a test voice message
83+
`rosservice call /lex_node/lex_conversation "{content_type: 'text/plain; charset=utf-8', accept_type: 'text/plain; charset=utf-8', text_request: 'make a reservation', audio_request: {data: ''}}"`
84+
85+
### Verify the test voice was received
86+
- Receive response from Amazon Lex and continue conversation
87+
88+
89+
## Configuration File and Parameters
90+
An example configuration file called `sample_configuration.yaml` is provided.
91+
92+
**Client Configuration**
93+
**Namespace**:
94+
95+
| Name | Type |
96+
| --- | --- |
97+
| region | *String* |
98+
| userAgent | *String* |
99+
| endpointOverride | *String* |
100+
| proxyHost | *String* |
101+
| proxyUserName | *String* |
102+
| proxyPassword | *String* |
103+
| caPath | *String* |
104+
| caFile | *String* |
105+
| requestTimeoutMs | *int* |
106+
| connectTimeoutMs | *int* |
107+
| maxConnections | *int* |
108+
| proxyPort | *int* |
109+
| useDualStack | *bool* |
110+
| enableClockSkewAdjustment | *bool* |
111+
| followRedirects | *bool* |
112+
113+
**Amazon Lex Configuration**
114+
**Namespace**:
115+
116+
| Key | Type | Description |
117+
| --- | ---- | ---- |
118+
| user_id | *string* | e.g. “lex_node” |
119+
| bot_name | *string* | e.g. “BookTrip” (corresponds to Amazon Lex bot) |
120+
| bot_alias | *string* | e.g. “Demo” |
121+
122+
123+
## Performance and Benchmark Results
124+
We evaluated the performance of this node by runnning the followning scenario on a Raspberry Pi 3 Model B:
125+
126+
- Launch a baseline graph containing the talker and listener nodes from the [roscpp_tutorials package](https://wiki.ros.org/roscpp_tutorials), plus two additional nodes that collect CPU and memory usage statistics. Allow the nodes to run for 60 seconds.
127+
- Launch the ROS `lex_node` using the launch file `lex_node.launch` as described above. At the same time, make calls to the `/lex_node/lex_conversation` service by running the following script in the background:
128+
129+
```bash
130+
rosservice call /lex_node/set_logger_level "{logger: 'ros.lex_node', level: 'debug'}"
131+
rosservice call /lex_node/lex_conversation "{content_type: 'text/plain; charset=utf-8', accept_type: 'text/plain; charset=utf-8', text_request: 'Make a reservation', audio_request: {data: ''}}" && sleep 1
132+
rosservice call /lex_node/lex_conversation "{content_type: 'text/plain; charset=utf-8', accept_type: 'text/plain; charset=utf-8', text_request: 'Seattle, WA', audio_request: {data: ''}}" && sleep 1
133+
rosservice call /lex_node/lex_conversation "{content_type: 'text/plain; charset=utf-8', accept_type: 'text/plain; charset=utf-8', text_request: 'Tomorrow', audio_request: {data: ''}}" && sleep 1
134+
rosservice call /lex_node/lex_conversation "{content_type: 'text/plain; charset=utf-8', accept_type: 'text/plain; charset=utf-8', text_request: 'Next Monday', audio_request: {data: ''}}" && sleep 1
135+
rosservice call /lex_node/lex_conversation "{content_type: 'text/plain; charset=utf-8', accept_type: 'text/plain; charset=utf-8', text_request: '40', audio_request: {data: ''}}" && sleep 1
136+
rosservice call /lex_node/lex_conversation "{content_type: 'text/plain; charset=utf-8', accept_type: 'text/plain; charset=utf-8', text_request: 'economy', audio_request: {data: ''}}" && sleep 1
137+
rosservice call /lex_node/lex_conversation "{content_type: 'text/plain; charset=utf-8', accept_type: 'text/plain; charset=utf-8', text_request: 'yes', audio_request: {data: ''}}"
138+
```
139+
140+
- Allow the nodes to run for 180 seconds.
141+
- Terminate the ROS `lex_node`, and allow the remaining nodes to run for 60 seconds.
142+
143+
The following graph shows the CPU usage during that scenario. The 1 minute average CPU usage starts at 15% during the launch of the baseline graph, and stabilizes around 9%. When we launch the `lex_node` node around second 85 the 1 minute average CPU increases up to a peak of 27.75%, and stabilizes around 24% while the `lex_node` node serves the service calls. After that the 1 minute average CPU usage is kept around 12% until we stop the `lex_node` node around second 360.
144+
145+
![cpu](wiki/images/cpu.svg)
146+
147+
The following graph shows the memory usage during that scenario. We start with a memory usage of around 253 MB for the baseline graph, that increases to a around 280 MB (+10.67%) after launching the `lex_node` node around second 85, and increases again to a peak of 300 (+18.58% wrt initial value) while the `lex_node` node serves the service calls. After that the memory usage decreases to around 278 MB, and keeps this way until going back to 253 MB after we stop the `lex_node` node.
148+
149+
![memory](wiki/images/memory.svg)
150+
151+
152+
## Node
153+
154+
### lex_node
155+
Enables a robot to comprehend natural language commands by voice or textual input and respond through a set of actions.
156+
157+
#### Services
158+
**Topic**: ~/lex_conversation
159+
160+
#### AudioTextConversation
161+
**Request**:
162+
163+
| Key | Type | Description |
164+
| --- | ---- | ----------- |
165+
| content_type | *string* | The input data type to request Amazon Lex |
166+
| accept_type | *string* | The Amazon Lex output data type desired |
167+
| text_request | *string* | Input text data for Lex |
168+
| audio_request | *uint8[]* | Common audio msg format, input audio data for Lex |
169+
170+
**Response**:
171+
172+
| Key | Type | Description |
173+
| --- | ---- | ----------- |
174+
|text_response | *string* | Output text from Lex, if accept type was text |
175+
|audio_response | *uint8[]* | Output audio data from Lex, if accept type was audio |
176+
|slots | *KeyValuePair[]*| Slots returned from Lex |
177+
|intent_name | *string* | The intent Amazon Lex is attempting to fulfill |
178+
|message_format_type | *string* | Format of output data from Lex |
179+
|dialog_state | *string* | Amazon Lex internal dialog_state |
180+
181+
#### Subscribed Topics
182+
None
183+
184+
#### Published Topics
185+
None
186+
187+
188+
## Bugs & Feature Requests
189+
Please contact the team directly if you would like to request a feature.
190+
191+
Please report bugs in [Issue Tracker].
192+
193+
194+
[Amazon Web Services (AWS)]: https://aws.amazon.com/
195+
[Apache 2.0]: https://aws.amazon.com/apache-2-0/
196+
[AWS Configuration and Credential Files]: https://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html
197+
[aws-sdk-c++]: https://github.com/aws/aws-sdk-cpp
198+
[Getting Started with Amazon Lex]: https://docs.aws.amazon.com/lex/latest/dg/getting-started.html
199+
[Issue Tracker]: https://github.com/aws-robotics/lex-ros1/issues
200+
[ROS]: http://www.ros.org
201+
[sample app]: https://github.com/aws/aws-robomaker-sample-application-voiceinteraction

‎lex_common_msgs/CMakeLists.txt‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
cmake_minimum_required(VERSION 2.8.3)
2+
project(lex_common_msgs)
3+
4+
find_package(catkin REQUIRED COMPONENTS
5+
roscpp
6+
std_msgs
7+
audio_common_msgs
8+
message_generation
9+
)
10+
11+
add_message_files(
12+
DIRECTORY msg
13+
FILES
14+
KeyValue.msg
15+
)
16+
17+
add_service_files(
18+
DIRECTORY srv
19+
FILES
20+
AudioTextConversation.srv
21+
)
22+
23+
# add_action_files(
24+
# FILES
25+
# Action1.action
26+
# Action2.action
27+
# )
28+
29+
generate_messages(
30+
DEPENDENCIES
31+
std_msgs
32+
audio_common_msgs
33+
)
34+
35+
catkin_package(
36+
CATKIN_DEPENDS message_runtime
37+
)

0 commit comments

Comments
 (0)