You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Helps visualize the resource utilisation of Amazon ECS clusters.
3
+
Helps visualize the resource reservation of Amazon ECS clusters.
4
4
5
5
Deploying software as “containers” promises to solve many problems with regards to interoperability of environments, speed to deploy, and cost reduction.
6
6
But understanding where our software lives now becomes more difficult both for development and operations teams.
@@ -16,118 +16,94 @@ Each unique Task Definition is represented as a different color, with the legend
16
16
Each Task will contain one or more containers, the task box shows accumulated reserved memory or CPU for all containers in the Task. ECS Services are not currently represented.
17
17
18
18
19
-
## Configure
19
+
## Configuration
20
20
21
-
Displaying live ECS data requires server-side AWS credentials.
21
+
See [CONFIGURATION](docs/CONFIGURATION.md) for details on server-side configurable options that affect cache entry TTL and AWS API call throttling.
22
22
23
-
### Region
23
+
##Configuring AWS SDK
24
24
25
-
Before running, add a file named ```aws_config.json``` to this project root directory. At a minimum set the default region:
25
+
See [AWS_SDK_CONFIGURATION](docs/AWS_SDK_CONFIGURATION.md) for instructions
26
+
on configuring the AWS SDK for server-side AWS connectivity.
26
27
27
-
```
28
-
{
29
-
"region": "<default-region>"
30
-
}
31
-
```
28
+
## Requirements
29
+
30
+
Node >= 0.12
32
31
33
-
Alternatively, set the environment variable "AWS_REGION" before starting the server.
32
+
## Building and Running
34
33
35
-
### Credentials
34
+
The c3vis server is based on ExpressJS. The client predominantly uses D3.js,
35
+
jQuery and Bootstrap.
36
36
37
-
AWS credentials properties "accessKeyId" and "secretAccessKey" can be added to the aws_config.json file as per https://docs.aws.amazon.com/AWSJavaScriptSDK/guide/node-configuring.html.
37
+
Run the following to build and run the server ("package.json" contains instructions to pre-install required node modules):
38
38
39
-
Otherwise, the credentials will be loaded from the Shared Credentials File or Environment Variables or IAM roles if deployed to an AWS instance.
39
+
```
40
+
npm start
41
+
```
40
42
41
-
When using an IAM role, ensure the role has the following access:
43
+
Now browse to the app at `http://localhost:3000`.
42
44
43
-
* ecs:listContainerInstances
44
-
* ecs:describeContainerInstances
45
-
* ecs:listTasks
46
-
* ecs:describeTasks
47
-
* ecs:describeTaskDefinition
48
-
* ecs:listClusters
49
-
* ec2:describeInstance
45
+
## Testing
50
46
51
-
Sample IAM Inline Policy:
47
+
To run the server-side unit test suite with mocha and chai:
48
+
52
49
```
53
-
{
54
-
"Version": "2012-10-17",
55
-
"Statement": [
56
-
{
57
-
"Effect": "Allow",
58
-
"Action": [
59
-
"ecs:listContainerInstances",
60
-
"ecs:describeContainerInstances",
61
-
"ecs:listTasks",
62
-
"ecs:describeTasks",
63
-
"ecs:describeTaskDefinition",
64
-
"ecs:listClusters",
65
-
"ec2:describeInstances"
66
-
],
67
-
"Resource": [
68
-
"*"
69
-
]
70
-
}
71
-
]
72
-
}
50
+
npm run test
73
51
```
74
52
75
-
**WARNING:** c3vis makes ECS data from the above API calls (including environment variables in task definitions) available to clients/browsers.
76
-
Ensure the c3vis server is available only to users that should have access to this information.
53
+
## Usage
77
54
55
+
### Approach
78
56
79
-
## Requirements
57
+
When a client browser first connects to the c3vis server, the Cluster dropdown will be populated with ECS cluster names for the configured region.
80
58
81
-
Node >= 0.12
59
+
Select from the dropdown to view the resources allocated to that cluster. If no cluster names appear in the dropdown, check the server logs and ensure the correct region is configured (see below).
82
60
83
-
## Building and Running
61
+
The list of clusters and the user's current selection are stored in cookies. Use the ```[refresh list]``` dropdown entry to refresh the list of clusters.
84
62
85
-
Server is based on ExpressJS. Client uses D3.js.
63
+
The Y axis shows total memory or CPU available for the instances. Memory is the default resource type represented. Use the "resourceType" query parameter to toggle between "memory" and "cpu". E.g. ```localhost:3000/?resourceType=cpu```
86
64
87
-
Run the following to build and run the server ("package.json" contains instructions to pre-install required node modules):
65
+
The X axis displays the Private IP Address for each EC2 instance. Right-clicking the IP address shows the context menu with links to browse the instance in the ECS and EC2 consoles.
88
66
89
-
```
90
-
npm start
91
-
```
67
+
### AWS API Call Throttling
92
68
93
-
This will run ```npm install``` and ```node --harmony ./bin/www```
94
-
(NOTE: ```"--harmony"``` is required for ES6 functionality such as Array.find())
69
+
In order to prevent AWS API Rate limiting issues for large clusters, the server:
95
70
96
-
Now browse to the app at `http://localhost:3000`.
71
+
* Introduces a delay between API calls (configurable via `aws.apiDelay` setting)
72
+
* Limits the number of items retrieved per page in `list` and `describe` API calls (configurable via `aws.*PageSize`)
73
+
* Limits the number of asynchronous API calls it makes at a time (configurable via `aws.maxSimultaneous*Calls`)
97
74
98
-
### Usage
75
+
You can increase or decrease each of these settings to suit each environment c3vis is deployed to.
99
76
100
-
When a client browser first connects to the c3vis server the Cluster dropdown will be populated with ECS cluster names for the configured region.
77
+
### Short Polling, Server-Side Caching and Fetch Status
101
78
102
-
Select from the dropdown to view the resources allocated to that cluster. If no cluster names appear in the dropdown, check the server logs and ensure the correct region is configured (see below).
79
+
For each cluster requested, the server caches cluster data in-memory while the client polls the server until the cache is populated.
103
80
104
-
The list of clusters and the user's current selection are stored in cookies. Use the ```[refresh list]``` dropdown entry to refresh the list of clusters.
81
+
For an explanation on how the client polls the server for cluster data and the applicable fetch statuses, see [SHORT_POLLING_FETCH_STATUS](docs/SHORT_POLLING_FETCH_STATUS.md).
105
82
106
-
The Y axis shows total memory or CPU available for the instances. Memory is the default resource type represented. Use the "resourceType" query parameter to toggle between "memory" and "cpu". E.g. ```localhost:3000/?resourceType=cpu```
107
83
108
-
The X axis displays the Private IP Address for each EC2 instance. Right-clicking the IP address shows the context menu with links to browse the instance in the ECS and EC2 consoles.
84
+
## Debugging
109
85
110
86
### Sample Clusters for Testing
111
87
112
88
From the browser, use a ```"?static=true"``` query parameter to have the server return static test data. Useful for testing when server is unable to connect to AWS.
113
89
114
90
Browse to `http://localhost:3000/?static=true`.
115
91
116
-
### Debugging
92
+
### Server Debug Logging
117
93
118
-
Add the following line to server-side Javascript code to add a breakpoint:
94
+
To see all debug log entries:
119
95
120
96
```
121
-
debugger;
97
+
DEBUG=* npm start
122
98
```
123
99
124
-
then run the debugger with:
100
+
To see just API debug log entries:
125
101
126
102
```
127
-
node debug --harmony ./bin/www
103
+
DEBUG=api npm start
128
104
```
129
105
130
-
###Running with Docker
106
+
## Running with Docker
131
107
132
108
Build and tag the image:
133
109
@@ -141,8 +117,14 @@ Run the container: (can remove ```AWS_ACCESS_KEY_ID``` and ```AWS_SECRET_ACCESS_
The c3vis server uses the AWS JavaScript SDK to connect to AWS APIs.
4
+
5
+
As per [Configuring the SDK for JavaScript](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/configuring-the-jssdk.html), the AWS JavaScript SDK will get its configuration from the server's environment.
6
+
7
+
## Provide Explicit AWS SDK Configuration with `aws_config.json` Configuration File
8
+
9
+
AWS SDK configuration can be overridden by providing an `aws_config.json` file (this file location is overridable with `aws.configFile` option, see [CONFIGURATION.md](CONFIGURATION.md)).
10
+
11
+
E.g. to set the region used by c3vis server to `us-east-1`, create an `aws_config.json` file in the root directory with the following:
12
+
13
+
```
14
+
{
15
+
"region": "us-east-1"
16
+
}
17
+
```
18
+
19
+
The contents of this file override all other sources of AWS SDK configuration.
20
+
The settings are applied to the AWS Global Configuration Object using `AWS.config.update()` as per [Using the Global Configuration Object](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/global-config-object.html)
21
+
22
+
## AWS Region
23
+
24
+
As per above section, AWS Region can be provided in local `aws_config.json` file.
25
+
26
+
Otherwise the Region will be configured as per [Setting the AWS Region](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-region.html).
27
+
28
+
## AWS Credentials
29
+
30
+
If using `aws_config.json` file as per above section, you can add AWS credentials properties `accessKeyId` and `secretAccessKey` to the `aws_config.json`
31
+
See [https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-json-file.html](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-json-file.html).
32
+
33
+
*NOTE: Storing credentials in plaintext file is not recommended, especially if there is a risk this file could be committed to version control.*
34
+
35
+
Otherwise, the credentials will be loaded as per priority listed [here](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html).
36
+
37
+
## IAM Role Permissions
38
+
39
+
### EC2 IAM Role Permissions
40
+
41
+
When running c3vis on EC2 instances using an IAM role, ensure the role has the
42
+
following permissions:
43
+
44
+
*`ecs:listContainerInstances`
45
+
*`ecs:describeContainerInstances`
46
+
*`ecs:listTasks`
47
+
*`ecs:describeTasks`
48
+
*`ecs:describeTaskDefinition`
49
+
*`ecs:listClusters`
50
+
*`ec2:describeInstance`
51
+
52
+
Sample IAM Inline Policy:
53
+
```
54
+
{
55
+
"Version": "2012-10-17",
56
+
"Statement": [
57
+
{
58
+
"Effect": "Allow",
59
+
"Action": [
60
+
"ecs:listContainerInstances",
61
+
"ecs:describeContainerInstances",
62
+
"ecs:listTasks",
63
+
"ecs:describeTasks",
64
+
"ecs:describeTaskDefinition",
65
+
"ecs:listClusters",
66
+
"ec2:describeInstances"
67
+
],
68
+
"Resource": [
69
+
"*"
70
+
]
71
+
}
72
+
]
73
+
}
74
+
```
75
+
76
+
### ECS IAM Task Role
77
+
78
+
When running c3vis on an ECS cluster, you can use an ECS Task IAM Role, which
79
+
can be created using the process documented [here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html#create_task_iam_policy_and_role).
80
+
Ensure the IAM Policy has the permissions listed above.
81
+
82
+
## Security Warning
83
+
84
+
**WARNING:** c3vis makes ECS data from the above API calls (including environment variables in task definitions) available to clients/browsers.
85
+
Ensure the c3vis server is available only to users that should have access to this information.
0 commit comments