Skip to content

Commit 0178998

Browse files
committed
Add microservices traffic graph
1 parent a18d1a7 commit 0178998

File tree

9 files changed

+257
-48
lines changed

9 files changed

+257
-48
lines changed

Chronos Slide Deck Notes.rtf

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
{\rtf1\ansi\ansicpg1252\cocoartf1561\cocoasubrtf610
2+
{\fonttbl\f0\fnil\fcharset0 Georgia;}
3+
{\colortbl;\red255\green255\blue255;}
4+
{\*\expandedcolortbl;;}
5+
\margl1011\margr1011\margb1445\margt1011\vieww14300\viewh16300\viewkind1\viewscale113
6+
\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\fi722\sl288\slmult1\pardirnatural\partightenfactor0
7+
8+
\f0\b\fs24 \cf0 about Chronos
9+
\b0 \
10+
- a microservice network monitoring tool\
11+
- designed to meet the needs of companies and developers who are breaking down their monolithic system architectures into decoupled distributed services, an system design known as microservices. Microservices applications are deployed as a family of independent services, each with a specific function to perform. When you break down a traditional monolithic system into manageable sections, it\'92s not always an easy task. You still need the individual pieces to communicate with one another and work in tandem and it requires more real-time attention and proactive monitoring. And at every one of these points of communication is a potential point of failure. Tracking an application requires correlating data from all these different services. So a common problem of breaking down a large-scale application can produce a series of problems, such as complex communication conflicts with the disparate teams that are now working on their own piece of the pie. \
12+
- Many companies utilize microservices, even though they are huge corporations such as Amazon, which turned to a microservice architecture back in 2001 and has their cloud monitoring tool called Amazon X-Ray, Twitter, and Netflix, which has their own in-house microservice tool.\
13+
\
14+
15+
\b how it works\
16+
17+
\b0 - install an NPM packages into each of your services, provide a database connection string, and you\'92re able to look at how your application is faring at any given moment.\
18+
- it uses a process called context propagation, where a wrapper is placed around incoming HTTP requests and when any subsequent requests are triggered, a correlation ID is given to that specific request as a request header. Thereby, that same correlation ID will connect multiple disparate endpoints of an application, and so if there\'92s any problems, a team of devs working on one area of an application can trace the origin of any issue, even if it originated from another team\'92s scope of purview. These endpoints converse with each other and return the operational status of a service and an indication of its ability to connect to downstream dependent services as an HTTP status code with JSON data, creating an aggregated web of responses that allows for better communication and accessibility.\
19+
- for example, Amazon does one-click ordering and they are the best at this. Let\'92s say, for instance, it usually takes 3ms to complete an order. If, for some reason it now takes 5ms, Amazon\'92s health monitoring tool would alert them to this issue, and because of the way context propagation works, they would be able to drill down their endpoints across various services and departments and directly assess the situation wherever it may occur in the application. \
20+
\
21+
22+
\b what you get\
23+
24+
\b0 - by accessing systems information API, you\'92re able to get data pertinent to the health of the server on which Chronos operates on. this is useful because if a service is running a bit behind or something unusual is going on, we\'92ll know exactly where it\'92s happening and whether or not this is a consistent problem or something unique.\
25+
- this data connects to your very own database, you can use either mongo or postgresql, which will keep your private data private.\
26+
- you\'92re able to see a dashboard display that provides data over time on distinctive metrics such as speed and latency tracking (intervals can be set from either every sec to every week), process monitoring, as well as memory usage. It should look like a graph display that records performance and health over time. It\'92ll provide much-needed context for an engineer to assess the situation handily. A key detail is that system performance is not binary; systems are not on or off. They can operate in degraded state that impacts performance, often leading to failure. A graph system provides context to a degraded state before total failure can occur.\
27+
- we plan on building a communication helper tool to send you an alert through Slack or email, alerts will be the first line of defense in assessing application health\
28+
- it helps you to develop, debug, and deliver code faster, which, in turn, makes for better productivity and more importantly, happiness in devs
29+
\b \
30+
\
31+
\
32+
technology stack
33+
\b0 \
34+
\
35+
-
36+
\i Docker\
37+
38+
\i0 - Provides containerization of microservice dummy app. We are also looking into orchestration tools such as Kubernetes to kill an instance of a service or raise an alert to Chronos in case of a failing health check. Containers and microservices go hand in hand by by sharing system resources, thereby making them leaner and reducing system overhead, and result in faster and easier scaling.\
39+
-
40+
\i React Context API
41+
\i0 \
42+
- Because our components are multiple and not directly connected, React\'92s Context API becomes very useful. It will help us cache queried information for use by functional React components associated with the visual display of information. It consists of three building blocks: Context Object refers to data shared across component boundaries, Context Provider allows us to use the Context API as a global state management tool by wrapping all child components that will eventually need access to the Context, and Context Consumer helps to update parent/child components based on some centralized state\
43+
- Context Object: data shared across component boundaries\
44+
- Context Provider: with Context created, we can now provide an object as a value to all components that interact with it (reading data, triggering methods). It is provided in a component that wraps all of its child components that will eventually need access to the Context. If you need data to be available throughout your entire app, just provide Context in the root component, like the App component. Allows us to use Context API as a global state management tool.\
45+
- Context Consumer: it is a wrapper component we can use to inject Context provided in some parent component into a child component. When data from our parent changes, the context object in our child component also changes and updates. This helps to update different components based on some centralized state\
46+
-
47+
\i React Hooks
48+
\i0 \
49+
- we can use Context and React Hooks in functional components\
50+
-
51+
\i React Router
52+
\i0 \
53+
\
54+
-
55+
\i Redux
56+
\i0 \
57+
- helps to manage the data you need to render user interface correctly, or state, of a React app\
58+
- In React, we sometimes run into the issue, especially with complex apps, of passing props (which is data from a parent component) that we don\'92t necessarily have to handle. Any changes to an app\'92s state or component structure would require significant refactoring. We won\'92t go into too much detail\
59+
\
60+
-
61+
\i Async Hooks\
62+
63+
\i0 - Node.js API will be used to persist the correlation ID across asynchronous callback functions so that a unique request can be traced back from wherever it was alerted.\
64+
-
65+
\i Electron
66+
\i0 \
67+
- an application used to render Chronos with full access to the Node environment\
68+
- Slack is built on Electron\
69+
\
70+
-
71+
\i Node.js, Express, Charts.js, Spectron, Jest, \
72+
- infrastructure monitoring: \
73+
- Prometheus\
74+
75+
\i0 - an open-sourced monitoring solution originally developed by SoundCloud. It is used to store and query data that describes actions over time. It will assist us in visualizing the time-series data and to provide dashboards. Often used with Grafana, something we\'92ll look into.\
76+
77+
\i \
78+
-Nagios\
79+
80+
\i0 - open-sourced, as well. Assists in continuous server monitoring, an automated approach essential for visibility into infrastructure and applications.\
81+
82+
\i \
83+
- application monitoring: Jaeger\
84+
\
85+
86+
\i0\b technical challenges\
87+
88+
\b0 - Diving into an unfamiliar technology stack\
89+
- Understanding how context propagation works under the hood\
90+
- Utilizing a proxy server to dynamically route client requests to the appropriate service\
91+
\
92+
\
93+
94+
\b stretch
95+
\b0 \
96+
- Travis CI for testing and deployment, linking containers with yaml files, orchestration system like Kubernetes, \
97+
\
98+
99+
\b what we plan to do\
100+
101+
\b0 - create an intuitive and modern UI-centric Electron app with a dashboard that is well-designed and accurately reflects the health of the microservice.\
102+
- use Nginx (\'93engine X\'94) to create a reverse proxy server for back-end routing and caching. This proxy server, in a way, acts as a middleware server to help us route client requests to server. Nginx is one of the top open-sourced web servers.\
103+
- implement metrics to flesh out our health report: average execution time of each of the top ten most frequently executed database queries, average response time for each service endpoint, success/failure ratio for each service. An aggregated set of data can alert us to any system-wide degradation of performance that can potentially lead to failure of the entire system.\
104+
\
105+
106+
\b conclusion\
107+
108+
\b0 - it is just as important to collect relevant data as it is to analyze data that is collected. This data is critical to support a distributed system that is resilient, reliable, and available. Chronos is here to help with that.\
109+
\
110+
}

app/charts/latency-chart.jsx

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ const LatencyChart = (props) => {
2828
name: `${props.service} CPU Latency`,
2929
marker: {
3030
color: '#155263',
31-
}
31+
},
3232
}]}
3333
layout = {{
3434
width: 500,
@@ -44,8 +44,10 @@ const LatencyChart = (props) => {
4444
xaxis: {
4545
tickmode: 'linear',
4646
tick0: 0,
47-
dtick: 2000
48-
}
47+
dtick: 200,
48+
rangemode: 'nonnegative'
49+
},
50+
yaxis: {rangemode: 'nonnegative'}
4951
}}
5052
/>
5153
)

app/charts/memory-chart.jsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ const MemoryChart = (props) => {
8080
xaxis: {
8181
tickmode: 'linear',
8282
tick0: 0,
83-
dtick: 500,
83+
dtick: 100,
8484
},
8585
}}
8686
/>

app/charts/microservice-traffic.jsx

Lines changed: 30 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
import React, { useContext } from 'react';
2-
import { Bar } from 'react-chartjs-2';
2+
import Plot from 'react-plotly.js';
33
import CommunicationsContext from '../context/OverviewContext';
44

55
const MicroServiceTraffic = (props) => {
66
const communicationsData = useContext(CommunicationsContext).overviewData;
77

8+
89
//initialize an empty object resObj. This object will store the microservice names as values and its corresponding correlatingId or correlatingid as keys. The microservice names will be stored in array within the order it was to the database.
910
const resObj = {};
1011

@@ -78,27 +79,35 @@ const MicroServiceTraffic = (props) => {
7879
],
7980
}
8081

81-
82-
// return div with Bar component and Trace Points data
8382
return (
84-
<div>
85-
<Bar
86-
data={myChart}
87-
width={100}
88-
height={50}
89-
options={{
90-
title:{
91-
display:true,
92-
text:'Microservices Overview',
93-
fontSize:20
94-
},
95-
legend:{
96-
display:true,
97-
position:'right'
98-
}
99-
}}
100-
/>
101-
</div>
83+
<Plot
84+
data = {[{
85+
type: 'bar',
86+
x: ['Orders', 'Customers', 'Books', 'Reverse-Proxy', 'ReverseProxy'],
87+
y: [...serverPingCount, 0, yAxisHeadRoom],
88+
fill: 'tozeroy',
89+
color: 'red',
90+
opacity: .4,
91+
mode: 'none',
92+
name: 'Times Server Pinged',
93+
showlegend: true
94+
}]}
95+
layout = {
96+
{
97+
width: 500,
98+
height: 500,
99+
paper_bgcolor: '#fffbe0',
100+
plot_bgcolor: '#fffbe0',
101+
legend: {
102+
orientation: 'h',
103+
xanchor: 'center',
104+
x: .5,
105+
y: 5
106+
},
107+
yaxis: {rangemode: 'nonnegative'}
108+
}
109+
}
110+
/>
102111
)
103112
};
104113

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
import React, { useContext } from 'react';
2+
import { Bar } from 'react-chartjs-2';
3+
import CommunicationsContext from '../context/OverviewContext';
4+
5+
const MicroServiceTraffic = (props) => {
6+
const communicationsData = useContext(CommunicationsContext).overviewData;
7+
8+
//initialize an empty object resObj. This object will store the microservice names as values and its corresponding correlatingId or correlatingid as keys. The microservice names will be stored in array within the order it was to the database.
9+
const resObj = {};
10+
11+
if(communicationsData.length>0 && communicationsData[0]["_id"]){
12+
//Sort the communication array from latest to earliest document
13+
communicationsData.sort((a,b)=>{
14+
if(new Date(a.timeSent) > new Date(b.timeSent)) return 1;
15+
if(new Date(a.timeSent) < new Date(b.timeSent)) return -1;
16+
return 0;
17+
});
18+
19+
//Iterate over sorted communicationsData array from the end to the beginning
20+
for (let i = 0; i < communicationsData.length; i+=1) {
21+
//declare a constant element and initialize it as the object at index i of the communicationsData array
22+
const element = communicationsData[i];
23+
//Pushes the microservice name into the object
24+
if (resObj[element.correlatingId]) {
25+
resObj[element.correlatingId].push(element.currentMicroservice);
26+
}
27+
else resObj[element.correlatingId] = [element.currentMicroservice];
28+
}
29+
}
30+
31+
else {
32+
for (let i = communicationsData.length-1; i >= 0; i--) {
33+
const element = communicationsData[i];
34+
if (resObj[element.correlatingid]) resObj[element.correlatingid].push(element.currentmicroservice);
35+
else resObj[element.correlatingid] = [element.currentmicroservice];
36+
// initializing the object with the first microservice
37+
};
38+
}
39+
40+
//use object values to destructure locations
41+
const tracePoints = Object.values(resObj);
42+
43+
// Declare Micro-server-count dictinary to capture the amount of times a particular server is hit
44+
const microServiceCountdictionary = {};
45+
46+
//array logging every ping present in communications table ---> flat used to flatten multidimensional array and return 1d array
47+
const tracePointLog = tracePoints.flat(Infinity);
48+
49+
// iterate over Trace Points
50+
for (let i = 0; i < tracePointLog.length; i+=1) {
51+
52+
// populate Micro-count dictionary
53+
if (!microServiceCountdictionary[tracePointLog[i]]) {
54+
microServiceCountdictionary[tracePointLog[i]] = 1;
55+
} else {
56+
microServiceCountdictionary[tracePointLog[i]]+=1
57+
}
58+
};
59+
60+
// capture values of microServiceCountdictionary to use as data to populate chart object
61+
const serverPingCount = Object.values(microServiceCountdictionary);
62+
63+
// variable 10 points higher than max number in microservicesDictionary aggregation --> variable allows for top level spacing on bar graph
64+
const yAxisHeadRoom = (Math.max(...serverPingCount) + 10);
65+
66+
// Create chart object data to feed into bar component
67+
const myChart = {
68+
//spread dictionary keys inorder to properly label chart x axis
69+
labels: [...Object.keys(microServiceCountdictionary)],
70+
datasets: [
71+
{
72+
label: 'Times server Pinged',
73+
backgroundColor: 'rgba(241, 207, 70,1)',
74+
borderColor: 'rgba(0,0,0,1)',
75+
borderWidth: 1,
76+
data: [...serverPingCount, 0, yAxisHeadRoom] // spread ping count array into data array to have chart populate the Y axis
77+
}
78+
],
79+
}
80+
81+
82+
// return div with Bar component and Trace Points data
83+
return (
84+
<div>
85+
<Bar
86+
data={myChart}
87+
width={100}
88+
height={50}
89+
options={{
90+
title:{
91+
display:true,
92+
text:'Microservices Overview',
93+
fontSize:20
94+
},
95+
legend:{
96+
display:true,
97+
position:'right'
98+
}
99+
}}
100+
/>
101+
</div>
102+
)
103+
};
104+
105+
export default MicroServiceTraffic;
106+

app/charts/processes-chart.jsx

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -35,17 +35,6 @@ const ProcessesChart = (props) => {
3535
return (
3636
<Plot
3737
data = {[
38-
{
39-
type: 'scatter',
40-
x: {autorange: true},
41-
y: totalProcesses,
42-
mode: 'markers',
43-
rangemode: 'nonnegative',
44-
name: 'Total Processes',
45-
marker: {
46-
color: 'red',
47-
size: 3
48-
}},
4938
{
5039
type: 'scatter',
5140
x: {autorange: true},

app/charts/route-trace.jsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ const RouteLocations = (props) => {
4949
<p id="routeText">Point {i + 1}: {tracePoints[position][i]}</p>
5050
</div>,
5151
);
52-
}
52+
}
5353

5454
console.log('resArray: ', resArray);
5555

app/charts/speed-chart.jsx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@ const SpeedChart = (props) => {
2929
data = {[{
3030
domain: { x: [0, 1], y: [0, 1] },
3131
type: 'indicator',
32-
value: yAxis[length],
33-
// title: {'text': "Speed Chart"},
32+
value: yAxis[yAxis.length - 1],
33+
title: {'text': "Speed Chart"},
3434
delta: {'reference': 3.5, 'increasing': {'color': "mistyrose"}},
3535
mode: "gauge+number+delta",
3636
gauge: { axis: { range: [null, 8] },

app/charts/temperature-chart.jsx

Lines changed: 2 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -27,11 +27,9 @@ const TemperatureChart = (props) => {
2727
return (
2828
<Plot
2929
data = {[{
30-
type: 'bar',
31-
width: 200,
30+
type: 'scatter',
3231
fill: 'tozeroy',
33-
color: '#00d1ff',
34-
opacity: .4,
32+
fillcolor: 'rgba(224, 62, 54, .6)',
3533
mode: 'none',
3634
x: yAxis,
3735
y: xAxis,
@@ -49,11 +47,6 @@ const TemperatureChart = (props) => {
4947
xanchor: 'center',
5048
x: .5
5149
},
52-
xaxis: {
53-
tickmode: 'linear',
54-
tick0: 0,
55-
dtick: 2000,
56-
},
5750
yaxis: {rangemode: 'nonnegative'}
5851
}
5952
}

0 commit comments

Comments
 (0)