Architecture

This page is intended to describe the top-level architecture of the project, what tools are being used, what features we want, etc. I am also summarizing the basic architecture of the MEAN stack, as I understand it and as it pertains to our project. Please make any changes/contributions/corrections you think are appropriate.

Goal

We want to help users understand their activity on GitHub, in particular through informative visualizations.

User Interface#

• Receives user queries/input, presents visualizations
• Composed of HTML/CSS that is generated/updated dynamically by AngularJS, with embedded elements
• Visualizations are produced by D3
• AngularJS sends requests to the backend, and displays results
• AngularJS is a .js file that is downloaded and run on the client-side in the user's browser, same with D3

MiddleWare: Express

• Express runs on the server, and handles Angular's requests
• Express must know to where to find the requested data. It will probably get the data from GitHub and store it in MongoDB for future use. This is where we have to deal with things like the GitHub API, caching logic, etc

Database: MongoDB

• MongoDB is a database that stores and retrieves Javascript objects. We have to make sure that we can efficiently retrieve what we need, particularly if a search is involved. We can also do some processing of the data here.

Node.js

• Node is the actual program that runs on the server and runs arbitrary JavaScript code. In particular, Express runs on top of Node. We can also find many libraries that run on top of Node to do interesting things. Any processing that isn't abstracted away by one of the other layers is done here.

The code we need to write:

• General UI: things like login box, maybe search bar, About page, instructions, demo, etc. This is mostly in Angular.
• Data retrieval: Access the GitHub API and/or use git clone. Code probably in Express/pure Javascript for Node.js
• Data Processing: Getting it ready for visualization, eg aggregation, filtering, sorting, etc. The goal is to pass as little actual data as possible over the network, and make sure that code runs where it's fastest. This may be partially on pure Node/Express, partially in MongoDB, and partially client-side
• Data Caching: Instead of making multiple requests to GitHub, and processing the same data repeatedly, we want to cache results in MongoDB
• Actual visualizations in D3, and integration into Angular
• Testing