Technologies: Node.js/Typescript & MongoDB
my node version: 14.13.1
my npm version: 6.14.8
The application using this following components:
- PM2 as process manager.
- GCP Pubsub service for asynchronous messaging service ( for high scale to the following services: VT service ,Who is service).
- MongoDB as a DB.
- Cache in-memory data structure store for higher performance.
- Tests - using Mocha and Chai.
methods:
POST: /scanDomain
GET:/getResultsByDomain
Intelligenter is an API NodeJS / Typescript component which enhances data about a given domain. The component is reading the domain from the database on a given interval and updating results. The results are also stored on a database for future use. The Data of each domain includes data from VirusTotal and from Whois reops (See example).
The component is asynchronous and supports restAPI requests using NodeJS-Express. The restAPI will include two methods, GET and POST. GET will extract all the stored data on a domain and will return a JSON response, if the domain does not exist, it will be sent to analysis and the user will get an “OnAnalysis” message. POST will get the domain name and will send it to analysis. In any case, all requests and results will be stored on the database.
The scheduler will verify that the record is valid and will automatically update the records occasionally. The scheduler will verify that every valid record is being updated once a month.
GET localhost:3000/getResultsByDomain?domain=google.com
response: { domain: google.com, VTData: {numberOfDetection: 1, numberOfScanners: 70, detectedEngines: [“CLEAN MX”], lastUpdated: “09.09.19” }, WhoisData:{ dateCreated: “09.15.97”, ownerName: “MarkMonitor, Inc.”, expriedOn: ”09.13.28” }.
POST localhost:3000/scanDomain BODY: { domain=google.com }
response: { domain: google.com, status: ”onAnalysis” }
API_KEY_VIRTUS_TOTAL = { YOUR_VT_KEY }
API_KEY_WHOIS = { YOUR_WI_KEY }
GOOGLE_APPLICATION_CREDENTIALS = { /path/to/the/credentials }
DATA_BASE = { YOUR_PATH_DB_CONNECTION }
https://cloud.google.com/docs/authentication/production.
- npm install pm2@latest -g.
- npm install.
- npm run build.
- npm start.
VirusTotal API key - https://support.virustotal.com/hc/en-us/articles/115002088769-Please-give-me-an-API-key.
Who is API key- https://whois.whoisxmlapi.com/overview.
GCP(google cloud platform) - register,create new your project and download your auth file.json from: https://cloud.google.com/docs/authentication/production.
