Skip to content

developer-guide

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

developer-guide

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
codacy-ebook-metrics.pdf
codacy-ebook-metrics.pdf
 
 

README.md

Developer Guide

code This guide is for those who want to contribute code to **parse-numeric**. This guide describes how to set up your development environment so that you can build and test **parse-numeric**.

Table of contents

1. Recommended skills

In order to work with parse-numeric as a developer, we recommend you feel comfortable with:

Skill Reason
code JavaScript parse-numeric is written in JavaScript (CommonJS)
markdown Documentation Docs build community and enable asyncronous collaboration
package NPM We use the npm dependency manager and publish to the npm public registry
platform Node.js parse-numeric runs on Node.js
terminal Command-line-interface We use the CLI to build, test, and publish parse-numeric
tests Unit testing Unit tests are a form of executable documentation that make collaboration, fixes, and refactoring possible

If that sounds like you, then continue reading to get started.

2. Development software

Before you can build and test parse-numeric, you must install and configure the following products on your development machine:

  1. Git link-external

    Why:

    parse-numeric is hosted on github and uses Git for source control. In order to obtain the source code, you must first install Git on your system. Instructions for installing and setting up Git can be found at https://help.github.com/articles/set-up-git.

  2. Node.js link-external, (version specified in the engines field of package.json).

    Why:

    parse-numeric uses Node.js modules to generate tables of contents, version, and publish documentation.

  3. NPM (which installs with Node.js) or Yarn link-external

    Why:

    NPM and Yarn install and update parse-numeric's third-party dependencies.

3. Getting the source code

Fork and clone the parse-numeric repository:

  1. Sign in to github.

  2. Fork the main parse-numeric repository (aka, "origin").

  3. Clone your fork of the parse-numeric repository and define an upstream remote pointing back to the parse-numeric repository that you forked in the first place.

    # Clone your github repository:
git clone git@github.com:<your-account>/parse-numeric.git

# Go to the repo root directory:
cd parse-numeric

# Add the main github repository as an upstream remote
# to your repository:
git remote add upstream https://github.com/gregswindle/parse-numeric.git
directory structure Toggle project directory structure view... 
parse-numeric/
├─┬ .git/**
├─┬ .github/
│ ├─┬ issue_templates/
│ │ ├── Defect.md
│ │ ├── Feature.md
│ │ ├── Metric.md
│ │ └── Refactoring.md
│ ├─┬ merge_request_templates/
│ │ └── merge_request.md
│ └─┬ rc/
│   ├─┬ bash/
│   │ └── .envvarsrc
│   ├─┬ jsdoc/
│   │ └── conf.json
│   ├─┬ md/
│   │ ├── .remarkignore
│   │ ├── .remarkrc.yml
│   │ └── markdown.config.js
│   └─┬ sonar/
│     └── scan.js
├─┬ .vscode/
│ └── settings.json
├─┬ docs/
│ ├─┬ .ci/
│ │ └── eslint-rules
│ ├─┬ developer-guide/
│ │ ├── codacy-ebook-metrics.pdf
│ │ ├── collaborator-guide.md
│ │ └── README.md
│ ├─┬ includes/
│ │ └── octicons.md
│ ├─┬ maintainer-guide/
│ │ ├─┬ governance-models/
│ │ │ ├── benevolent-dictator-governance-model.md
│ │ │ ├── governance-role-terminology-comparision.md
│ │ │ ├── loosely-coupling-work-from-orgs.md
│ │ │ ├── meritocratic-governance-model.md
│ │ │ └── README.md
│ │ ├── codacy-ebook-code-reviews.pdf
│ │ ├── displaying-product-maturity.md
│ │ ├── governance.md
│ │ ├── issues.md
│ │ ├── pull-requests.md
│ │ ├── README.md
│ │ └── releases.md
│ ├── README.md
│ └── STYLE_GUIDES.md
├─┬ lib/
│ ├─┬ __tests__/
│ │ ├─┬ __fixtures__/
│ │ │ ├── gl-got-resp-failure.json
│ │ │ ├── gl-got-resp-success.js
│ │ │ ├── options.json
│ │ │ ├── regexes.json
│ │ │ └── signatures.json
│ │ ├─┬ __mocks__/
│ │ │ └── is-main.js
│ │ ├── cli.test.js
│ │ ├── parse-numeric.test.js
│ │ └── env-config.test.js
│ ├─┬ cli/
│ │ ├─┬ __tests__/
│ │ │ ├── cli-client.test.js
│ │ │ └── run.test.js
│ │ ├─┬ signature-ctrl/
│ │ │ ├─┬ __tests__/
│ │ │ │ ├─┬ __fixtures__/
│ │ │ │ │ ├── gl-got-resp-success.js
│ │ │ │ │ └── signatures.json
│ │ │ │ └── signature-ctrl.test.js
│ │ │ └── index.js
│ │ ├── cli-client.js
│ │ ├── options.js
│ │ └── run.js
│ ├─┬ github/
│ │ ├─┬ __tests__/
│ │ │ ├─┬ __fixtures__/
│ │ │ │ ├── git-lab-error.json
│ │ │ │ ├── github-response-mock.json
│ │ │ │ └── got-resp.json
│ │ │ ├── assign-content-string.test.js
│ │ │ ├── github.test.js
│ │ │ ├── resolve-by.test.js
│ │ │ ├── sanitize.test.js
│ │ │ └── to-resource-path.test.js
│ │ ├── api-defaults.js
│ │ ├── assign-content-string.js
│ │ ├── index.js
│ │ ├── is-git-lab-error.js
│ │ ├── resolve-by.js
│ │ ├── sanitize.js
│ │ └── to-resource-path.js
│ ├─┬ signature/
│ │ ├─┬ __tests__/
│ │ │ ├── signature-class.test.js
│ │ │ └── signature.test.js
│ │ ├── index.js
│ │ ├── null-signature.js
│ │ ├── signature.js
│ │ ├── valid-signature-parts.js
│ │ └── valid-signature-types.js
│ ├── cli.js
│ ├── env-config.js
│ └── index.js
├── .editorconfig
├── .env.defaults
├── .env.schema
├── .eslintignore
├── .eslintrc.yml
├── .fossa.yml
├── .gitattributes
├── .gitignore
├── .github-ci.yml
├── .npmignore
├── .npmrc
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE
├── npm-shrinkwrap.json
├── package.json
├── README.md
├── ROADMAP.md
├── signatures.json
└── sonar-project.properties

4. Installing dependencies

Next, install the JavaScript modules needed to build and test parse-numeric:

# Install all project dependencies (package.json)
npm install
Toggle **parse-numeric** view...
Dependency Description Version License Type
atob@^2.1.2 atob for Node.JS and Linux / Mac / Windows CLI (it's a one-liner) 2.1.2 (MIT OR Apache-2.0) production
dotenv-extended@^2.3.0 A module for loading .env files and optionally loading defaults and a schema for validating all values are present. 2.3.0 MIT production
git-repo-info@^2.1.0 Retrieve current sha and branch name from a git repo. 2.1.0 MIT production
gl-got@^9.0.3 Convenience wrapper for got to interact with the github API 9.0.3 MIT production
json-stringify-safe@5.0.1 Like JSON.stringify, but doesn't blow up on circular refs. 5.0.1 ISC production
lodash.set@^4.3.2 The lodash method _.set exported as a module. 4.3.2 MIT production
lodash.topairs@^4.3.0 The lodash method _.toPairs exported as a module. 4.3.0 MIT production
meow@^5.0.0 CLI app helper 5.0.0 MIT production
jsdoc@^3.5.5 An API documentation generator for JavaScript. 3.5.5 Apache-2.0 optional
markdown-magic@^0.1.25 Automatically update markdown files with content from external sources 0.1.25 MIT optional
markdown-magic-dependency-table@^1.3.2 Generate table of information about dependencies automatically in markdown 1.3.2 MIT optional
markdown-magic-directory-tree@^1.2.3 Print an archy tree for markdown file 1.2.3 MIT optional
markdown-magic-package-scripts@^1.2.1 Print list of scripts in package.json with descriptions 1.2.1 MIT optional
remark@^10.0.1 Markdown processor powered by plugins 10.0.1 MIT optional
remark-cli@^6.0.1 CLI to process markdown with remark using plugins 6.0.1 MIT optional
remark-preset-lint-markdown-style-guide@^2.1.2 remark preset to configure remark-lint with rules that enforce the markdown style guide 2.1.2 MIT optional
@semantic-release/changelog@^3.0.2 semantic-release plugin to create or update a changelog file 3.0.2 MIT dev
@semantic-release/commit-analyzer@^6.1.0 semantic-release plugin to analyze commits with conventional-changelog 6.1.0 MIT dev
@semantic-release/git@^7.0.8 semantic-release plugin to commit release assets to the project's git repository 7.0.8 MIT dev
@semantic-release/github@^3.1.2 semantic-release plugin to publish a github release 3.1.2 MIT dev
@semantic-release/npm@^5.1.4 semantic-release plugin to publish a npm package 5.1.4 MIT dev
@semantic-release/release-notes-generator@^7.1.4 semantic-release plugin to generate changelog content with conventional-changelog 7.1.4 MIT dev
commitplease@^3.2.0 Validates strings as commit messages 3.2.0 MIT dev
eslint@^5.12.1 An AST-based pattern checker for JavaScript. 5.12.1 MIT dev
eslint-config-prettier@4.0.0 Turns off all rules that are unnecessary or might conflict with Prettier. 4.0.0 MIT dev
eslint-config-standard@^12.0.0 JavaScript Standard Style - ESLint Shareable Config 12.0.0 MIT dev
eslint-config-xo@^0.26.0 ESLint shareable config for XO 0.26.0 MIT dev
eslint-find-rules@3.3.1 Find built-in ESLint rules you don't have in your custom config. 3.3.1 MIT dev
eslint-plugin-import@^2.15.0 Import with sanity. 2.15.0 MIT dev
eslint-plugin-json@^1.3.2 Lint JSON files 1.3.2 ISC dev
eslint-plugin-no-unsafe-innerhtml@^1.0.16 custom ESLint rule to disallows unsafe innerHTML, outerHTML and insertAdjacentHTML 1.0.16 MPL-2.0 dev
eslint-plugin-no-unsanitized@^3.0.2 ESLint rule to disallow unsanitized code 3.0.2 MPL-2.0 dev
eslint-plugin-node@^8.0.1 Additional ESLint's rules for Node.js 8.0.1 MIT dev
eslint-plugin-prettier@^3.0.1 Runs prettier as an eslint rule 3.0.1 MIT dev
eslint-plugin-promise@^4.0.1 Enforce best practices for JavaScript promises 4.0.1 ISC dev
eslint-plugin-scanjs-rules@^0.2.1 ESLint plugin that contains ScanJS rules 0.2.1 MPL-2.0 dev
eslint-plugin-security@^1.4.0 Security rules for eslint 1.4.0 Apache-2.0 dev
eslint-plugin-sonarjs@^0.2.0 SonarJS rules for ESLint 0.2.0 LGPL-3.0 dev
eslint-plugin-standard@^4.0.0 ESlint Plugin for the Standard Linter 4.0.0 MIT dev
eslint-plugin-unicorn@^7.1.0 Various awesome ESLint rules 7.1.0 MIT dev
eslint-plugin-xss@^0.1.9 Validates XSS related issues of mixing HTML and non-HTML content in variables. 0.1.9 ISC dev
execa@^1.0.0 A better child_process 1.0.0 MIT dev
generator-jest@1.6.0 Add jest support to any projects 1.6.0 MIT dev
get-stream@^4.1.0 Get a stream as a string, buffer, or array 4.1.0 MIT dev
husky@^1.3.1 Prevents bad commit or push (git hooks, pre-commit/precommit, pre-push/prepush, post-merge/postmerge and all that stuff...) 1.3.1 MIT dev
jest@24.0.0 Delightful JavaScript Testing. 24.0.0 MIT dev
jest-junit@6.2.0 A jest reporter that generates junit xml files 6.2.0 Apache-2.0 dev
jest-sonar-reporter@2.0.0 A Sonar test reporter for Jest. 2.0.0 MIT dev
lec@^1.0.1 Command Line Wrapper for Line Ending Corrector (An utility that makes sure your files have consistent line endings) 1.0.1 MIT dev
lint-staged@8.1.1 Lint files staged by git 8.1.1 MIT dev
lodash.noop@^3.0.1 The lodash method _.noop exported as a module. 3.0.1 MIT dev
prettier@^1.16.1 Prettier is an opinionated code formatter 1.16.1 MIT dev
semantic-release@^15.13.... Automated semver compliant package publishing 15.13.3 MIT dev
sonarqube-scanner@2.1.2 SonarQube/SonarCloud Scanner for the JavaScript world 2.1.2 LGPL-3.0 dev
yo@2.0.5 CLI tool for running Yeoman generators 2.0.5 BSD-2-Clause dev

5. Running in a Terminal

Run npm-link npm-link documentation to user parse-numeric's CLI (Command-Line Interface).

  1. Open a Terminal and change directories to your local repository/project folder:

    cd /path/to/local/parse-numeric

  2. Create a global symbolic link to parse-numeric:

    npm link

  3. Now you can test and use parse-numeric with two (2) CLI commands:

    parse-numeric --help

# ...or

get-signatures --help

  4. If successful, you should see output like this:

    NAME
  parse-numeric - GET signatures.json, a single collection of
                        data-leakage definitions.
                        (alias: get-signatures)

Usage
  $ parse-numeric [options]
  $ get-signatures [options]

Options
  --base-url    The common base URL to the github v4 API.
                Defined in the ENV variable github_ENDPOINT.
                [Default: https://github.com/api/v4]
  --file-path   The resource path to signatures.json.
                [Default: signatures.json]
  --project-id  The namespace or unique identifier for the repo
                with the master signatures.json file.
                [Default: 10416318]
  --ref         The name of branch, tag or commit.
                [Default: master]
  --token       github personal access token.
                Defined in the ENV variable github_TOKEN.
  --verbose     Display the entire response.
                [Default: true]

  --help        Show this message and exit.
  --version     Show your installed version and exit.

Examples
  $ parse-numeric

  # => [<Signature>,<Signature>,...,<Signature>]
  $ **parse-numeric** --base-url="https://github.example.com/api/v4"
  # => [<Signature>,<Signature>,...,<Signature>]
  $ get-signatures --token="<your-access-token>"
  # => [<Signature>,<Signature>,...,<Signature>]

6. Testing

Jest BDD parse-numeric uses Jest for Behavior-Driven Development (BDD) and unit tests link-external.

Your test suites must pass within coverage thresholds before your Merge Request will be reviewed on github.

To run tests:

$ npm test
# => Run all **parse-numeric** tests on node

7. Source code style guidelines

verified parse-numeric uses

  1. ESLint link-external to evaluate and format source code;
  2. Prettier link-external to format JSON, Markdown, and YAML.
  3. Standard JS code style link-external for code clarity and community conventions.

You can both evaluate and format your all sources by running:

$ npm run lint
# => Formats and lints all JavaScript, JSON, Markdown, and
#    package.json.

You can also format sources by type:

# Evaluate and format JavaScript:
npm run lint:js

# Format JSON:
npm run lint:json

# Format all markdown files:
npm run lint:md
View all available npm-scripts...

To execute any of the following scripts, open a Terminal and run:

npm run <Script>

Examples:

# Auto-generate TOCs, dependency tables, etc.
npm run docs:md

# Lint all *.js source code files
npm run lint:js
Script Description
ci:lint:js eslint . --fix -f json -o eslint-report.json
preci:test npm run pretest && source ./.github/rc/bash/.envvarsrc
ci:test dotenv-extended --path=.env --defaults=.env.defaults --errorOnMissing=false jest --coverage --verbose true
postci:test npm run posttest && source ./.github/rc/bash/.envvarsrc && npm run qa:review:sonar
cli:lec lec lib/cli.js -c LF
docs:api rm -rf ./docs/api/ && ./node_modules/.bin/jsdoc ./lib/**/*.js -d ./docs/api/ -c ./.github/rc/jsdoc/conf.json -d ./docs/api/
docs:lint:js:rules eslint-find-rules
docs:lint:js:rules:current eslint-find-rules --current --no-error .eslintrc.yml --verbose > ./docs/.ci/eslint-rules.txt
docs:lint:js:rules:unused eslint-find-rules --unused --no-error .eslintrc.yml --verbose >> ./docs/.ci/eslint-rules.txt
predocs:md npm run docs:lint:js:rules
docs:md node .github/rc/md/markdown.config.js
lint npm run lint:js && npm run lint:json && npm run lint:md
lint:js eslint . --fix -f json -o eslint-report.json
lint:json prettier --write **/*.json
lint:manifest npm prune && npm dedupe && npm shrinkwrap
lint:md remark --rc-path "./.github/rc/md/.remarkrc.yml" --ignore-path "./.github/rc/md/.remarkignore" .
ls npm ls --depth=0
postinstall npm run security:audit
prepare npm run cli:lec && npm run deps:dedupe
preqa:review:sonar npm start
qa:review:sonar sonar-scanner -Dproject.settings=./sonar-project.properties -Dsonar.projectKey=$SONAR_PROJECT_KEY -Dsonar.organization=$SONAR_ORGANIZATION -Dsonar.host.url=https://sonarcloud.io -Dsonar.login=$SONAR_TOKEN
security:audit npm audit --json
security:fix npm audit fix --depth=100 --json
semantic-release semantic-release
start node -r dotenv-extended ./lib/env-config.js
pretest npm run lint:js
test jest --coverage --verbose true
posttest npm run security:audit
test:create yo jest:test --
vars:git:branch:name `git branch
vars:package:description node -pe "require('./package.json').description"
vars:package:keywords node -pe "require('./package.json').keywords.join(', ')"
vars:package:name node -pe "require('./package.json').name"
vars:package:version node -pe "require('./package.json').version"

8. DevSecOps

github CI/CD logo

parse-numeric uses github CI/CD github documentation for continuous integration and delivery.

All tests are executed with Continuous Integration services.

  1. Pull Requests will only be approved and merged once all tests pass.
  2. github CI/CD will fail if any of the test suites fails, or a linting rule is violated.

alert

CI will fail and the Pull Requests cannot be merged unless changes pass:

  • Linting,
  • Unit tests, and
  • Code coverage thresholds

9. Design

parse-numeric is a Node.js module. parse-numeric is published to npm, which is a public registry of Node.js packages. The community usually installs Node.js packages with a dependency manager. The two most popular Node.js dependency managers are

  1. npm
  2. Yarn
  • Standalone software (with np)
  1. Single source-of-truth—-signatures.json—-for defining and detecting data-leakage.
  2. REST API module and Terminal client for retrieving all signatures.

All data-leakage definitions (e.g., regular expressions) MUST be defined in the signatures.json file. All other applications, components, modules, etc., that require data-loss detection SHOULD use parse-numeric instead of defining their own.

parse-numeric component diagram

10. Roadmap

Telesope View the ROADMAP for parse-numeric.