feat(command): add integrate command (#47)

Added a new `integrate` command which supports running integration tests from a separate `integration-tests/` "project" within the library. Docs for the new `integrate` command were added as well.

I also added integration tests for `@benmvp/cli` for the build command to verify that build actually does what it is supposed to. So I was able to Fix #32 and also dogfood the `integrate` command.

As part of building out the `integrate` functionality, I needed to add the result of `npm pack` as a dev dependency to the project, which kept failing in Yarn. `yarn add --dev /path/to/tarball.tgz` would return before it was _really_ done and a result the follow on commands with fail. As a result I switched the project over from Yarn to npm. All docks were updated accordingly.

BREAKING CHANGE: The `unit` test mode was renamed to `spec` in order to make more sense with integration tests.

The way the integration tests are run in the project is by running `npx benmvp test`. The parameters passed to `benmvp integrate` are ultimately passed as the parameters to `npx benmvp test`. Have a mode called `unit` when running integration tests seemed confusion.  Really it's just running Jest spec tests, so the rename was to better reflect that. In order to just run **unit** tests, you run `benmvp test -m spec` in order to just run **integration** tests you run `benmvp integration -m spec`.

I also set `maxWarnings` for eslint to `0` so that all warnings are also errors as well.

Author: benmvp

.travis.yml

@@ -4,15 +4,16 @@ node_js:
   - '12'
   - '10'
   - '8'
-cache: yarn
+cache: npm
 
-script: yarn test
+script: npm run test && npm run integrate
 
 jobs:
   include:
-    - stage: NPM Release
+    - stage: deploy
+      name: "NPM Release"
       node_js: '10'
-      script: yarn build
+      script: npm run build
 
       # Using semantic-release, deploy a new version of the library
       deploy:

CONTRIBUTING.md

@@ -13,12 +13,11 @@ Thank you for your interest in contributing to my CLI!
 
 ## Setup
 
-1. This project uses [yarn](https://yarnpkg.com/en/) as a package management system. If you don't have it installed, you can follow the [installation instructions](https://yarnpkg.com/lang/en/docs/install/)
 1. [For the repository](https://help.github.com/articles/fork-a-repo/)
 1. Clone your new forked repository to your local computer
 1. Set the `benmvp` repository as your branch's upstream branch: `git remote add upstream https://github.com/benmvp/benmvp-cli.git`
 1. Navigate to the root directory of your newly cloned repository: `cd /path/to/benmvp-cli`
-1. `yarn install` to install local dependencies
+1. `npm install` to install local dependencies
 
 ## Using branches to submit changes
 
@@ -58,8 +57,8 @@ Please try to conform to the coding style of the code base.
 
 ## Steps to submit:
 
-1. Please ensure that your changes are fully covered by one or more unit test(s).
+1. Please ensure that your changes are fully covered by one or more unit tests.
 1. Check to make sure that your changes are documented properly (inline comments for interesting lines, READMEs, etc.)
-1. Run `yarn test` to ensure that all tests pass, the linter is satisfied and your changes are typescript compliant.
+1. Run `npm test` to ensure that all tests pass, the linter is satisfied and your changes are typescript compliant.
 1. PR titles must be prefixed by the type of changes the PR contains followed by the scope of what the PR touches. We are following the [angular commit guidelines](https://github.com/angular/angular.js/blob/master/DEVELOPERS.md#-git-commit-guidelines). Please use one of `feat, fix, docs, style, refactor, perf, test, chore` as the prefix. The " is the the direct product your changes affect. Example: `chore(build): Add encrypted ssh key for semantic-release` because its a chore and it touches the build. For multiple scope items, you can comma separate 2 or 3 but if there are more than that please use a `*` instead.
 1.  Please use a [closing issue keyword](https://help.github.com/articles/closing-issues-using-keywords/) to indicate the issue that your fix addresses in the description section of the pull request template. Example: `fixes #32` to close issue #32

README.md

@@ -54,36 +54,44 @@ Read the API docs for more on [`benmvp create`](docs/cli/create.md).
 
 ## Quick Usage
 
-### Testing
+### Unit Testing
 
 ```sh
-yarn test
+npm test
 ```
 
-In your continuous integration (CI) environment, run `yarn test` (or `npm test`) to do a one-time pass of Typescript typings, ESLint linting, and Jest tests. This can also be run locally (i.e. for commit hooks).
+In your continuous integration (CI) environment, run `npm test` (or `yarn test`) to do a one-time pass of Typescript typings, ESLint linting, and Jest unit tests. This can also be run locally (i.e. for commit hooks).
 
 Read the API docs for more on [`benmvp test`](docs/cli/test.md).
 
 ### Developing
 
 ```sh
-yarn start
+npm start
 ```
 
-When developing, run `yarn start` (or `npm start`) which will validate Typescript typings, ESLint linting, and Jest tests as you develop and change code.
+When developing, run `npm start` (or `yarn start`) which will validate Typescript typings, ESLint linting, and Jest unit tests as you develop and change code.
 
 Read the API docs for more on [`benmvp start`](docs/cli/start.md).
 
 ### Building
 
 ```sh
-yarn build
+npm run build
 ```
 
-In your continuous integration (CI) environment, run `yarn build` (or `npm build`) to generated transpiled versions of your code in ESM (ECMAScript module) and CJS (CommonJS).
+In your continuous integration (CI) environment, run `npm run build` (or `yarn build`) to generate transpiled versions of your code in ESM (ECMAScript module) and CJS (CommonJS), as well as TypeScript definition files.
 
 Read the API docs for more on [`benmvp build`](docs/cli/build.md).
 
+### Integration Testing
+
+```sh
+npm run integrate
+```
+
+In your continuous integration (CI) environment, run `npm run integrate` (or `yarn integrate`) to run additional integration tests for the library. The integration "project" will also be typed and linted. This can also be run locally (i.e. for commit hooks but is not recommended).
+
 ## Docs
 
 `@benmvp/cli` has two interfaces: a CLI and a Node API. View the [full docs](docs/).

bin/benmvp

@@ -7,7 +7,7 @@
   const {code, message, error} = await run(args);
 
   if (code) {
-    console.error(error);
+    // console.error(error);
     console.error(message);
   }
 

docs/README.md

@@ -10,6 +10,7 @@ The CLI exposes the following commands:
 - [`benmvp test`](cli/test.md) - Runs a one-time pass of typing, linting, unit tests & code coverage for the library
 - [`benmvp start`](cli/start.md) - Runs the lib's tests in on-going watch mode during active development
 - [`benmvp build`](cli/build.md) - Builds the library into the desired module formats at the specified location
+- [`benmvp integrate`](integrate.md) - Runs additional integration tests for the library
 
 ## Node API docs
 
@@ -19,4 +20,5 @@ The Node API exposes the following functions:
 - [`test()`](api/test.md) - Runs a one-time pass of the specified modes of tests
 - [`start()`](api/start.md) - Runs the specified modes of tests in on-going watch mode during active development
 - [`build()`](api/build.md) - Builds the library into the desired module formats at the specified location
+- [`integrate()`](integrate.md) - Runs additional integration tests for the library
 - [`run()`](api/run.md) - Parses the specified array of CLI arguments to runs the desired command

docs/api/README.md

@@ -6,5 +6,6 @@ The Node API exposes the following functions:
 - [`test()`](test.md) - Runs a one-time pass of the specified modes of tests
 - [`start()`](start.md) - Runs the specified modes of tests in on-going watch mode during active development
 - [`build()`](build.md) - Builds the library into the desired module formats at the specified location
+- [`integrate()`](integrate.md) - Runs additional integration tests for the library
 - [`run()`](run.md) - Parses the specified array of CLI arguments to runs the desired command
 

docs/api/build.md

@@ -49,7 +49,7 @@ build({
 `build()` has the following [TypeScript](https://www.typescriptlang.org/) signature:
 
 ```js
-([options]: Options): Promise<Result>
+(options?: Options): Promise<Result>
 ```
 
 ## Options
@@ -76,7 +76,7 @@ Optional. Defaults to `./lib`.
 
 ### `watch`
 
-A flag indicating whether or not to continuously generate the built module formats whenever source files change. This is most useful if you've linked your library into a host application (with [`yarn link`](https://yarnpkg.com/lang/en/docs/cli/link/) or [`npm link`](https://docs.npmjs.com/cli/link)).
+A flag indicating whether or not to continuously generate the built module formats whenever source files change. This is most useful if you've linked your library into a host application (with [`npm link`](https://docs.npmjs.com/cli/link) or [`yarn link`](https://yarnpkg.com/lang/en/docs/cli/link/)).
 
 Optional. Defaults to `false`.
 

docs/api/create.md

@@ -1,9 +1,12 @@
 # `create()` Documentation
 
+> NOTE: `create()` is still under development
+
+
 Creates a new library with the specified name set up with infrastructure using `@benmvp/cli`, returning a `Promise` indicating whether the creation succeeded or failed.
 
-- Add `"test"`, `"start"`, and `"build"` scripts in the `package.json` to call [`benmvp test`](test.md), [`benmvp start`](start.md), and [`benmvp build`](build), respectively
-- After the `package.json` is created (or updated), it will install `@benmvp/cli` as a dev dependency, using [Yarn](https://yarnpkg.com/) if available. If Yarn is unavailable, it will fallback to [NPM](https://docs.npmjs.com/)
+- Add `"test"`, `"start"`, `"build"` and `"integrate"` scripts in the `package.json` to call [`benmvp test`](test.md), [`benmvp start`](start.md), [`benmvp build`](build.md), and [`benmvp integrate`](integrate.md), respectively
+- After the `package.json` is created (or updated), it will install `@benmvp/cli` as a dev dependency, using [Yarn](https://yarnpkg.com/) if available. If Yarn is unavailable, it will fallback to [npm](https://docs.npmjs.com/)
 - Will add (or overwrite) a `.travis.yml` file w/ [build stages](https://docs.travis-ci.com/user/build-stages/) for testing and deploying the library
 
 Looking for CLI docs? View companion [`benmvp create` documentation](../cli/create.md).
@@ -45,7 +48,7 @@ Add custom setup to an existing library:
 import {create} from '@benmvp/cli'
 
 create({
-  modes: ['type', 'unit'],
+  modes: ['type', 'spec'],
   out: './built',
   formats: ['esm', 'cjs'],
 })
@@ -56,7 +59,7 @@ create({
 `create()` has the following [TypeScript](https://www.typescriptlang.org/) signature:
 
 ```js
-([options]: Options): Promise<Result>
+(options?: Options): Promise<Result>
 ```
 
 ## Options
@@ -103,11 +106,11 @@ An `Array` of the types or modes of tests to run. Available modes:
 
 - `'type'` - Runs Typescript type-checking
 - `'lint'` - Runs ESLint
-- `'unit'` - Runs Jest-based unit tests
+- `'spec'` - Runs Jest-based tests
 
 Optional. Defaults to all modes when unspecified.
 
-This will initialize the `"start"` and `"test"` scripts in the `package.json` to pass the matching argument.
+This will initialize the `"start"`, `"test"` and `"integrate"` scripts in the `package.json` to pass the matching argument.
 
 
 ## Return Value

docs/api/integrate.md

@@ -0,0 +1,117 @@
+# `integrate()` Documentation
+
+Runs additional integration tests for the library, returning a `Promise` indicating whether the test run succeeded or failed.
+
+> NOTE: `integrate()` assumes the integration tests live within the `integration-tests/src` folder of the current working directory where the script is being called.
+
+The integration tests generally are run in your continuous integration (CI) environment to verify that your library when packaged can successfully be used by clients. The goal is dummy library/project in which you write tests that import and use the library like a normal client would.
+
+The integration test process is as follows:
+
+1. [`npm pack`](https://docs.npmjs.com/cli/pack.html) the library to create the _same_ `.tgz` tarball that would be published in the registry
+1. Copy the integration tests "project" at `integration-tests/` over to a temporary directory
+1. `npm install` the packed library (from Step 1) and any other dependencies specified in the `package.json` of the project
+1. Run `npx benmvp test` on the project to run the tests
+
+Looking for CLI docs? View companion [`benmvp integrate` documentation](../cli/integrate.md).
+
+## Examples
+
+To run all modes of integration tests on all files (default behavior):
+
+```js
+import {integrate} from '@benmvp/cli'
+
+integrate()
+```
+
+To run just the integration tests themselves (excluding linting & typing) on all files:
+
+```js
+import {integrate} from '@benmvp/cli'
+
+integrate({
+  modes: ['spec'],
+})
+```
+
+To run just linting & typing on all files in the integration tests project:
+
+```js
+import {integrate} from '@benmvp/cli'
+
+integrate({
+  modes: ['lint', 'type'],
+})
+```
+
+To run all modes only on files within `utils/` directories of the integration tests project:
+
+```js
+import {integrate} from '@benmvp/cli'
+
+integrate({
+  pattern: 'utils/',
+})
+```
+
+To just run linting on files within `api/` directories of the integration tests project:
+
+```js
+import {integrate} from '@benmvp/cli'
+
+integrate({
+  modes: ['lint'],
+  pattern: 'api/',
+})
+```
+
+## Signature
+
+`integrate()` has the following [TypeScript](https://www.typescriptlang.org/) signature:
+
+```js
+type Mode = 'spec' | 'type' | 'lint'
+namespace TestOptions {
+  modes: Mode[];
+  pattern: string;
+}
+
+(options?: TestOptions): Promise<Result>
+```
+
+## Options
+
+The optional `TestOptions` object supports the following properties:
+
+### `modes`
+
+An `Array` of the types or modes of tests to run. Available modes:
+
+- `'spec'` - Runs Jest-based tests (files ending in `.spec.ts` or in `__tests__` folder)
+- `'type'` - Runs Typescript type-checking (files ending in `.ts`)
+- `'lint'` - Runs ESLint (files ending in `.ts`)
+
+Optional. Defaults to all modes when unspecified. 
+
+### `pattern`
+
+A regexp pattern string that is matched against all tests paths before executing the test.
+
+Optional. Defaults to `''` (signifying no filter)
+
+## Return Value
+
+`integrate()` returns a `Promise`.
+
+When `integrate()` finishes successfully, the resolved value will be an `object` with a `code` property set to `0`.
+
+If `integrate()` exits unsuccessfully, the resolved value will be an `object` with a non-zero `code` property, a user-friendly `message` property, and an `error` property pointing to the inner exception.
+
+---
+
+## More help
+
+Looking for CLI docs? View companion [`benmvp integrate` documentation](../cli/integrate.md).
+
+Still unsure of how to use `@benmvp/cli`? [Ask for help](https://github.com/benmvp/benmvp-cli/issues)!

docs/api/run.md

@@ -29,15 +29,15 @@ To run just unit tests:
 ```js
 const {run} = require('@benmvp/cli');
 
-run(['test', '--modes', 'unit'])
+run(['test', '--modes', 'spec'])
 ```
 
 ## Type
 
 `run()` has the following [TypeScript](https://www.typescriptlang.org/) signature:
 
 ```js
-([args]: Array<string>): Promise<Result>
+(args?: string[]): Promise<Result>
 ```
 
 ## Arguments