Testing cf-abacus-dataflow failed.

[RadhikaKS]

Hi Team,

I am just trying to clone cf-abacus and build and run it locally.

yarn run build .. fails at this point.

yarn -v : 1.3.2
node -v: v8.9.4

Testing...

abacus-dataflow
√ log input (110ms)
dataflow mapper
√ runs a map transform and stores inputs and outputs (1140ms)
√ expects map func to be called with system scope (63ms)
√ save to output db when it gets duplicate (409) (781ms)
√ does not store map outputs when error exists (610ms)
√ detects error when there is no sink (1015ms)
√ propagates mapper error from the sink, does not store to output (594ms)
√ limits db connections to error db (94ms)
on error
√ stores map errors to error db (719ms)
√ removes mapper error docs from db (1203ms)
with documents missing in the output db
√ replays data (469ms)
dataflow reducer
1) runs a reduce transform and stores its inputs and outputs
2) save to output db when it gets duplicate (409)
3) runs a reduce transform and down samples outputs
4) does not store reduce outputs when error exists
5) detects error when there is no sink
6) propagates reducer error from the sink and does not store to output
√ does batch post, and gather the errors (282ms)
√ limits db connections to error db (109ms)
on error
7) stores reducer errors to error db
8) removes reducer errors docs from db
with documents missing in the output db
9) replays data
dataflow sink
√ returns correct URI
√ distributes sampled db data evenly (94ms)
√ distributes random orgs evenly (187ms)
√ organization sticks to a node (187ms)

17 passing (49s)
9 failing

1. abacus-dataflow dataflow reducer runs a reduce transform and stores its inputs and outputs:
   Uncaught AssertionError: expected 't/0001517381507772-0-0-0-1/k/4' to match /t/[0-9].*-0-0-0/k/4/
   at postspy (src\test\test.js:1656:31)
   at Object.batch_post (src\test\test.js:54:29)
   at Timeout.flush [as _onTimeout] (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\abacus-batch\src\index.js:79:5)

2. abacus-dataflow dataflow reducer save to output db when it gets duplicate (409):
   Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it r
   esolves.

3. abacus-dataflow dataflow reducer runs a reduce transform and down samples outputs:
   Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it r
   esolves.

4. abacus-dataflow dataflow reducer does not store reduce outputs when error exists:
   Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it r
   esolves.

5. abacus-dataflow dataflow reducer detects error when there is no sink:
   Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it r
   esolves.

6. abacus-dataflow dataflow reducer propagates reducer error from the sink and does not store to output:
   Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it r
   esolves.

7. abacus-dataflow dataflow reducer on error stores reducer errors to error db:
   Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it r
   esolves.

8. abacus-dataflow dataflow reducer on error removes reducer errors docs from db:
   Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it r
   esolves.

9. abacus-dataflow dataflow reducer with documents missing in the output db replays data:
   Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

Child process exited with code 9
----------|----------|----------|----------|----------|----------------|
File | % Stmts | % Branch | % Funcs | % Lines |Uncovered Lines |
----------|----------|----------|----------|----------|----------------|
All files | 86.94 | 75.46 | 96.26 | 87.87 | |
index.js | 86.94 | 75.46 | 96.26 | 87.87 |... 4,1305,1306 |
----------|----------|----------|----------|----------|----------------|
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Please help.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/154831047

The labels on this github issue will be updated when the story is started.

[hsiliev]

Cannot reproduce the issue on my Mac & Linux machines. Is this reproducible on your side and if yes what is the OS version you have?

You ran the tests on Feb 1st right?

[RadhikaKS]

Hi hsiliev,

OS Version that I have is Windows 10 64 bit OS.
And I am using Git Bash terminal to run the commands.

Node Version : v8.9.4
npm Version : 4.2.0
yarn Version : 1.3.2

Yes I have run this test on Feb 1st.

Is there any other pre requisite for building cf-abacus in local ??? Or any Environment Variables I need to set or cross check ???

Thanks In Advance.

[hsiliev]

@RadhikaKS There are no additional prerequisites for building Abacus.

What happens if you re-rerun Abacus build? Is the problem still there?

[RadhikaKS]

@hsiliev I just tried to Rebuild Abacus locally. Though previously mentioned error is not occurring now, it is failing yet at another point mentioned below...and the build ends.

node_modules/abacus-mocha: yarn test
< node_modules/abacus-cfpush: yarn test
$ abacus-eslint && nyc abacus-mocha

Testing...

Test command line args
√ verify all arguments parsed
√ verify optional arguments
√ verify mandatory arguments

Test abacus cfpush
when application is successfully pushed
1) verify CF_HOME content is copied to tmp dir
2) verify manifest is adjusted
3) verify prepareZdm
4) verify cf push executed
when application push fails
5) verify cf push was retried until retry attempts is reached
6) verify cf push was retried until successful push
with missing app manifest
7) fails

3 passing (97ms)
7 failing

1. Test abacus cfpush when application is successfully pushed verify CF_HOME content is copied to tmp dir:
   AssertError: expected copySync to be called with exact arguments
   at Object.fail (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:96:21)
   at failAssertion (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:55:16)
   at Object.assert.(anonymous function) [as calledWithExactly] (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:80:13)
   at Context.it (src\test\cfpush-test.js:174:14)

2. Test abacus cfpush when application is successfully pushed verify manifest is adjusted:
   AssertError: expected writeFile to be called once but was called 0 times
   at Object.fail (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:96:21)
   at failAssertion (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:55:16)
   at Object.assert.(anonymous function) [as calledOnce] (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:80:13)
   at Context.it (src\test\cfpush-test.js:179:14)

3. Test abacus cfpush when application is successfully pushed verify prepareZdm:
   AssertError: expected exec to be called with exact arguments
   at Object.fail (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:96:21)
   at failAssertion (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:55:16)
   at Object.assert.(anonymous function) [as calledWithExactly] (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:80:13)
   at Context.it (src\test\cfpush-test.js:192:14)

4. Test abacus cfpush when application is successfully pushed verify cf push executed:
   AssertError: expected stub to be called 4 times but was called 0 times
   at Object.fail (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:96:21)
   at failAssertion (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:55:16)
   at Object.assertCallCount [as callCount] (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:137:13)
   at Context.it (src\test\cfpush-test.js:205:14)

5. Test abacus cfpush when application push fails verify cf push was retried until retry attempts is reached:
   AssertionError: expected [Function: runCLI] to throw an error
   at Context.it (src\test\cfpush-test.js:239:37)

6. Test abacus cfpush when application push fails verify cf push was retried until successful push:
   AssertError: expected exec to be called twice but was called 0 times
   at Object.fail (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:96:21)
   at failAssertion (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:55:16)
   at Object.assertCallCount [as callCount] (C:\Users\i338670\Desktop\TestCFAbacus\node_modules\sinon\lib\sinon\assert.js:137:13)
   at verifyPushRetryAttempts (src\test\cfpush-test.js:220:14)
   at Context.it (src\test\cfpush-test.js:250:7)

7. Test abacus cfpush with missing app manifest fails:
   AssertionError: expected [Function: runCLI] to throw an error
   at Context.it (src\test\cfpush-test.js:261:37)

Child process exited with code 7
------------------|----------|----------|----------|----------|----------------|
File | % Stmts | % Branch | % Funcs | % Lines |Uncovered Lines |
------------------|----------|----------|----------|----------|----------------|
All files | 36.22 | 7.14 | 25 | 38.02 | |
src | 41.24 | 12.5 | 29.17 | 42.55 | |
index.js | 41.24 | 12.5 | 29.17 | 42.55 |... 173,174,175 |
src/lib | 20 | 0 | 0 | 22.22 | |
remanifester.js | 20 | 0 | 0 | 22.22 |... 40,41,43,45 |
------------------|----------|----------|----------|----------|----------------|
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I have tried installing ESLint through npm install -g eslint , But still it fails at this point.

[hsiliev]

What you can do is to build Abacus with yarn provision. This will only take care to get all dependencies, without running the tests. Then you should be able to do yarn start and get Abacus running locally.

I'll leave this issue open so we can establish Windows build since now we're running CI on Mac/Linux. That's another option if you're still having problems with the build.

[RadhikaKS]

@hsiliev

FYI...
I tried building abacus with yarn provision and it was success.
But yarn start is not working.

$ yarn start
yarn run v1.3.2
$ bin/start
'bin\start' is not recognized as an internal or external command,
operable program or batch file.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Even if I go till bin folder of cf-abacus and start it , it will start and then again same error I am getting When I try to do yarn run demo

[hsiliev]

You'll have to use *nix system (Linux/MacOS) to get Abacus running at the moment.

We got rid of zip executable requirement (#239) but it seems we have more work to do here.