fix: removed failfast false and attempting single vm upload to s3 (#199)

* fix: removed failfast false and attempting single vm upload to s3

* v0.7.0: GH-497 (#211) (#215)

* GH-497: first stamp; roughly moved the core shared dao utils

* GH-497: small refactoring; thinking about nicer facade

* GH-497: probably finely usable now; next I will have to analyze the places where to use the shared method, possibly also to start adjusting the number data types which will be a chaos

* GH-497: added some marks of where dao methods will be replaced and etc; also found quite a lot of dead code

* GH-497: savepoint; starting to experimentally implement the new boundholder

* GH-497: changing by small steps; added the special primitive type (but may not be used); now before adjusting PayableAccount

* GH-497: a small convinient method added; now stuck thinking of a way out of this

* GH-497: pretty dug up - big changes in progress; now finished some types replacement and want to try to integrate the shared methods for dealing with the DB

* GH-497: shared methods roughly integrated; now I will probs need to change threshold time params to take u64

* GH-497: compiling but many failing tests; I screwed a few things I guess :)

* GH-497: new direction of the architecture; key params and obvious sign attribute

* GH-497: interim commit

* GH-497: a few more tests and features hooked up

* GH-497: interim commit

* GH-497: finally in a working state where I can start to think of changing data types in table declarations.

* GH-497: interim commit

* GH-497: DB migration deployed + reorganization in some db utils for testing

* GH-497: advanced migration with realistic converting i64 to blobs almost implemented

* GH-497: migration practically complete now

* GH-497: db-migration satisfactory

* GH-497: a few more tests made pass

* GH-497: a few insignificant tests fulfilled; elaborating the concept of paymnet thresholds - and correction of the given code as it probably had not worked as intended

* GH-497: paid delinquencies tests fixed

* GH-497: backup after a long day; stabilized exact computations upon the sloped part of the payment thresholds, corner limits at the same place, etc.

* GH-497: new delinquency model seems working at the fundamental level

* GH-497: metadata collection optimized

* GH-497: fixed totals for payable and receivable and some small defects elsewhere

* GH-497: fixing tests for blockchain_interface and little cleaning

* GH-497: completed tests for upsert in blob utils

* GH-497: a lot of minor improvements, refactoring

* GH-497: clean-up before I make step toward designing views into payable and receivable tables

* GH-497: first portion of work for masq financials command, advanced

* GH-497: financials command constructor tested

* GH-497: top payable rendering works

* GH-497: works on rendring records continue

* GH-497: setting with all information to be printed works fine

* GH-497: financial command finished at masq

* GH-497: moved dao_utils over under accountent; financials backend fundamental structure is finished

* GH-497: backend: payable top and range query work

* GH-497: receivable work too now, though certain niches are yet to be worked out

* GH-497: top records and custom query mostly finished in the backend, some minor todos are still left out there"

* GH-497: knocked out two more todos in the backend and also repaired masq console output from rowid to hash

* GH-497: maybe last three todos, they were db errors in row queries; also little renaming and testing top records not using usize anymore but u16

* GH-497: taking care of forced changes in integration tests

* GH-497: documentation added; changed imperfections in rather marginal names and structures

* GH-497: late fixes; mainly Clippy

* GH-497: multinode tests repaired

* GH-574: fix clippy warnings

* GH-574: add more rpc urls for polygon mainnet

* GH-497: lint for multinode tests

* GH-497: defaults in rate pack adjusted; making help messages better; slight refactoring

* GH-497: fixing new delinquencies for a corner case with a fully empty db

* GH-497: verify bill payment a little fix in numeric orders

* GH-497: some window dressing

* GH-497: adding safe migration for rate_pack

* GH-497: verify_bill_payments enriched by the 'financials' call

* GH-497: fixing display tools to show just Gwei-pregnant accounts; payable fixed

* GH-497: receivable gwei-wei custom query fixed; engine for this type of query enhanced; accountant mod tests modified accordingly

* GH-497: backend fixed(Accountant/mod, dao_utils)

* GH-497: financials command fixed to work in gwei - if demanded

* GH-497: close to beating the monster; tests failing, errors left over but illustrative so they will lead me

* GH-497: better treatment of decimal numbers

* GH-497: refactoring - new macro added

* GH-497: refactoring...better arrangement and readability

* GH-497: urgent todos are gone

* GH-497: shorthand args implemented

* GH-497: savepoint before implementing regex

* GH-497: temp and masq num params validation rock on, a few todos are left

* GH-497: quite complicated checks and neatening of the users inputs for costum queries

* GH-497: all existing tests in FinancialsCommand repaired

* GH-497: financials command finished

* GH-497: last sweeping in node and masq with allowence of Clippy

* GH-497: fixing bad assumption with an overflow catch; this is moment before documentation check and then auto review

* GH-497: I revised MASQ-UI-v2 docs and help output for the financials command, also enhanced one confusing log in accountant/mod.rs

* GH-497: state after auto review

* GH-497: fix within all.sh

* GH-497: I set up new possible configuration of arguments for the command; also begun implementing new format for displaying the output -- at least top records works in this regard.

* GH-497: new formatting for financials command completed

* GH-497: tuning the commands features; all tests except one passing

* GH-497: help message and planning to eliminate complicated fn with using regex instead

* GH-497-review-two: financialscommand tests are passing after refactoring a lot of code

* GH-497-review-two: details, but savepoint before trying to collect strings first and compute widths out of them

* GH-497: fair simplification; some duplications and repeated computation

* GH-497: deleting unnecessary code and tests in the financials command

* 497-review-two: ordering by balance and age is defined in the backend too...

* GH-497-review-two: custom_query and all its possible suboptions thoroughly tested; code refactored

* GH-497-review-two: custom_query and all its possible suboptions thoroughly tested

* GH-497-review-two: starting minor refactoring to address individual, less important points from the first review, but just few are behind me

* GH-497-review-two: code refactoring in blob utils

* GH-497-review-two: a bit more refactoring + test on no rows returned

* GH-497-TwoInt: I've got a first set of methods or dealing with two integers

* GH-497-TwoInt: db migration things finished

* GH-497-TwoInt: basic structure stands

* GH-497-TwoInt: continuing designing the SQLParamsBuilder and the product

* GH-497: TwoInt: insert part of upsert mostly implemented

* GH-497-TwoInt: first successful pass on the variant of update with overflow

* GH-497-TwoInt: deleting some commented out code left over from the previous version

* GH-497-TwoInt: parametrizing tests and trying to cover or possibilities...so far mainly for plain updates

* GH-497-TwoInt: big int processor completed and cleaned up

* GH-497-TwoInt: started making tests of payable_dao running; added 'strict' kw assertions

* GH-497-TwoInt: first success on top records with devided big int, yeahoo!

* GH-497-TwoInt: payable tests checked and passing (except one, which to be removed maybe); refactoring of compute_financials()

* GH-497-TwoInt: small refactoring in dao_utils.rs

* GH-497-TwoInt: paid delinquencies repaired + some bringing tests back to life

* GH-497-TwoInt: interim commit

* GH-497-TwoInt: repaired more_money_received and some related stuff

* GH-497-TwoInt: our own sqlite scalar functions added to decompose big intigers at speed

* GH-497-TwoInt: added the complicated test for new sqlite fns hook up; cleaning some stuff from the i128 version in receivable dao

* GH-497-TwoInt: reworking mig config into something with more general purpose - InitConfig; interim commit

* GH-497-TwoInt: transformation from migrator config tu superior init_config is done; most of the process of hooking up sqlite functions as well

* GH-497-TwoInt: all prepared for the start of fixing the new_delinquncies() functionality

* GH-497-TwoInt: all about conn_special_setup thoroughly tested since now

* GH-497-TwoInt: receivables finally also fixed, yahooo

* GH-497-TwoInt: finishing at various places; awaiting work at big_int_db_processor...prepared

* GH-497-TwoInt: all checks done in big_int_db_procesor

* GH-497-TwoInt: tore out the mockable code being unnecessary

* GH-497-TwoInt: Clippy + multinode tests clean-up...still a failing test there though

* GH-497-TwoInt: added more entry checks for financials

* GH-497-TwoInt: more refactoring in the financials command

* GH-497-TwoInt: a bit more of refactoring for the financials command

* GH-497-TwoInt: FinancilsCommand response reworked to return a common set of records for both modes, now on the masq side

* GH-497-TwoInt: the backend side also taken care of

* GH-497-TwoInt: continuing refactoring and hardening--but hit a perplexing issue; want to look at previous commits

* GH-497-TwoInt: finally fixed last problems and cleaned up after me

* GH-497-TwoInt: added integration test for the financials command

* GH-497-TwoInt: multinode test adjustment, changes in documentation

* GH-497-TwoInt: auto review

* GH-497-TwoInt: tests for CLI failing for colors in terminal fixed

* GH-497: sort- exchanged for better order-; changes in wording in docs

* GH-497-TwoInt: an anused import in multinodes

* GH-497-from-sec-review: first portion of repairs

* GH-497-from-sec-review: savepoint before big change...trying to unify column formatted writting of accounts and headings

* GH-497-from-sec-review: simplification with one function covering all types of pretty printed lines

* GH-497-from-sec-review: another portion of minor fixes for FinancialsCommand

* GH-497-from-sec-review: rearranging by meaning group

* GH-497-from-sec-review: fixing financials command

* GH-497-from-sec-review: another small portion of fixes, still operating in financials command

* GH-497-from-second-review: after partial refactoring in big_int_db_processor

* GH-497-from-second-review: still fixes in big_int_db_processor

* GH-497-from-second-review: some more stuff, but at the end moment when deciding if BigIntDivider should return Result (I cannot make my mind up now)

* GH-497-from-second-review: finally progressed on better arragement of tests in the damn challenging file

* GH-497-from-second-review: save point when I realized that relative subtracting at the overflow section won't work this way, I need to specify absolute values

* GH-497-from-second-review: main functionality in big_int_db_processor repaired and new tests written; will have to fix some more things though

* GH-497-from-second-review: more tests running fine; one remaining actually - but I kind of hit the wall here, will see

* GH-497-from-second-review: some more improvement, also still following the suggestions

* GH-497-from-second-review: another big portion of repaired things

* GH-497-from-second-review: more fixes, this time Accountant... a lot about PaymentThresholds

* GH-497-from-second-review: fixed last issues...waiting for another portion from the ongoing review

* GH-497-from-second-review: a fex mixed issue corrections and whole payable_dao fixed

* GH-497-from-second-review: all remaining fixes of the recent batch; gonna wait up more of it

* GH-497-from-sec-review: minor fixes...which tried my will in a day that all just sucked

* GH-497-from-sec-review: various fixes: some of those found in db_mig, and rest...; refactoring in persistent configuration

* GH-497: finished enhancement of the design of Migration configuration etc.; going to attempt to remove the unnecessary create_if_necessary flag

* GH-497-from-second-review: the rest of issues taken care of; stuff around db a lot

* GH-497-from-second-review: this is the commit; I didn't add in all the code, now I do-- the rest of issues taken care of; stuff around db a lot

* GH-497-from-second-review: corrections, all good now, upcoming merge of master

* GH-497-from-third-review: first portion of fixes

* GH-497-from-third-review: another small portion of fixes

* GH-497-from-third-review: another fixes: this time big_int_processor...update with overflow tests

* GH-497-from-third-review: half work done regarding elimination of the heavy floating numbers usage; changing it to use big integers for the slope parameter instead; next will have to fix our defined SQL functions

* GH-497-from-third-review: fixed the slope to be computed as integer, avoiding more expensive translations to f64

* GH-497-from-third-review: few more improvements...in testing user defined sql functions, big_int_db_processor interface and testing user values range limits

* GH-497-from-third-review: nearing the finish line...before the rework of payable_exceeded_threshold

* GH-497-from-third-review: payable_exceeded_threshold got proper test coverage; late but better than never

* GH-497-from-third-review: refactoring macros and arranging less important functions for financials into a smart module

* GH-497-from-third-review: formatting

* GH-497-from-third-review: last changes; clean run

* GH-497-from-fourth-review: fixing the interpretting proces about gwei into MASQ operation; added total supply constant and wrote an integ test to fasten it to the true contract; other optimizations in financials in UI

* GH-497-from-fourth-review: corrected Wei to wei and Gwei to gwei

* GH-497-from-fourth-review: a few more fixes from the review

* GH-497-from-fourth-review: last changes as response to the found issues

* GH-497-from-fourth-review: formatting

* GH-497-from-fourth-review: previous review fully answered

* GH-497-from-fifth-review: previous review fully answered

* GH-497: elevating the code version to 0.7.0

* GH-497-from-fifth-rev: last editor work of the text

* GH-497-sixth-review: fixes from previous reviews

* GH-497: version bump

* GH-497-seventh-review: financials command arranged better into separate files

* GH-497-seventh-review: rearrangement for big int utils in accountant/mod

* GH-497-seventh-review: clippy and fixing an integration test in masq

* GH-497-seventh-review: juss a few details after the review

* v0.7.0_GH-497: last fiddling...passing locally

Co-authored-by: Bert <[email protected]>
Co-authored-by: utkarshg6 <[email protected]>

Co-authored-by: Bert <[email protected]>
Co-authored-by: Bert <[email protected]>
Co-authored-by: utkarshg6 <[email protected]>

* Update ci-matrix.yml

* Update ci-matrix.yml

* Update ci-matrix.yml

* fix: merged s3 release into conditional ci-matrix actions

Co-authored-by: Bert <[email protected]>
Co-authored-by: Bert <[email protected]>
Co-authored-by: utkarshg6 <[email protected]>

Author: 4 people

.github/workflows/ci-matrix.yml

@@ -9,17 +9,19 @@ on:
 jobs:
   build:
     strategy:
-      fail-fast: false
+      fail-fast: true
       matrix:
         target:
-          - { name: linux, os: ubuntu-latest }
-          - { name: macos, os: macOS-latest }
-          - { name: windows, os: windows-latest }
+          - { name: linux, os: ubuntu-22.04 }
+          - { name: macos, os: macos-12 }
+          - { name: windows, os: windows-2022 }
 
     name: Build node on ${{ matrix.target.os }}
     runs-on: ${{ matrix.target.os }}
     steps:
       - uses: actions/checkout@v3
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
       - name: Stable with rustfmt and clippy
         uses: actions-rs/toolchain@v1
         with:
@@ -36,8 +38,7 @@ jobs:
             ~/.cargo/registry/cache/
             ~/.cargo/git/db/
             target/
-          key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
-          restore-keys: ${{ runner.OS }}-cargo-
+          key: ${{ runner.os }}-cargo
       - name: Build ${{ matrix.target.os }}
         run: |
           ./ci/all.sh
@@ -49,3 +50,53 @@ jobs:
         with:
           name: Node-${{ matrix.target.name }}
           path: results
+  deploy_to_s3:
+    needs: build
+    strategy:
+      matrix:
+          os: [linux, macos, windows]
+    runs-on: ubuntu-22.04
+    steps:
+        - uses: actions/checkout@v3
+          with:
+            ref: ${{ github.event.pull_request.head.sha }}
+
+        - uses: actions/download-artifact@v3
+
+        - name: Display structure of downloaded files
+          run: ls -R
+
+        - if: startsWith(github.head_ref, 'GH')
+          name: Versioned S3 Sync
+          uses: jakejarvis/[email protected]
+          with:
+           args: --acl private --follow-symlinks --delete
+          env:
+            AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
+            AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+            AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+            AWS_REGION: 'us-west-2'
+            DEST_DIR: 'Node/${{ github.event.workflow_run.head_branch }}/Node-${{ matrix.os }}'
+            SOURCE_DIR: 'Node-${{ matrix.os }}/generated/bin/'
+
+        - if: startsWith(github.head_ref, 'v')
+          name: Latest S3 Sync
+          uses: jakejarvis/[email protected]
+          with:
+           args: --acl private --follow-symlinks --delete
+          env:
+            AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
+            AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+            AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+            AWS_REGION: 'us-west-2'
+            DEST_DIR: 'Node/latest/Node-${{ matrix.os }}'
+            SOURCE_DIR: 'Node-${{ matrix.os }}/generated/bin/'
+
+        - name: Invalidate Binaries CloudFront
+          uses: chetan/[email protected]
+          env:
+            DISTRIBUTION: ${{ secrets.DISTRIBUTION }}
+            PATHS: "/Node*"
+            AWS_REGION: "us-west-2"
+            AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+            AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}