Skip to content

Commit 3059d47

Browse files
authored
Merge branch 'develop' into patch-1
2 parents 081b605 + 6e1dcd2 commit 3059d47

File tree

5 files changed

+90
-17
lines changed

5 files changed

+90
-17
lines changed

.github/workflows/branch-release.yml

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ jobs:
4646
GIT_BRANCH: ${{ steps.set_vars.outputs.GIT_BRANCH }}
4747
RELEASE_BRANCH_PATTERN: ${{ steps.set_vars.outputs.RELEASE_BRANCH_PATTERN }}
4848
RELEASE_TAG: ${{ steps.set_vars.outputs.RELEASE_TAG }}
49+
GIT_BRANCH_UNFORMATTED: ${{ steps.branch-name.outputs.current_branch }}
4950
steps:
5051
- name: Get branch name
5152
id: branch-name
@@ -114,9 +115,19 @@ jobs:
114115
token: ${{ secrets.GITHUB_TOKEN }}
115116
env: pr-${{ github.event.number }}
116117

118+
generate-types:
119+
name: Generate types
120+
needs: [ init, build-and-test ]
121+
uses: ./.github/workflows/reusable-generate-types.yml
122+
secrets: inherit
123+
with:
124+
branch: ${{ needs.init.outputs.GIT_BRANCH_UNFORMATTED }}
125+
parachainDocker: ${{ github.event.inputs.parachainDocker }}
126+
globalVersion: ${{ needs.init.outputs.GLOBAL_VERSION }}
127+
117128
run-e2e-test:
118129
name: Run e2e tests
119-
needs: [init, validate-branch-name, build-and-test]
130+
needs: [init, validate-branch-name, build-and-test, generate-types]
120131
uses: ./.github/workflows/reusable-e2e-tests.yml
121132
secrets: inherit
122133
permissions:

.github/workflows/reusable-build-and-test.yml

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -30,13 +30,13 @@ permissions:
3030
jobs:
3131
build-node-image:
3232
name: Build Docker image
33-
runs-on: [compile]
33+
runs-on: [compile-gke]
3434
container:
3535
image: ${{ inputs.builder_image }}
36-
# env:
37-
# DOCKER_HOST: "unix:///run/docker/docker.sock"
38-
# volumes:
39-
# - /run/docker:/run/docker
36+
env:
37+
DOCKER_HOST: "unix:///run/docker/docker.sock"
38+
volumes:
39+
- /run/docker:/run/docker
4040
steps:
4141
- uses: actions/checkout@v3
4242
- name: Authenticate to Google Cloud
@@ -150,7 +150,7 @@ jobs:
150150

151151
unit-test:
152152
name: Unit tests
153-
runs-on: [compile]
153+
runs-on: [compile-gke]
154154
container:
155155
image: ${{ inputs.builder_image }}
156156
steps:
@@ -178,7 +178,7 @@ jobs:
178178

179179
coverage-report:
180180
name: Coverage report
181-
runs-on: [compile]
181+
runs-on: [compile-gke]
182182
container:
183183
image: ${{ inputs.builder_image }}
184184
options: --security-opt seccomp=unconfined
@@ -243,7 +243,7 @@ jobs:
243243

244244
build-and-run-try-runtime:
245245
name: Run try-runtime checks
246-
runs-on: [compile]
246+
runs-on: [compile-gke]
247247
container:
248248
image: ${{ inputs.builder_image }}
249249
steps:
@@ -280,7 +280,7 @@ jobs:
280280
run-benchmarks:
281281
name: Run runtime benchmarks
282282
# `performance` self-hosted runners have 8 cores and 16GB of RAM
283-
runs-on: [compile]
283+
runs-on: [performance-gke]
284284
env:
285285
STEPS: 2
286286
REPEATS: 1
@@ -346,7 +346,7 @@ jobs:
346346

347347
build-wasms:
348348
name: Export wasm artifacts
349-
runs-on: [compile]
349+
runs-on: [compile-gke]
350350
env:
351351
STEPS: 2
352352
REPEATS: 1

.github/workflows/reusable-e2e-tests.yml

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ jobs:
126126
- command: "yarn test-parallel-3rdPartyRewards"
127127
fast: true
128128

129-
runs-on: [compile]
129+
runs-on: [e2e-gke]
130130
timeout-minutes: 180
131131
env:
132132
API_URL: "ws://127.0.0.1:9946"
@@ -288,10 +288,6 @@ jobs:
288288
run: |
289289
docker kill $(docker ps -q) 2>/dev/null && echo $?
290290
docker-compose down -v
291-
292-
- name: Fix permissions on self-hosted runner
293-
if: always()
294-
run: chown -R 1100:1100 $GITHUB_WORKSPACE
295291
296292
test-complete:
297293
needs: [setup-report, e2e-test-matrix]

.github/workflows/reusable-perfomance-tests.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ jobs:
2222
performance-tests:
2323
# Allows to keep e2e tests jobs running even if performance-tests fail
2424
continue-on-error: true
25-
runs-on: [compile]
25+
runs-on: [compile-gke]
2626
timeout-minutes: 180
2727
env:
2828
ENV_REF: ${{ inputs.targetEnv || format('pr-{0}', github.event.number) }}

Mangata.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Block construction/execution
2+
Mangata implements front-running bots prevention on automated market maker crypto exchange. For that reason it was decided to split block production and block execution into two following blocks. Execution of the transaction included in block N is delayed by 1 block comparing to origin substrate implementation. It is following block producer (N+1) who calculates the order of transactions execution from the previous block. This way none is able to foresee in what order transactions will be performed.
3+
4+
Affected creates:
5+
- [sc-block-builder](https://docs.rs/sc-block-builder/0.8.0/sc_block_builder/index.html) - execution of extrisnics from previous block on block creation
6+
7+
# Shuffling
8+
Extrinsics are randomly shuffled but still preservers original order per every account - motivation for such algorithm was the fact that following transactions from a given account may depend on other transactions from the same account.
9+
10+
11+
Origin order:
12+
```
13+
| Alice TX1 | Alice TX2 | Alice TX3 | Bob TX1 | Bob TX2 |
14+
```
15+
example shuffled order:
16+
17+
```
18+
| Alice TX1 | Bob TX1 | Alice TX2 | Bob TX2 | Alice TX3 |
19+
```
20+
21+
As shuffling occurs on block execution not creation, every node needs to calculate the same order and agree on storage state afterwards. That is achieved using Fisher-Yates shuffling algorithm with Xoshiro256++ as PRNG initialized with seed stored in blockchain storage. Usage of sr25519 key pair together with VRF capabilities guarantees that block producer cannot impact execution order and it's easy to validate if the seed was properly calculated by validators/following nodes (thanks to VRF verification that only requires block author public key). Seed itself is stored into the storage and is publically accessible through dedicated runtime API [RandomSeedApi](https://github.com/mangata-finance/mangata-node/blob/59b8e6d27c76f89cddbad777ffbeafd1d7f86297/pallets/random-seed/runtime-api/src/lib.rs).
22+
23+
PRNG initialization seed for block `N-1` is signature of 'input message' signed with block producer private key where 'input message' is defined as:
24+
- Seed from block `N-1`
25+
- babe epoch randomness (changed in every epoch)
26+
27+
There is no way for block producer to manipulate seed value in order to impact execution order of extrinsics - also block producer is not able to calculate seed for a block he is going to author sooner than N-1 block is produced.
28+
29+
Seed value is injected into the block as [InherentData](https://docs.rs/sp-inherents/2.0.0/sp_inherents/struct.InherentData.html) by Babe consensus protocol. Whether block producer finds out it's his turn to create a block new seed is being calculated. Seeds value is validated (VRF verify) by other nodes when new block is being imported - in case of seed calculation violation block will be rejected.
30+
31+
32+
Affected crates
33+
- [sc-block-builder](https://docs.rs/sc-block-builder/0.8.0/sc_block_builder/index.html) - extracting seed value on inherents creation
34+
- [sc-basic-authorship](https://docs.rs/sc-basic-authorship/0.8.0/sc_basic_authorship/index.html) - shuffling extrinsics before passing them to block builder
35+
- [sc-consensus-babe](https://docs.rs/sc-basic-authorship/0.8.0/sc_basic_authorship/index.html) - calcuating and injecting shuffling seed value into InherentData, shuffling seed verification
36+
- [sc-service](https://docs.rs/sc-service/0.8.0/sc_service/index.html) - fetching seed value and extrinsic shuffling for 'following nodes'
37+
#Tokens
38+
39+
Mangata uses concept of liquidity tokens. One can create a liquidity pool(together with liquidity pool token) using a pair of tokens and then use the tokens of that pool to stake. New currency trait [`MultiTokenCurrency`](https://github.com/mangata-finance/mangata-node/blob/0846c42a7b7fd29e19fd1b30043ddb3b55a8f250/pallets/tokens/src/multi_token_currency.rs#L14) was introduced and integrated with [staking pallet](https://github.com/mangata-finance/mangata-node/tree/59b8e6d27c76f89cddbad777ffbeafd1d7f86297/pallets/staking). Origin implementation was not sufficient as it only works with single, statically defined currency in our case we require supporting multiple dynamically created currencies.
40+
41+
42+
Affected crates
43+
- [orml-tokens](https://docs.rs/orml-tokens/0.3.1/orml_tokens/index.html) - Introduced new `MultiTokenCurrency` trait, using it as `Balances` replacement
44+
- [pallet-staking](https://docs.rs/pallet-staking/2.0.0/pallet_staking/index.html) - Alining with `MultiTokenCurrency` trait.
45+
46+
#JS API
47+
Because of shuffled delayed transaction execution same applies to events triggered by extrinsics. Origin JS API is not sufficient to map extrinsics to triggered events. Link between transaction <-> event is done based on transaction/events order. In Mangata order of extrinsics inside the block differs from their actual execution order. For that reason dedicated async API methods were introduced to provide missing functionality. When transaction is sent:
48+
1. API waits for notification that it has been included in block N
49+
2. API fetches block N and N+1
50+
3. API stores information about list of events produced in block N+1
51+
3. API reads seed from block N+1 and calculate execution order of extrinsics from block N
52+
4. API maps events to shuffled extrinsics list
53+
54+
from API client perspective it's as easy as
55+
```
56+
signAndWaitTx(
57+
api.tx.tokens.transfer(to, asset_id, amount),
58+
from,
59+
nonce
60+
).then( (events) => {
61+
events.forEach((e) => {
62+
console.log(e.toHuman())
63+
});
64+
})
65+
66+
```

0 commit comments

Comments
 (0)