You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[Simulation on different applications (models)](#simulation-on-different-applications-models)
9
+
-[deploy new model](#1-deploy-model)
10
+
-[store model hash](#2-add-the-model-hash-to-blockchain)
11
+
-[define data handling](#3-change-the-datahandlerpy)
12
+
-[model optimization](#4-change-the-trainpy)
13
+
-[basic config set up](#5-change-the-configpy)
14
+
-[accuracy simulation](#accuracy-simulation)
15
+
-[Federated learning + central server](#federated-learning--central-server)
16
+
-[set up](#1-set-up-1)
17
+
-[run an ipfs node locally](#2-run-ipfs-node)
18
+
-[run the central server](#3-run-a-server-after-running-the-ipfs-daemon)
19
+
-[run training nodes](#4-run-the-training-nodes-1)
20
+
-[downloading training script](#server-for-downloading-the-training-scripts)
21
+
-[set up](#1-set-up-2)
22
+
-[run the server](#2-run-the-server)
3
23
## Federated learning + blockchain + ipfs
4
24
5
25
### **1. Set up**
@@ -14,9 +34,9 @@ pip install -r requirements.txt
14
34
15
35
### **2. Account Preparation:**
16
36
17
-
Since you need to provide the private key to make the transiction, you need to create a file called **accounts.py** at /src/training-scripts/py/[modelName]/ and add your account info
37
+
Since you need to provide the private key to make the transiction, you need to create a file called **accounts.py** at `/src/training-scripts/py/[modelName]/` and add your account info
18
38
19
-
e.g. /src/training-scripts/py/cifar10/accounts.py
39
+
e.g. `/src/training-scripts/py/cifar10/accounts.py`
20
40
21
41
```python
22
42
# this is for single account simulation without the incentive mechanism
@@ -39,11 +59,30 @@ accounts = [
39
59
]
40
60
```
41
61
62
+
> you need to at least have an account for ropsten testnet;
63
+
64
+
**to quickly create an ethereum account, you can use `metaMask` as a `wallet` to create accounts and manage all your accounts**
65
+
66
+
1. go to [chrome web store](https://chrome.google.com/webstore/detail/metamask/nkbihfbeogaeaoehlefnkodbefgpgknn) to install the metaMask plugin.
67
+
2. set up your wallet following the instruction and keep your wallet private key words safe.
68
+
3. after creating a wallet, you can now create an account.
69
+
70
+

71
+
72
+
4. after your account has been created, you can see your `account address` and import the `private key` from metaMask. Besides, one account can be used in `different ethereum networks` including `ropsten testnet`, you can switch to different networks in metaMask as well(the ETH can not be shared through different networks)
73
+
74
+

75
+
76
+
5. then you can go to [Ropsten Testnet Explorer](https://ropsten.etherscan.io/) to get info of your account
77
+
6. lastly, you may want some ETH for the simulation, you can find a [ropsten faucet](https://faucet.egorfine.com/) to request some ETH. There should be a few different faucets. you can also ask me to give you some.
78
+
79
+
> The reason why you need to provide the `account private key` is that we use the [Infura](https://infura.io/dashboard) as a `Provider` to connect to the blockchain. `Provider` can be considered as one of the nodes of a blockchain network. When we do a `state-mutated transaction`, the `Provider` need our account address with private key to execute the transaction.
80
+
> To use the `Infura`, we need to apply a key for each project, currently is using mine, you can change it [here](https://github.com/s-elo/DNN-Blockchain/blob/33a142e431bc5710b8ecdaea2224df94bd9e0727/src/training-scripts/py/cifar10/contract.py#L11)
42
81
### **3. Dataset Preparation:**
43
82
44
83
**\*This is not necessary any more, since the data will be loaded using tensorflow API for quick simulation**
45
84
46
-
Put the training dataset at /src/training-scripts/py/modelName/dataset. Use the names of the classes as the sub-folder name
85
+
Put the training dataset at `/src/training-scripts/py/modelName/dataset`. Use the names of the classes as the sub-folder name
47
86
48
87
### **4. Run the training nodes:**
49
88
@@ -68,14 +107,18 @@ python main.py
68
107
69
108
- The limited number of the nodes and training rounds can be set by modifying the two variables **NODE_NUM** and **ROUND** in the **config.py**.
70
109
110
+
> Note that we have deployed a `contract` and `model` for simulation.
111
+
> The `contract info` you can find from [here](https://ropsten.etherscan.io/address/0xecc03bcae3944ff618787c209d64f8f5cfee1456#code) or at the `/src/training-scripts/py/cirfar10/contract.json`. The code is at [UI branch](https://github.com/s-elo/DNN-Blockchain/blob/contractsAndUI/contracts/PureDnn.sol).
112
+
> The model is deployed to the ipfs through [WebStorage](https://web3.storage/account/)
113
+
> Upload files to ipfs through WebStorage needs a `apiToken`, currently is using mine. you can change it [here](https://github.com/s-elo/DNN-Blockchain/blob/33a142e431bc5710b8ecdaea2224df94bd9e0727/src/ipfs_deploy/deploy.ts#L13)
71
114
## Simulation on different applications (models)
72
115
73
-
If you want to try simulation on differnt models, you can do some changes based on the cifar10 training scripts **/src/training-scripts/py/cirfar10**
116
+
If you want to try simulation on differnt models, you can do some changes based on the cifar10 training scripts `/src/training-scripts/py/cirfar10`
74
117
75
118
### **1. Deploy model**
76
119
77
120
Firstly we need to deploy the new model **(only for tensorflow models)** on the ipfs.
78
-
at /src/ipfs_deploy/ create a new model folder just like **/src/ipfs_deploy/cifar10** which is able to generate the model.json file corresponding to the new model.
121
+
at `/src/ipfs_deploy/` create a new model folder just like `/src/ipfs_deploy/cifar10` which is able to generate the model.json file corresponding to the new model.
79
122
80
123
After getting the model.json file:
81
124
@@ -85,26 +128,26 @@ npm run deploy [new model name(the folder name)]
85
128
86
129
### **2. Add the model hash to blockchain**
87
130
88
-
After deployment, we should be able to get the hash of the model at /src/ipfs_deploy/deploy_hash.json (the last one), then you need to add this hash to the blockchain by updating the smart contract state
131
+
After deployment, we should be able to get the hash of the model at `/src/ipfs_deploy/deploy_hash.json` (the last one), then you need to add this hash to the blockchain by updating the smart contract state
89
132
using the explorer [here](https://ropsten.etherscan.io/address/0xecc03bcae3944ff618787c209d64f8f5cfee1456#writeContract). We should be able to use the **addNewModel** function and add the hash with the model name. The testset_hash can just be any string since we dont get the testset from ipfs for simulation.
90
133
91
134
### **3. Change the dataHandler.py**
92
135
93
-
After the above deployment, you need to customize how to get your training dataset as what the original cifar10 model does at **/src/training-scripts/py/cirfar10/dataHandler.py**. You can choose to load the dataset using tf API or download the dataset manually.
136
+
After the above deployment, you need to customize how to get your training dataset as what the original cifar10 model does at `/src/training-scripts/py/cirfar10/dataHandler.py`. You can choose to load the dataset using tf API or download the dataset manually.
94
137
95
138
### **4. Change the train.py**
96
139
97
-
The train.py at **/src/training-scripts/py/cirfar10/train.py** determines how to compile and optimize the fetched model from ipfs and model evalation which can be also customized.
140
+
The train.py at `/src/training-scripts/py/cirfar10/train.py` determines how to compile and optimize the fetched model from ipfs and model evalation which can be also customized.
98
141
99
142
### **5. Change the config.py**
100
143
101
-
The config.py at **/src/training-scripts/py/cirfar10/config.py** is used to set some simulation parameters, at this point, the must-do change is the MODEL_NAME param to the name of your new deployed model and feel free to try other different params.
144
+
The config.py at `/src/training-scripts/py/cirfar10/config.py` is used to set some simulation parameters, at this point, the must-do change is the MODEL_NAME param to the name of your new deployed model and feel free to try other different params.
102
145
103
146
After that, you should be able to do the simulation using the new application.
104
147
105
148
## Accuracy simulation
106
149
107
-
At the /acc_simul, you can simulate the model using normal training and federated learning
150
+
At the `/acc_simul`, you can simulate the model using normal training and federated learning
108
151
109
152
```bash
110
153
# at /acc_simul/[model name]
@@ -129,7 +172,7 @@ python fl_simul.py
129
172
pip install -r requirements.txt
130
173
```
131
174
132
-
Everything is the same as that for **Build the network using Blockchain** except:
175
+
Everything is the same as that for **Building the network using Blockchain** except:
0 commit comments