Merge branch 'dev' of https://github.com/OpenMined/syft.js

Author: cereallarceny

API-REFERENCE.md

@@ -29,7 +29,7 @@
 
 ## Syft
 
-Syft client for static federated learning.
+Syft client for model-centric federated learning.
 
 ### Parameters
 
@@ -45,16 +45,16 @@ Syft client for static federated learning.
 const client = new Syft({url: "ws://localhost:5000", verbose: true})
 const job = client.newJob({modelName: "mnist", modelVersion: "1.0.0"})
 job.on('accepted', async ({model, clientConfig}) => {
-  // execute training
+  // Execute training
   const [...newParams] = await this.plans['...'].execute(...)
   const diff = await model.createSerializedDiff(newParams)
   await this.report(diff)
 })
 job.on('rejected', ({timeout}) => {
-  // re-try later or stop
+  // Retry later or stop
 })
 job.on('error', (err) => {
-  // handle errors
+  // Handle errors
 })
 job.start()
 ```
@@ -70,7 +70,7 @@ Authenticates the client against PyGrid and instantiates new Job with given opti
     -   `options.modelVersion` **[string][26]** FL Model version.
 
 
--   Throws **any** Error
+-   Throws **any** Error if grid client authentication failed.
 
 Returns **[Promise][28]<[Job][29]>** 
 

README.md

@@ -7,7 +7,9 @@
 ![OpenCollective](https://img.shields.io/opencollective/all/openmined)
 
 <!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
+
 [![All Contributors](https://img.shields.io/badge/all_contributors-8-orange.svg?style=flat-square)](#contributors-)
+
 <!-- ALL-CONTRIBUTORS-BADGE:END -->
 
 # Syft.js
@@ -55,7 +57,7 @@ If you're not using a package manager, you will be able to include Syft.js withi
 
 As a developer, there are few steps to building your own secure federated learning system upon the OpenMined infrastructure:
 
-1. :robot: [Develop ML model and training procedure](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/static-fl/Part%2001%20-%20Create%20Plan.ipynb) (aka `Plan` in PySyft terminology) using [PySyft](https://github.com/OpenMined/PySyft).
+1. :robot: [Develop ML model and training procedure](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/model-centric-fl/Part%2001%20-%20Create%20Plan.ipynb) (aka `Plan` in PySyft terminology) using [PySyft](https://github.com/OpenMined/PySyft).
 1. :earth_americas: Host model and Plans on [PyGrid](https://github.com/OpenMined/PyGrid), which will deal with all the federated learning components of your pipeline.
 1. :tada: Execute the training on the variety of end-user devices using the client library (syft.js, [SwiftSyft](https://github.com/OpenMined/SwiftSyft), [KotlinSyft](https://github.com/OpenMined/KotlinSyft), [PySyft](https://github.com/OpenMined/PySyft)).
 1. :lock: Securely aggregate trained user models in PyGrid.
@@ -96,7 +98,7 @@ job.on('accepted', async ({ model, clientConfig }) => {
   const batches = LOAD_DATA(batchSize);
 
   // Load model parameters.
-  let modelParams = model.params.map(p => p.clone());
+  let modelParams = model.params.map((p) => p.clone());
 
   // Main training loop.
   for (let [data, labels] of batches) {
@@ -127,7 +129,7 @@ job.on('rejected', ({ timeout }) => {
   // Handle the job rejection, e.g. re-try after timeout.
 });
 
-job.on('error', err => {
+job.on('error', (err) => {
   // Handle errors.
 });
 ```
@@ -209,7 +211,7 @@ PyGrid URL will need to be adjusted accordingly in further steps.
 ##### Create Model & Plan
 
 After PyGrid is running, the next step is to create the model and training plan and host them in PyGrid.
-PySyft tutorials include [MNIST example jupyter notebooks](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/static-fl) that guide you through this process.
+PySyft tutorials include [MNIST example jupyter notebooks](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/model-centric-fl) that guide you through this process.
 
 Fire up jupyter notebook in PySyft root folder:
 
@@ -218,7 +220,7 @@ cd ~/fl-demo/PySyft
 jupyter notebook --notebook-dir=$(pwd)
 ```
 
-In the console, you should see URL you should open, or the browser will open automatically. After this, navigate to `examples/tutorials/static-fl` and [run the first notebook](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/static-fl/Part%2001%20-%20Create%20Plan.ipynb). At this point, you can pull down the model and training plan with syft.js. However, if you'd like to see how to execute the plan using the PySyft FL worker, [try running the second notebook](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/static-fl/Part%2002%20-%20Execute%20Plan.ipynb).
+In the console, you should see URL you should open, or the browser will open automatically. After this, navigate to `examples/tutorials/model-centric-fl` and [run the first notebook](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/model-centric-fl/Part%2001%20-%20Create%20Plan.ipynb). At this point, you can pull down the model and training plan with syft.js. However, if you'd like to see how to execute the plan using the PySyft FL worker, [try running the second notebook](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/model-centric-fl/Part%2002%20-%20Execute%20Plan.ipynb).
 
 ##### PyGrid Clean-up
 
@@ -311,6 +313,7 @@ These people were integral part of the efforts to bring syft.js to fruition and
 
 <!-- markdownlint-enable -->
 <!-- prettier-ignore-end -->
+
 <!-- ALL-CONTRIBUTORS-LIST:END -->
 
 ## License

examples/mnist/README.md

@@ -6,7 +6,7 @@ with [PyGrid](https://github.com/OpenMined/pygrid) to train a plan on local data
 ## Quick Start
 
 1. Install and start [PyGrid](https://github.com/OpenMined/pygrid)
-2. Install [PySyft](https://github.com/OpenMined/PySyft) and [execute the "Part 01 - Create Plan" notebook](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/static-fl/Part%2001%20-%20Create%20Plan.ipynb) from `examples/tutorials/static-fl` folder to seed the MNIST plan and model into PyGrid.
+2. Install [PySyft](https://github.com/OpenMined/PySyft) and [execute the "Part 01 - Create Plan" notebook](https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/model-centric-fl/Part%2001%20-%20Create%20Plan.ipynb) from `examples/tutorials/model-centric-fl` folder to seed the MNIST plan and model into PyGrid.
 3. Now back in this folder, execute `npm install`
 4. And then execute `npm start`
 

examples/multi-armed-bandit/README.md

@@ -10,7 +10,7 @@ While this demo is inherently simple, it's easy to see how one could extend it t
 ## Quick Start
 
 1. Install and start [PyGrid](https://github.com/OpenMined/pygrid)
-2. Install [PySyft](https://github.com/OpenMined/PySyft) and run the [Bandit create plan](<https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/static-fl/Part%2003%20-%20Create%20Plan%20(Bandit%20Demo).ipynb>).
+2. Install [PySyft](https://github.com/OpenMined/PySyft) and run the [Bandit create plan](<https://github.com/OpenMined/PySyft/blob/master/examples/tutorials/model-centric-fl/Part%2003%20-%20Create%20Plan%20(Bandit%20Demo).ipynb>).
 3. Now back in this folder, execute `npm install`
 4. And then execute `npm start`
 

src/grid-api-client.js

@@ -4,12 +4,12 @@ import { GRID_ERROR } from './_errors';
 import EventObserver from './events';
 
 const HTTP_PATH_VERB = {
-  'model_centric/get-plan': 'GET',
-  'model_centric/get-model': 'GET',
-  'model_centric/get-protocol': 'GET',
-  'model_centric/cycle-request': 'POST',
-  'model_centric/report': 'POST',
-  'model_centric/authenticate': 'POST'
+  'model-centric/get-plan': 'GET',
+  'model-centric/get-model': 'GET',
+  'model-centric/get-protocol': 'GET',
+  'model-centric/cycle-request': 'POST',
+  'model-centric/report': 'POST',
+  'model-centric/authenticate': 'POST',
 };
 
 export default class GridAPIClient {
@@ -42,10 +42,10 @@ export default class GridAPIClient {
       `Authenticating against ${modelName} ${modelVersion} with ${authToken}...`
     );
 
-    const response = await this._send('model_centric/authenticate', {
+    const response = await this._send('model-centric/authenticate', {
       model_name: modelName,
       model_version: modelVersion,
-      auth_token: authToken
+      auth_token: authToken,
     });
 
     return response;
@@ -56,13 +56,13 @@ export default class GridAPIClient {
       `[WID: ${workerId}] Requesting cycle for model ${modelName} v.${modelVersion} [${ping}, ${download}, ${upload}]...`
     );
 
-    const response = this._send('model_centric/cycle-request', {
+    const response = this._send('model-centric/cycle-request', {
       worker_id: workerId,
       model: modelName,
       version: modelVersion,
       ping: ping,
       download: download,
-      upload: upload
+      upload: upload,
     });
 
     return response;
@@ -74,11 +74,11 @@ export default class GridAPIClient {
     );
 
     const response = await this._sendHttp(
-      'model_centric/get-model',
+      'model-centric/get-model',
       {
         worker_id: workerId,
         request_key: requestKey,
-        model_id: modelId
+        model_id: modelId,
       },
       'arrayBuffer'
     );
@@ -91,12 +91,12 @@ export default class GridAPIClient {
     );
 
     const response = await this._sendHttp(
-      'model_centric/get-plan',
+      'model-centric/get-plan',
       {
         worker_id: workerId,
         request_key: requestKey,
         plan_id: planId,
-        receive_operations_as: 'tfjs'
+        receive_operations_as: 'tfjs',
       },
       'arrayBuffer'
     );
@@ -118,10 +118,10 @@ export default class GridAPIClient {
       `[WID: ${workerId}, KEY: ${requestKey}] Submitting report...`
     );
 
-    const response = await this._send('model_centric/report', {
+    const response = await this._send('model-centric/report', {
       worker_id: workerId,
       request_key: requestKey,
-      diff
+      diff,
     });
 
     return response;
@@ -131,35 +131,35 @@ export default class GridAPIClient {
     const speedTest = new SpeedTest({
       downloadUrl:
         this.httpUrl +
-        '/model_centric/speed-test?worker_id=' +
+        '/model-centric/speed-test?worker_id=' +
         encodeURIComponent(workerId) +
         '&random=' +
         Math.random(),
       uploadUrl:
         this.httpUrl +
-        '/model_centric/speed-test?worker_id=' +
+        '/model-centric/speed-test?worker_id=' +
         encodeURIComponent(workerId) +
         '&random=' +
         Math.random(),
       pingUrl:
         this.httpUrl +
-        '/model_centric/speed-test?is_ping=1&worker_id=' +
+        '/model-centric/speed-test?is_ping=1&worker_id=' +
         encodeURIComponent(workerId) +
         '&random=' +
-        Math.random()
+        Math.random(),
     });
 
     const ping = await speedTest.getPing();
     // start tests altogether
     const [download, upload] = await Promise.all([
       speedTest.getDownloadSpeed(),
-      speedTest.getUploadSpeed()
+      speedTest.getUploadSpeed(),
     ]);
 
     return {
       ping,
       download,
-      upload
+      upload,
     };
   }
 
@@ -182,20 +182,20 @@ export default class GridAPIClient {
 
     if (method === 'GET') {
       const query = Object.keys(data)
-        .map(k => encodeURIComponent(k) + '=' + encodeURIComponent(data[k]))
+        .map((k) => encodeURIComponent(k) + '=' + encodeURIComponent(data[k]))
         .join('&');
       response = await fetch(this.httpUrl + '/' + path + '?' + query, {
         method: 'GET',
-        mode: 'cors'
+        mode: 'cors',
       });
     } else {
       response = await fetch(this.httpUrl + '/' + path, {
         method: 'POST',
         mode: 'cors',
         headers: {
-          'Content-Type': 'application/json'
+          'Content-Type': 'application/json',
         },
-        body: JSON.stringify(data)
+        body: JSON.stringify(data),
       });
     }
 
@@ -229,7 +229,7 @@ export default class GridAPIClient {
       const cleanUp = () => {
         // Remove all handlers related to message.
         this.wsMessageQueue = this.wsMessageQueue.filter(
-          item => item !== onMessage
+          (item) => item !== onMessage
         );
         this.observer.unsubscribe('ws-error', onError);
         this.observer.unsubscribe('ws-close', onClose);
@@ -241,7 +241,7 @@ export default class GridAPIClient {
         reject(new Error('Response timeout'));
       }, this.responseTimeout);
 
-      const onMessage = data => {
+      const onMessage = (data) => {
         if (data.type !== message.type) {
           this.logger.log('Received invalid response type, ignoring');
           return false;
@@ -250,7 +250,7 @@ export default class GridAPIClient {
         resolve(data.data);
       };
 
-      const onError = event => {
+      const onError = (event) => {
         cleanUp();
         reject(new Error(event));
       };
@@ -280,12 +280,12 @@ export default class GridAPIClient {
         this.ws = ws;
         resolve();
       };
-      ws.onerror = event => {
+      ws.onerror = (event) => {
         // couldn't connect
         this._handleWsError(event);
         reject(new Error(event));
       };
-      ws.onclose = event => {
+      ws.onclose = (event) => {
         // couldn't connect
         this._handleWsClose(event);
         reject(new Error('WS connection closed during connect'));