Backend v2

.gitignore

@@ -1,5 +1,5 @@
-/sent-backend.db*
-/config.py
+/*.db
+/src/config.py
 /__pycache__
 /oxend/*.sock
 */__pycache__

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "session-token-contracts"]
+	path = session-token-contracts
+	url = https://github.com/oxen-io/eth-sn-contracts.git

README.md

@@ -1,4 +1,4 @@
-# SENT Staking Backend
+# Session Staking Backend
 
 ## Running the backend
 
@@ -8,6 +8,7 @@ The `liboxenc-dev` and `liboxenmq-dev` packages require the development headers
 [Oxen Deb Repository](https://deb.oxen.io). Follow those instructions then they can be installed with `apt`.
 
 To run the backend on **Ubuntu >= 24.04**:
+
 ```shell
 apt install build-essential python3-pip python3-dev pybind11-dev liboxenc-dev liboxenmq-dev
 python3 -m pip install eth_utils web3 PyNaCl Flask uWSGI
@@ -20,19 +21,40 @@ Instructions available at:
 - [oxen-pyoxenc](https://github.com/oxen-io/oxen-pyoxenc)
 - [oxen-pyoxenmq](https://github.com/oxen-io/oxen-pyoxenmq)
 
+### Structure
+
+The backend is split into three parts:
+
+- **Fetcher**: `fetcher.py` is the main server that handles all the fetching, processing, and main database writing.
+- **Api**: `api.py` is the main API server that handles most requests.
+- **Registrations API**: `api_registrations.py` is the registration API server that handles all registration requests
+  and registration database management.
+
+You can just run whichever service you want, but the intended usage is to run all three:
+
+- The fetcher will create and update the main database with network, node, contract, and arbitrum information.
+- The API is purely read-only and is a glorified wrapper for the main database with caching.
+- The registration API is used to store and retrieve registration information for nodes.
+
 ### Instance
 
-Before running an instance, `oxend` must be running and its address/smart contracts configured in
-`config.py`.
+Before running the Fetcher or API, `oxend` must be running and its address/smart contracts configured in `config.py`.
 
-It's possible to run the service in flask directly but the timers to poll the smart contracts
-requires WSGI. Both methods are detailed below:
+### Running the Fetcher
+
+The fetcher is a pure python script that runs in a loop and fetches data from the smart contracts and the oxend RPC. You
+can simply run it with `python3 fetcher.py`.
+
+### Running the API
+
+It's possible to run the API in flask directly, but you'll want to use uwsgi in production. Both methods are detailed
+below:
 
     FLASK_APP=sent flask run --reload --debugger
-    uwsgi --http 127.0.0.1:5000 --master -p 4 -w sent --callable app
+    uwsgi --http 127.0.0.1:5000 --master -p 4 -w api --callable app
 
-You may optionally append `--fs-reload sent.py` to the `uwsgi` invocation to
-automatically restart the server when `sent.py` is modified.
+You may optionally append `--fs-reload api.py` to the `uwsgi` invocation to
+automatically restart the server when `api.py` is modified.
 
 After the server is running, visit `127.0.0.1:5000/info` to verify that the server is up and
 responding correctly with a payload like the following:
@@ -53,6 +75,10 @@ responding correctly with a payload like the following:
 }
 ```
 
+### Running the Registrations API
+
+**Follow the same instructions as the API above. Replacing `api` with `registrations` in the commands.**
+
 ## Setting up an oxend instance
 
 The default configuration for mainnet.py, testnet.py, devnet.py look for mainnet.sock, testnet.sock,
@@ -63,7 +89,7 @@ There are a few ways to make this work.
 
 ### Symlinks
 
-You can add symlinks here to existing oxend sockets.  If oxend and the backend code are running as
+You can add symlinks here to existing oxend sockets. If oxend and the backend code are running as
 the same user then you can simply create a symlink:
 
     ln -s /var/lib/oxend/oxend.sock mainnet.sock
@@ -75,7 +101,7 @@ you can make it work, but will need an extra step to configure socket permission
 
 and then do one of:
 
-- Set the active group of the running sent staking backend to the `_loki` user.  Whether this is
+- Set the active group of the running sent staking backend to the `_loki` user. Whether this is
   easy or not depends on how the backend service is running.
 
 - Add the `_loki` group to the supplemental groups of the user that will be running this backend.
@@ -89,14 +115,14 @@ and then do one of:
 
 Do *NOT*:
 
-- Run any production service as root (including under sudo).  Don't be tempted just because "it
+- Run any production service as root (including under sudo). Don't be tempted just because "it
   works" under sudo: by running things under root/sudo you compromise the security of your entire
-  system as a solution to properly setting up permissions.  Please don't do this, ever.
+  system as a solution to properly setting up permissions. Please don't do this, ever.
 
 ### Make oxend create the socket
 
 The `lmq-public=ipc:///path/to/sent-staking-backend/oxend/mainnet.sock` can be added to oxen.conf
-(or run with `--lmq-public=...`) to have it listen on that socket.  Note that when oxend and
+(or run with `--lmq-public=...`) to have it listen on that socket. Note that when oxend and
 sent-staking-backend are running as separate users, this has the same permission issues as the
 Symlinks approach (see above for solutions).
 
@@ -106,3 +132,4 @@ You can configure oxend with `lmq-public=tcp://127.0.0.1:6789` (choose whatever
 place of `6789`) in the oxen.conf config file [alternatively: run oxend with
 `--lmq-public=tcp://127.0.0.1:6789`] and then add/uncomment the `mainnet_rpc=...` (or `testnet_rpc=`
 or `devnet_rpc=`) line in config.py.
+