Backend v2 #4

Aerilym · 2024-11-20T06:30:51Z

https://gist.github.com/Aerilym/1a541dbb923498b04c89233293e02936

TODO: docs

… backup support

…ds handler

Fix: getLogs limits, request chunking, and daily rewards

feat: add price range endpoint

…egacy addresses

Fix/limits v2

Copilot

Pull Request Overview

This PR implements a backend v2 architecture that completely removes legacy components and introduces a new web3 client system for smart contract interaction. The changes include removing the old timer and testnet modules, and adding a comprehensive event scanning and smart contract integration framework.

Complete removal of legacy timer and testnet modules
Introduction of a new web3 client with contract factory and event scanning capabilities
Implementation of WebSocket-based event monitoring for smart contracts

Reviewed Changes

Copilot reviewed 92 out of 109 changed files in this pull request and generated 8 comments.

File	Description
timer.py	Completely removed legacy uwsgi timer functionality
testnet.py	Removed testnet configuration module
src/web3client/*	New comprehensive web3 client implementation with event scanning, contract interfaces, and WebSocket support

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-08-19T23:53:57Z

src/web3client/event_ws.py

+
+    event_queue = EventQueueManager(w3=w3, log=log, start_block=start_block, max_run_depth=config.ws_max_run_depth, get_logs_cap=config.get_logs_cap)
+
+    if len(config.vesting_contract_details) > 0 :


Remove the extra space before the colon. Python style guide recommends no space before colons in conditionals.

Suggested change

if len(config.vesting_contract_details) > 0 :

if len(config.vesting_contract_details) > 0:

Copilot · 2025-08-19T23:53:58Z

src/web3client/contracts_ws/service_node_contribution_factory.py

+        for event in events:
+            existing_topic = self.topic_map.get(event().topic)
+            if existing_topic is None:
+                print(f"Adding topic f{event.name}: {event().topic}")


Use logging instead of print statements. There appears to be a logging object available as self.log that should be used for consistency with the rest of the codebase.

Suggested change

print(f"Adding topic f{event.name}: {event().topic}")

self.log.debug(f"Adding topic {event.name}: {event().topic}")

Copilot · 2025-08-19T23:53:58Z

src/web3client/event_queue_manager.py

+            get_logs_calls = 0
+            for e in queue:
+                from_block = e.start_block if e.start_block else start_block
+                e.get_logs_calls=(ceil((block_current - from_block) / self.get_logs_cap) if self.get_logs_cap > 0 else 1)


Add spaces around the assignment operator for better readability: e.get_logs_calls = (

Suggested change

e.get_logs_calls=(ceil((block_current - from_block) / self.get_logs_cap) if self.get_logs_cap > 0 else 1)

e.get_logs_calls = (ceil((block_current - from_block) / self.get_logs_cap) if self.get_logs_cap > 0 else 1)

Copilot · 2025-08-19T23:53:58Z

src/web3client/contract_factory.py

+
+            abis[name] = self.w3.eth.contract(abi=compiled_sol[key]["abi"])
+        return abis[name]
+


[nitpick] Consider extracting the contract key mapping logic into a separate method to reduce complexity in the get() method and improve maintainability.

Suggested change

key = self._get_contract_key(name)

abis[name] = self.w3.eth.contract(abi=compiled_sol[key]["abi"])

return abis[name]

def _get_contract_key(self, name: str) -> str:

if name == "Ownable2StepUpgradeable":

return "node_modules/@openzeppelin/contracts-upgradeable/access/Ownable2StepUpgradeable.sol:Ownable2StepUpgradeable"

elif name == "PausableUpgradeable":

return "node_modules/@openzeppelin/contracts-upgradeable/utils/PausableUpgradeable.sol:PausableUpgradeable"

elif name == "IERC1967":

return "node_modules/@openzeppelin/contracts/interfaces/IERC1967.sol:IERC1967"

elif name == "TokenVestingStaking":

return "utils/TokenVestingStaking.sol:TokenVestingStaking"

elif name == "TokenVestingNoStaking":

return "utils/TokenVestingNoStaking.sol:TokenVestingNoStaking"

else:

return f"{name}.sol:{name}"

Copilot · 2025-08-19T23:53:58Z

src/web3client/event_ws.py

+async def load_vesting_staking_contracts(w3: AsyncWeb3, details: list[VestingContractDetails]):
+    contract_interface = TokenVestingStaking(w3=w3, db_writer=global_db_writer, log=log, event_queue=event_queue)
+    #token_contract = Token(w3=w3, db_writer=global_db_writer, log=log).factory(details[0].SESH)
+


Remove commented-out code. If this code is needed for future reference, it should be documented in comments or tracked in version control history.

Suggested change

Copilot · 2025-08-19T23:53:59Z

src/web3client/event_ws.py

+            # assert SESH == known_details.SESH, f"Expected {known_details.SESH}, got {SESH}"
+            # assert rewards_contract == known_details.rewards_contract, f"Expected {known_details.rewards_contract}, got {rewards_contract}"
+            # assert sn_contrib_factory == known_details.sn_contrib_factory, f"Expected {known_details.sn_contrib_factory}, got {sn_contrib_factory}"
+


Remove the large block of commented-out code (lines 128-159). This makes the code harder to read and maintain. If this logic is needed for future implementation, document it properly or track it separately.

Suggested change

res = []

contracts = []

for known_details in details:

Copilot · 2025-08-19T23:53:59Z

src/web3client/contracts_ws/service_node_contribution.py

+        )
+
+    batch_items = 7
+    # TODO: use this to get details for old contracts


The TODO comment is too vague. Provide more specific information about what needs to be implemented or when this method should be used.

Suggested change

# TODO: use this to get details for old contracts

# TODO: Implement logic to retrieve and process contract details for legacy (pre-upgrade) service node contracts.

# This method should be used during migration or auditing tasks to collect parameters, operator info,

# BLS pubkeys, contributions, status, manual finalize state, and reserved contributors from older contracts.

Copilot · 2025-08-19T23:53:59Z

src/web3client/event_ws.py

+        # assert self.transferable_beneficiary is not None
+        # assert is_checksum_address(self.revoker)
+        # assert is_checksum_address(self.SESH)
+        # assert is_checksum_address(self.rewards_contract)


Replace vague TODO comments with specific actionable items or remove them if the decisions have been made. These comments don't provide useful guidance for future development.

Suggested change

# assert is_checksum_address(self.rewards_contract)

Aerilym added 11 commits November 20, 2024 17:20

feat: create logger, performance tracker and time keeper

6469528

feat: web3 client and contract function

b002be3

feat: eth event scanner

c44989c

feat: registration api

5563aea

chore: move oxen and omq logic to package

00f8b51

feat: db manager

1f4fec3

feat: registrations reader

8f19288

feat: overhaul config

e7e14c2

feat: create fetcher

9968ad7

feat: read only api

5d549b5

chore: update README.md

a653a36

Aerilym changed the title ~~Backend v2~~ [WIP] Backend v2 Nov 20, 2024

Aerilym added 18 commits November 21, 2024 14:11

fix: config rpc validation

b552013

fix: omq rpc test

65c01d6

fix: rpc shared config var

83c2f92

fix: perf logger and rpc arg call

1cb7630

fix: add self logger as non if perf is off

cfd8aa6

fix: performance logger

e095381

feat: add median operator fee to network info api endpoint

4317c28

feat: add api route to get stakes by sn pubkey

906328c

feat: add arbitrum network info and event endpoints

aaa27e7

fix: change arbiturm provider url to list of urls for later adding of…

92e0641

… backup support

feat: add rewards fetching and endpoints to fetcher and api

cd10d49

chore: move registration getter api endpoints to main api

e16f1f7

fix: update db schema with arbitrum info and rewards info tables

7949251

feat: add timestamps to events

a51fd12

feat: add node counts to network info

b8993c1

feat: add node exit list parsing to fetcher

6f4fee3

fix: rewards address parsing

e208a27

fix: add contribution contract api endpoints

2f41817

Aerilym and others added 28 commits April 2, 2025 14:50

fix: rework price api to simplify token fetching behaviour

c1451e4

fix: add db read mode utils

0c75a79

feat: add ws contract classes

aaa1555

fix: truncate price stale time

ad32a25

feat: add precision to coingecko api

e6decaa

feat: add dynamic price cache management to price api

d274c55

wip: to clean

485394c

Update readme and requirements with latest dependencies

2f1b496

fix: update token contracts submodule to use the foundation repo

aab080d

fix: update requirements.txt with oxenmq, oxenc, and web3 requirements

1050b82

chore: update config_defaults.py with mainnet values

66b2135

feat: add daily rolling rewards endpoint

f69b82d

feat: add block cap and getLogs chunking to event scanner and new hea…

bcf7848

…ds handler

chore: add utility setup scripts

ef46433

feat: add price range endpoint

195ff24

Merge pull request #7 from session-foundation/fix/limits

8ea7c6e

Fix: getLogs limits, request chunking, and daily rewards

Merge pull request #8 from session-foundation/feat/price_ranges_api

2113bba

feat: add price range endpoint

fix: seeded nodes in fetcher

04127bc

fix: seed node pubkey arg

2ac2091

fix: seed node bls pubkey arg

ed129c4

fix: add and use fetched time for stale time for token prices

4ccd968

fix: add network mock validation for registrations

92b9745

fix: combine new service nodes with seeded service nodes and handle l…

0ecc93f

…egacy addresses

fix: multicontributor contract bootstrapping

3e2104f

fix: checksum vesting contract beneficiary addresses

b9126b9

Merge pull request #9 from session-foundation/fix/limits

5ee40d2

Fix/limits v2

fix: sort arbitrum events in db call

7968e29

feat: add hardfork info and contract nodes endpoints

72a4a7f

Copilot AI review requested due to automatic review settings August 19, 2025 23:52

Copilot AI reviewed Aug 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Backend v2 #4

Backend v2 #4

Uh oh!

Aerilym commented Nov 20, 2024 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Aug 19, 2025

Uh oh!

Copilot AI Aug 19, 2025

Uh oh!

Copilot AI Aug 19, 2025

Uh oh!

Copilot AI Aug 19, 2025

Uh oh!

Copilot AI Aug 19, 2025

Uh oh!

Copilot AI Aug 19, 2025

Uh oh!

Copilot AI Aug 19, 2025

Uh oh!

Copilot AI Aug 19, 2025

Uh oh!

Uh oh!


		event_queue = EventQueueManager(w3=w3, log=log, start_block=start_block, max_run_depth=config.ws_max_run_depth, get_logs_cap=config.get_logs_cap)

		if len(config.vesting_contract_details) > 0 :

	print(f"Adding topic f{event.name}: {event().topic}")
	self.log.debug(f"Adding topic {event.name}: {event().topic}")

	e.get_logs_calls=(ceil((block_current - from_block) / self.get_logs_cap) if self.get_logs_cap > 0 else 1)
	e.get_logs_calls = (ceil((block_current - from_block) / self.get_logs_cap) if self.get_logs_cap > 0 else 1)


		abis[name] = self.w3.eth.contract(abi=compiled_sol[key]["abi"])
		return abis[name]

+            key = self._get_contract_key(name)
+            abis[name] = self.w3.eth.contract(abi=compiled_sol[key]["abi"])
+        return abis[name]
+    def _get_contract_key(self, name: str) -> str:
+        if name == "Ownable2StepUpgradeable":
+            return "node_modules/@openzeppelin/contracts-upgradeable/access/Ownable2StepUpgradeable.sol:Ownable2StepUpgradeable"
+        elif name == "PausableUpgradeable":
+            return "node_modules/@openzeppelin/contracts-upgradeable/utils/PausableUpgradeable.sol:PausableUpgradeable"
+        elif name == "IERC1967":
+            return "node_modules/@openzeppelin/contracts/interfaces/IERC1967.sol:IERC1967"
+        elif name == "TokenVestingStaking":
+            return "utils/TokenVestingStaking.sol:TokenVestingStaking"
+        elif name == "TokenVestingNoStaking":
+            return "utils/TokenVestingNoStaking.sol:TokenVestingNoStaking"
+        else:
+            return f"{name}.sol:{name}"

+        res = []
+        contracts = []
+        for known_details in details:

-    # TODO: use this to get details for old contracts
+    # TODO: Implement logic to retrieve and process contract details for legacy (pre-upgrade) service node contracts.
+    #       This method should be used during migration or auditing tasks to collect parameters, operator info,
+    #       BLS pubkeys, contributions, status, manual finalize state, and reserved contributors from older contracts.

Backend v2 #4

Are you sure you want to change the base?

Backend v2 #4

Uh oh!

Conversation

Aerilym commented Nov 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Aerilym commented Nov 20, 2024 •

edited

Loading