-
Notifications
You must be signed in to change notification settings - Fork 15
fix: get pieces from contract instead of pdpServer #249
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SgtPooki
wants to merge
11
commits into
master
Choose a base branch
from
feat/get-piece-leaf-count
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+694
−48
Open
Changes from 3 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
cf89ea3
feat: allow consumers to get leafcount and size of pieces
SgtPooki 96d6a5a
docs: provide example script for using new method
SgtPooki 4c5e0a6
chore: fix lint
SgtPooki 2018a6e
refactor: migrate to getAllActivePieces
SgtPooki aaccbe1
fix: bring back getPiecesWithDetails
SgtPooki 5972649
chore: remove some old code, add comments for reviewers
SgtPooki caaf368
Merge branch 'master' into feat/get-piece-leaf-count
SgtPooki 80bd959
chore: remove getPieceLeafCount from pdpverifier
SgtPooki b8a4028
refactor: migrating to getPieces
SgtPooki d7ab36e
test: getDataSetPieces tests now validate contract functionality
SgtPooki 732a91f
chore: add a note about getActivePieces SDK method
SgtPooki File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,124 @@ | ||
#!/usr/bin/env node | ||
|
||
/** | ||
* Piece Details Example - Demonstrates how to get piece information with leaf counts | ||
* | ||
* This example shows how to use the new piece details functionality to get | ||
* leaf count and calculated raw size information for pieces in your data sets. | ||
* | ||
* The script will: | ||
* 1. Find your data sets | ||
* 2. Get detailed piece information including leaf counts and raw sizes | ||
* 3. Display a summary of all pieces with their calculated sizes | ||
* | ||
* Usage: | ||
* PRIVATE_KEY=0x... node example-piece-details.js | ||
*/ | ||
|
||
import { Synapse } from '@filoz/synapse-sdk' | ||
|
||
const PRIVATE_KEY = process.env.PRIVATE_KEY | ||
const RPC_URL = process.env.RPC_URL || 'https://api.calibration.node.glif.io/rpc/v1' | ||
const WARM_STORAGE_ADDRESS = process.env.WARM_STORAGE_ADDRESS // Optional - will use default for network | ||
|
||
if (!PRIVATE_KEY) { | ||
console.error('ERROR: PRIVATE_KEY environment variable is required') | ||
console.error('Usage: PRIVATE_KEY=0x... node example-piece-details.js') | ||
process.exit(1) | ||
} | ||
|
||
async function main() { | ||
console.log('=== Synapse SDK Piece Details Example ===\n') | ||
|
||
// Create Synapse instance | ||
const synapseOptions = { | ||
privateKey: PRIVATE_KEY, | ||
rpcURL: RPC_URL, | ||
} | ||
|
||
if (WARM_STORAGE_ADDRESS) { | ||
synapseOptions.warmStorageAddress = WARM_STORAGE_ADDRESS | ||
} | ||
|
||
const synapse = await Synapse.create(synapseOptions) | ||
console.log('✅ Synapse instance created') | ||
|
||
// Declare dataSetInfo in the outer scope | ||
let dataSetInfo = null | ||
|
||
try { | ||
// Find data sets with pieces | ||
console.log('\n📊 Finding data sets...') | ||
const dataSets = await synapse.storage.findDataSets() | ||
console.log(`Found ${dataSets.length} data set(s)`) | ||
|
||
if (dataSets.length === 0) { | ||
console.log('❌ No data sets found. Please upload some data first using example-storage-simple.js') | ||
return | ||
} | ||
|
||
// Find a data set with pieces (currentPieceCount > 0) | ||
const dataSetWithPieces = dataSets.find((ds) => ds.currentPieceCount > 0) | ||
if (!dataSetWithPieces) { | ||
console.log('❌ No data sets with pieces found. Please upload some data first using example-storage-simple.js') | ||
return | ||
} | ||
|
||
// Map the data set properties to what we expect | ||
dataSetInfo = { | ||
dataSetId: dataSetWithPieces.pdpVerifierDataSetId, | ||
providerId: dataSetWithPieces.providerId, | ||
pieceCount: dataSetWithPieces.currentPieceCount, | ||
clientDataSetId: dataSetWithPieces.clientDataSetId, | ||
isLive: dataSetWithPieces.isLive, | ||
withCDN: dataSetWithPieces.withCDN, | ||
} | ||
|
||
console.log(`\n📊 Data Set Summary:`) | ||
console.log(` PDP Verifier Data Set ID: ${dataSetInfo.dataSetId}`) | ||
console.log(` Client Data Set ID: ${dataSetInfo.clientDataSetId}`) | ||
console.log(` Provider ID: ${dataSetInfo.providerId}`) | ||
console.log(` Piece Count: ${dataSetInfo.pieceCount}`) | ||
console.log(` Is Live: ${dataSetInfo.isLive}`) | ||
console.log(` With CDN: ${dataSetInfo.withCDN}`) | ||
|
||
// Get all pieces with details including leaf counts | ||
console.log('\n--- Getting Pieces with Details ---') | ||
try { | ||
const context = await synapse.storage.createContext({ | ||
dataSetId: dataSetInfo.dataSetId, | ||
providerId: dataSetInfo.providerId, | ||
}) | ||
|
||
const piecesWithDetails = await context.getDataSetPiecesWithDetails() | ||
console.log(`✅ Retrieved ${piecesWithDetails.length} pieces with details:`) | ||
|
||
piecesWithDetails.forEach((piece, index) => { | ||
console.log(`\n Piece ${index + 1}:`) | ||
console.log(` ID: ${piece.pieceId}`) | ||
console.log(` CID: ${piece.pieceCid}`) | ||
console.log(` Leaf Count: ${piece.leafCount}`) | ||
console.log(` Raw Size: ${piece.rawSize} bytes (${(piece.rawSize / 1024).toFixed(2)} KB)`) | ||
console.log(` Sub-piece CID: ${piece.subPieceCid}`) | ||
console.log(` Sub-piece Offset: ${piece.subPieceOffset}`) | ||
}) | ||
|
||
// Calculate totals | ||
const totalLeafCount = piecesWithDetails.reduce((sum, piece) => sum + piece.leafCount, 0) | ||
const totalRawSize = piecesWithDetails.reduce((sum, piece) => sum + piece.rawSize, 0) | ||
|
||
console.log(`\n📈 Data Set Summary:`) | ||
console.log(` Total Pieces: ${piecesWithDetails.length}`) | ||
console.log(` Total Leaf Count: ${totalLeafCount}`) | ||
console.log(` Total Raw Size: ${totalRawSize} bytes (${(totalRawSize / 1024).toFixed(2)} KB)`) | ||
console.log(` Average Piece Size: ${(totalRawSize / piecesWithDetails.length).toFixed(2)} bytes`) | ||
} catch (error) { | ||
console.error('❌ Error getting pieces with details:', error.message) | ||
} | ||
} catch (error) { | ||
console.error('❌ Error:', error.message) | ||
console.error('Stack trace:', error.stack) | ||
} | ||
} | ||
|
||
main().catch(console.error) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.