Skip to content

Commit 506dc4c

Browse files
NoahMaizelsNoahMaizels
committed
pinning and tracking pages
1 parent 32c8032 commit 506dc4c

File tree

4 files changed

+284
-278
lines changed

4 files changed

+284
-278
lines changed

‎docs/documentation/pinning.md‎

Lines changed: 134 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,141 @@ slug: /pinning
55
sidebar_label: Pinning
66
---
77

8-
## 🚧 Under Construction 🚧
9-
:::caution 🚧 This page is under construction
8+
Pinning allows you to guarantee that your uploaded content will always be available by storing it **locally on your own Bee node**. However, pinning **guarantee availability across the Swarm network**. Therefore you must use pinning along with the **stewardship utilities** included in `bee-js` to monitor the availability of your pinned content and reupload it if needed.
109

11-
This section is still being worked on. Check back soon for updates!
10+
In this section, you'll learn how to:
1211

13-
:::
12+
- Pin content
13+
- Check whether pinned content is still retrievable from the network
14+
- Reupload missing content
15+
- View all currently pinned references
16+
- Remove pins that are no longer required
17+
18+
19+
20+
## Pinning and Unpinning Content
21+
22+
To pin a reference (so it remains stored on your node):
23+
24+
```js
25+
await bee.pin(reference)
26+
console.log('Reference pinned locally.')
27+
```
28+
29+
To stop tracking and remove it from local pin storage:
30+
31+
```js
32+
await bee.unpin(reference)
33+
console.log('Reference unpinned and no longer tracked.')
34+
```
35+
36+
37+
## Checking if a Reference is Retrievable
38+
39+
Use `isReferenceRetrievable(reference)` to verify if the content for a given Swarm reference is currently accessible on the network:
40+
41+
```js
42+
const isAvailable = await bee.isReferenceRetrievable(reference)
43+
44+
if (isAvailable) {
45+
console.log('Data is retrievable from the network.')
46+
} else {
47+
console.log('Data is missing from the network.')
48+
}
49+
```
50+
51+
## Reuploading Pinned Data
52+
53+
If content is missing but was previously pinned, you can reupload it using `reuploadPinnedData(postageBatchId, reference)`:
54+
55+
```js
56+
await bee.reuploadPinnedData(postageBatchId, reference)
57+
console.log('Data has been reuploaded to the network.')
58+
```
59+
60+
## Listing All Pinned References
61+
62+
You can get all currently pinned references with:
63+
64+
```js
65+
const pins = await bee.getAllPins()
66+
console.log('Pinned references:', pins.map(ref => ref.toHex()))
67+
```
68+
69+
To check if a specific reference is pinned:
70+
71+
```js
72+
const pinStatus = await bee.getPin(reference)
73+
console.log('Pin info:', pinStatus)
74+
```
75+
76+
## Deleting a Tag After Use
77+
78+
Once you’re done tracking an upload, you can delete the tag to keep your node clean:
79+
80+
```js
81+
await bee.deleteTag(tag.uid)
82+
console.log("Deleted tag:", tag.uid)
83+
```
84+
85+
86+
## Example Script
87+
88+
The following script automates the process of checking all locally pinned references, verifying their retrievability from the network, and reuploading any that are missing. This ensures that your pinned data remains available even if it has dropped out of the Swarm network.
89+
90+
```js
91+
import { Bee } from "@ethersphere/bee-js"
92+
93+
const bee = new Bee('http://localhost:1633')
94+
const postageBatchId = "129903062bedc4eca6fc1c232ed385e93dda72f711caa1ead6018334dd801cee"
95+
96+
async function reuploadMissingPins() {
97+
try {
98+
// Get all currently pinned references
99+
const pinnedRefs = await bee.getAllPins()
100+
101+
if (!pinnedRefs.length) {
102+
console.log("No pinned references found.")
103+
return
104+
}
105+
106+
console.log(`Found ${pinnedRefs.length} pinned references.`)
107+
108+
let repaired = 0
109+
110+
// Loop through all references and check retrievability
111+
for (const ref of pinnedRefs) {
112+
const reference = ref.toHex()
113+
const isAvailable = await bee.isReferenceRetrievable(reference)
114+
115+
if (isAvailable) {
116+
console.log(`✅ ${reference} is retrievable.`)
117+
} else {
118+
console.log(`⚠️ ${reference} is missing — reuploading...`)
119+
await bee.reuploadPinnedData(postageBatchId, reference)
120+
console.log(`🔁 Reuploaded ${reference}`)
121+
repaired++
122+
}
123+
}
124+
125+
console.log(`\nDone. ${repaired} reference(s) were reuploaded.`)
126+
} catch (error) {
127+
console.error("Error:", error.message)
128+
}
129+
}
130+
131+
reuploadMissingPins()
132+
```
133+
134+
Example terminal output:
135+
136+
```bash
137+
Found 2 pinned references.
138+
⚠️ 1880ff0bbd23997dfa46921ba2ab0098824d967fe60c6ca1ae2e8fd722f4db78 is missing — reuploading...
139+
🔁 Reuploaded 1880ff0bbd23997dfa46921ba2ab0098824d967fe60c6ca1ae2e8fd722f4db78
140+
✅ fd79d5e0ebd8407e422f53ce1d7c4c41ebf403be55143900f8d1490560294780 is retrievable.
141+
142+
Done. 1 reference(s) were reuploaded.
143+
```
14144

15145

16-
* Show an example of pinning a reference
17-
* Show an example of listing references
18-
* Show an example of re-uploading a reference

‎docs/documentation/tracking-uploads.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
---
2+
title: Tracking Uploads
3+
id: tracking-uploads
4+
slug: /tracking-uploads
5+
sidebar_label: Tracking Uploads
6+
---
7+
8+
You can track the progress of your uploads using "tags". Each tag tracks how many chunks were **split**, **stored**, **seen**, and **synced** by the network. By creating a tag before uploading and passing it to the upload function, you make the upload *trackable, allowing you to confirm whether your uploaded data has been fully synced.
9+
10+
## How It Works
11+
12+
### 1. Create a Tag
13+
14+
Before uploading, create a new tag using `bee.createTag()`. This returns a unique tag UID that will be used to monitor the upload.
15+
16+
```js
17+
const tag = await bee.createTag()
18+
console.log("Created new tag with UID:", tag.uid)
19+
```
20+
21+
Alternatively, you can use an existing tag from `bee.getAllTags()` (useful for testing or reuse):
22+
23+
```js
24+
const allTags = await bee.getAllTags()
25+
if (allTags.length > 0) {
26+
tag = allTags[0]
27+
console.log("Using existing tag with UID:", tag.uid)
28+
}
29+
```
30+
31+
### 2. Upload a File with the Tag
32+
33+
To enable tracking, pass the tag UID into the upload options under the `tag` key:
34+
35+
```js
36+
const result = await bee.uploadFile(postageBatchId, fileData, 'nodes.json', {
37+
tag: tag.uid,
38+
contentType: 'application/json'
39+
})
40+
```
41+
42+
This links the upload to your tag so you can monitor its progress.
43+
44+
### 3. Track Tag Progress
45+
46+
Use `bee.retrieveTag(tagUid)` to check how many chunks have been split and how many are synced. Poll repeatedly until `synced === split` to know when the upload has fully propagated.
47+
48+
```js
49+
const tag = await bee.retrieveTag(tagUid)
50+
console.log(` - Total split: ${tag.split}`)
51+
console.log(` - Synced: ${tag.synced}`)
52+
```
53+
54+
## Example Script
55+
56+
```js
57+
import { Bee } from "@ethersphere/bee-js"
58+
import fs from "fs/promises"
59+
60+
const bee = new Bee('http://localhost:1633')
61+
const postageBatchId = "129903062bedc4eca6fc1c232ed385e93dda72f711caa1ead6018334dd801cee"
62+
63+
async function waitForTagSync(tagUid, interval = 800) {
64+
while (true) {
65+
const tag = await bee.retrieveTag(tagUid)
66+
67+
console.log(`Progress (Tag ${tagUid}):`)
68+
console.log(` - Total split: ${tag.split}`)
69+
console.log(` - Stored: ${tag.stored}`)
70+
console.log(` - Seen: ${tag.seen}`)
71+
console.log(` - Synced: ${tag.synced}`)
72+
73+
if (tag.split > 0 && tag.synced >= tag.split) {
74+
console.log("Upload fully synced!")
75+
break
76+
}
77+
78+
await new Promise(resolve => setTimeout(resolve, interval))
79+
}
80+
}
81+
82+
async function uploadNodesJsonWithTag() {
83+
try {
84+
const fileData = await fs.readFile('./nodes.json')
85+
86+
const tag = await bee.createTag()
87+
console.log("Created new tag with UID:", tag.uid)
88+
89+
const result = await bee.uploadFile(postageBatchId, fileData, 'nodes.json', {
90+
tag: tag.uid,
91+
contentType: 'application/json'
92+
})
93+
94+
console.log("Uploaded reference:", result.reference.toHex())
95+
96+
await waitForTagSync(tag.uid)
97+
} catch (error) {
98+
console.error("Error uploading nodes.json:", error.message)
99+
}
100+
}
101+
102+
uploadNodesJsonWithTag()
103+
```
104+
105+
Example terminal output:
106+
107+
```bash
108+
Created new tag with UID: 85
109+
Uploaded reference: 78e5247e97b1a3362b6c3f054924dce734e0ffd7df0cb5ed9b636cb6a4a14d93
110+
Progress (Tag 85):
111+
- Total split: 1078
112+
- Stored: 0
113+
- Seen: 0
114+
- Synced: 546
115+
Progress (Tag 85):
116+
- Total split: 1078
117+
- Stored: 0
118+
- Seen: 0
119+
- Synced: 1078
120+
Upload fully synced!
121+
```
122+
123+
## Deleting Tags
124+
125+
You can delete tags you no longer need using their uid:
126+
127+
```js
128+
await bee.deleteTag(tag.uid)
129+
console.log("Deleted tag:", tag.uid)
130+
```
131+
132+
## References
133+
134+
- [Bee docs – Syncing / Tags](https://docs.ethswarm.org/docs/develop/access-the-swarm/syncing)
135+
- [Bee API Reference – `/tags`](https://docs.ethswarm.org/api/#tag/Tag)

0 commit comments

Comments
 (0)