Skip to content

Commit 1f69375

Browse files
committed
edits + added summary in retreive.md
1 parent 5fb2e64 commit 1f69375

File tree

3 files changed

+113
-69
lines changed

3 files changed

+113
-69
lines changed

docs/concepts/lifecycle.md

Lines changed: 30 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,41 @@
11
---
22
title: The lifecycle of data in IPFS
3-
description:
3+
description: Learn about the lifecycle of data in IPFS.
44
---
55

66
# The lifecycle of data in IPFS
77

8-
The lifecycle of data in IPFS can be summarised as follows:
8+
The lifecycle of data in IPFS is summarized below.
99

10-
1. **Content-addressable representation**: The file is transformed into a content-addressable representation with a CID. The basic idea is that this representation makes files and directories **content-addressable** via CIDs by chunking files into smaller blocks, calculating their hashes, and constructing a [Merkle DAG](./merkle-dag.md).
11-
2. **Pinning:** refers to the stage where the blocks of the CID are saved on an IPFS node (or pinning service) and made retrievable to the network. Since saving is not enough for the CID to be retrievable, pinning typically refers to two things:
12-
- **Advertising:** Making each CID discoverable to the IPFS network by advertising a record linking between the CID and the server's IP address to the [DHT](./dht.md). This advertising stage is a continuous process that repeats typically every 12 hours. The term **publishing** is also commonly used to refer to this step.
13-
- **Providing:** The content-addressable representation of the CID is persisted on one of web3.storage's IPFS nodes (servers running an IPFS node) and made publicly available to the IPFS network.
14-
3. **Retrieval:** refers to when an IPFS node fetches the blocks of the CID and constructs the Merkle DAG. This usually involves several steps:
15-
- **Block fetching:** An IPFS node fetches the blocks of the Merkle DAG (of the file or folder) from providers.
16-
- **Verifying** the IPFS node verifies the blocks fetched by hashing them and ensuring the resulting hash is correct. Note that blocks can come from any node in the network and this type of retrieval is _trustless_.
17-
- **Local access:** once all the blocks are present, the Merkle DAG can be constructed making the file or directory underlying the CID successfully replicated and accessible.
18-
4. **Deleting:** refers to the deletion of a CID from a node. Deletion is always a local operation. If a CID has been replicated by other nodes, it will continue to be available on the IPFS network.
10+
## 1. Content-addressable representation
1911

20-
> **Note:** once the CID is replicated by another node, it is typically advertised by default, even if it isn't explicitly pinned.
12+
The file is transformed into a content-addressable representation using a CID. The basic idea is that this representation makes files and directories **content-addressable** via CIDs by chunking files into smaller blocks, calculating their hashes, and constructing a [Merkle DAG](./merkle-dag.md).
13+
14+
## 2. Pinning
15+
16+
In this stage, the blocks of the CID are saved on an IPFS node (or pinning service) and made retrievable to the network. Simply saving the CID on the node does not mean the CID is retrievable, so pinning must be used. Pinning allows the node to advertise that it has the CID, and provide it to the network.
17+
18+
- **Advertising:** In this step, a CID is made discoverable to the IPFS network by advertising a record linking the CID and the server's IP address to the [DHT](./dht.md). Advertising is a continuous process that repeats typically every 12 hours. The term **publishing** is also commonly used to refer to this step.
19+
20+
- **Providing:** The content-addressable representation of the CID is persisted on one of web3.storage's IPFS nodes (servers running an IPFS node) and made publicly available to the IPFS network.
21+
22+
## 3. Retrieval
23+
24+
In this stage, an IPFS node fetches the blocks of the CID and constructs the Merkle DAG. This usually involves several steps:
25+
26+
- **Block fetching:** An IPFS node fetches the blocks of the Merkle DAG (of the file or folder) from providers.
27+
28+
- **Verifying:** The IPFS node verifies the blocks fetched by hashing them and ensuring that the resulting hash is correct. Note that this type of retrieval is _trustless_; that is, blocks can come from any node in the network.
29+
30+
- **Local access:** Once all blocks are present, the Merkle DAG can be constructed, making the file or directory underlying the CID successfully replicated and accessible.
31+
32+
## 4. Deleting
33+
34+
At this point, the CID is deleted from a node. Deletion is always a local operation. If a CID has been replicated to other nodes, it will continue to be available on the IPFS network.
35+
36+
:::callout
37+
Once the CID is replicated by another node, it is typically advertised by default, even if it isn't explicitly pinned.
38+
:::
2139

2240
Learn more about the lifecycle of data in IPFS in the following video:
2341

docs/quickstart/publish.md

Lines changed: 32 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,37 @@
11
---
22
title: 'Publishing a file with IPFS using a pinning service'
3-
description: 'learn how to publish content with IPFS by pinning a file to a pinning service'
3+
description: 'Learn how to publish content with IPFS by pinning a file to a pinning service.'
4+
sidebarDepth: 0
45
---
56

67
# Publish a file with IPFS
78

8-
In this quickstart guide, you will learn about [pinning services](../concepts/persistence.md#pinning-in-context) and how to use them to publish content-addressed data with IPFS. To do so, you will upload the file to a pinning service called [web3.storage](https://web3.storage/). By the end of this guide, you should have a better understanding of how content addressing and CIDs work from a high level.
9+
In this quickstart guide, you will learn about [pinning services](../concepts/persistence.md#pinning-in-context) and how to use them to publish content-addressed data with IPFS. To learn the process, you will upload the file to a pinning service called [web3.storage](https://web3.storage/). By the end of this guide, you should have a better understanding of how content addressing and CIDs work from a high level.
910

10-
> **Note:** The web3.storage pinning service was chosen purely for demonstration purposes, and is one of many [pinning services](../concepts/persistence.md#pinning-in-context) you can choose from. While each pinning services has different SDKs and APIs, their fundamental role is the same - to store files and make them available to the IPFS network. In fact, one of the main benefits of IPFS is that files can be pinned to multiple pinning services, thereby reducing vendor lock-in.
11+
:::callout
12+
The web3.storage pinning service was chosen purely for demonstration purposes, and is one of many [pinning services](../concepts/persistence.md#pinning-in-context) you can choose from. While each pinning services has different SDKs and APIs, their fundamental role is the same - to store files and make them available to the IPFS network. In fact, one of the main benefits of IPFS is that files can be pinned to multiple pinning services, thereby reducing vendor lock-in.
13+
:::
1114

1215
## Contents <!-- omit from toc -->
1316

14-
- [What is pinning?](#what-is-pinning)
17+
- [Pinning overview](#overview)
1518
- [Prerequisites](#prerequisites)
1619
- [Uploading and pinning a file](#uploading-and-pinning-a-file)
17-
- [What's a CID?](#whats-a-cid)
20+
- [CIDs explained](#cids-explained)
1821
- [Summary and next steps](#summary-and-next-steps)
1922

20-
## What is pinning?
23+
## Overview
2124

22-
Pinning refers to the process of ensuring that a particular piece of content is retrievable with IPFS. In other words, pinning is equivalent to storing a file on a computer or server that is connected to the internet, thereby making it available to the rest of the IPFS network.
25+
_Pinning_ refers to the process of ensuring that a particular piece of content is retrievable with IPFS. In other words, pinning is equivalent to storing a file on a computer or server that is connected to the internet, thereby making it available to the rest of the IPFS network.
2326

2427
Pinning can be done at various levels, from individual files to entire directories that are addressed by a CID. You can also pin CIDs to multiple IPFS nodes to increase the redundancy and resilience of the file on the network.
2528

2629
### Pinning services
2730

28-
[Pinning services](../concepts/persistence/#pinning-services) are like hosting services that run an IPFS node for you and ensure that your files are available to the IPFS network.
31+
[Pinning services](../concepts/persistence/#pinning-services) are similar to hosting services, in that they run an IPFS node for you and ensure that your files are available to the IPFS network.
2932

3033
:::callout
31-
Data pinned to the IPFS network is public by default and retrievable by anyone. Make sure to avoid publishing private data or take the necessary measures to adequately encrypt it before publishing.
34+
Data pinned to the IPFS network is public by default and retrievable by anyone. Avoid publishing private data or adequately encrypt it before publishing.
3235
:::
3336

3437
## Prerequisites
@@ -40,39 +43,37 @@ Data pinned to the IPFS network is public by default and retrievable by anyone.
4043

4144
## Uploading and pinning a file
4245

43-
1. Sign into your account on web3.storage:
46+
1. Sign into your account on web3.storage.
4447

4548
![web3.storage login window](./images/login-web3-storage.png)
4649

47-
1. After logging in, click on **Upload files** to open the upload tab:
50+
1. After logging in, click on **Upload files** to open the upload tab.
4851

4952
![web3.storage upload button](./images/web3-upload-button.png)
5053

51-
The Upload a file tab:
54+
1. Upload and pin the file by doing either dragging the [image file](/images/welcome-to-IPFS.jpg) to the dashed rectangle labeled **Drag and drop your files here**, or clicking on the dashed rectangle to select the [image](/images/welcome-to-IPFS.jpg) file.
5255

5356
![web3.storage upload tab](./images/web3-upload-tab.png)
5457

55-
1. Upload and pin the file by either dragging the [image file](/images/welcome-to-IPFS.jpg) to the dashed rectangle, or clicking on the dashed rectangle to select the [image](/images/welcome-to-IPFS.jpg) file.
56-
5758
Once the file has been successfully uploaded, the following displays:
5859

5960
![web3.storage file uploaded](./images/web3-file-uploaded.png)
6061

61-
If you close the upload tab, you should also be able to see a shortened **Content Identifier (CID)** of the uploaded image, **`bafyb...d32wm3q4`**:
62+
If you close the upload tab, you should also be able to see a shortened **Content Identifier (CID)** of the uploaded image, **`bafyb...d32wm3q4`**, similar to the image below:
6263

6364
![file after upload](./images/web3-file-after-upload.png)
6465

6566
Congratulations, you have successfully **pinned** a file to IPFS! 🎉
6667

6768
Let's unpack what just happened, by looking at CIDs.
6869

69-
## What's a CID?
70+
## CIDs explained
7071

7172
In IPFS, every file and directory is identified with a Content Identifier ([CID](../concepts/content-addressing.md)). The CID serves as the **permanent address** of the file and can be used by anyone to find it on the IPFS network.
7273

73-
When a file is first added to an IPFS node (like the image used in this guide), it's first transformed into a content-addressable representation by which the file is split into smaller chunks (if above ~1MB) which are linked and hashed to produce the CID.
74+
When a file is first added to an IPFS node (like the image used in this guide), it's first transformed into a content-addressable representation in which the file is split into smaller chunks (if above ~1MB) which are linked together and hashed to produce the CID.
7475

75-
The CID for the uploaded image is:
76+
In this guide, the CID for the uploaded image is:
7677

7778
```plaintext
7879
bafybeicn7i3soqdgr7dwnrwytgq4zxy7a5jpkizrvhm5mv6bgjd32wm3q4
@@ -82,29 +83,32 @@ You can now share the CID with anyone and they can fetch the file using IPFS.
8283

8384
To dive deeper into the anatomy of the CID, check out the [CID inspector](https://cid.ipfs.tech/#bafybeicn7i3soqdgr7dwnrwytgq4zxy7a5jpkizrvhm5mv6bgjd32wm3q4)
8485

85-
> **Note:** the transformation into a content-addressable representation is a local operation that doesn't require any network connectivity. With web3.storage, this transformation happens on the client-side (in the browser.)
86+
:::callout
87+
The transformation into a content-addressable representation is a local operation that doesn't require any network connectivity. With web3.storage, this transformation happens client-side (in the browser).
88+
:::
8689

8790
## Retrieving with a gateway
8891

89-
Now that your file is pinned to a pinning service, you will fetch it using an IPFS gateway. An [**IPFS Gateway**](/concepts/ipfs-gateway/) is an HTTP interface that serves as a bridge to the IPFS network. In other words, it allows fetching CIDs from IPFS with HTTP.
92+
Now that your file is pinned to a pinning service, you will fetch it using an IPFS gateway. An [**IPFS Gateway**](/concepts/ipfs-gateway/) is an HTTP interface that serves as a bridge to the IPFS network. In other words, it allows you to fetch CIDs from IPFS using HTTP.
9093

91-
Pinning services typically offer an IPFS gateway as a way to easily retrieve your CIDs, for example, Web3.storage operates the [w3s.link](https://w3s.link) gateway, with which you can retrieve the uploaded CID.
94+
Pinning services typically offer an IPFS gateway as a way to easily retrieve your CIDs. For example, Web3.storage operates the [w3s.link](https://w3s.link) gateway, from which you can retrieve the uploaded CID.
9295

9396
To retrieve the CID, open the following URL:
9497
[bafybeicn7i3soqdgr7dwnrwytgq4zxy7a5jpkizrvhm5mv6bgjd32wm3q4.ipfs.w3s.link](https://bafybeicn7i3soqdgr7dwnrwytgq4zxy7a5jpkizrvhm5mv6bgjd32wm3q4.ipfs.w3s.link/)
9598

96-
> **Note:** When pinning a file to IPFS, the filename is not stored by default. To ensure the filename is retained, it's common to wrap the file in a directory. In such instances, both the file and the directory will have unique CIDs. Web3.storage wraps files in a directory by default, which is why you see a directory listing with the file `welcome-to-IPFS.jpg` followed by a shorthand CID: `bafk…beom` of the file.
99+
:::callout
100+
When pinning a file to IPFS, the filename is not stored by default. To ensure the filename is retained, it's common to wrap the file in a directory. In such instances, both the file and the directory will have unique CIDs. Web3.storage wraps files in a directory by default, which is why you see a directory listing with the file `welcome-to-IPFS.jpg` followed by a shorthand CID: `bafk…beom` of the file.
101+
:::
97102

98103
## Summary and next steps
99104

100105
In this quickstart guide, you learned about [pinning services](../concepts/persistence.md#pinning-in-context), and how to use them to publish content-addressed data with IPFS. You also learned how CIDs address files and directories in IPFS by uploading a file to a pinning service called [web3.storage](https://web3.storage/).
101106

102107
Pinning services provide a convenient alternative to running IPFS nodes and infrastructure. However, the two are not mutually exclusive; you can combine a pinning service with an IPFS node on your computer to increase the resilience of your CIDs.
103108

104-
Check out [the lifecycle of data in IPFS](../concepts/lifecycle.md) to learn more about how publishing by pinning fits into the full lifecycle of data in IPFS.
105-
106-
As a next step, try fetching the pinned file, by following the [retrieval quickstart](./retrieve.md).
107-
108-
If you prefer programmatically uploading with JavaScript or Go, check out the [Web3.storage docs](https://web3.storage/docs/how-tos/store/).
109+
Possible next steps include:
109110

110-
Alternatively, try out [Filebase](https://filebase.com/), a pinning service that offers an [AWS S3-compatible API for pinning](https://docs.filebase.com/getting-started/s3-api-getting-started-guide), so you can use any S3-compatible SDK, e.g. [aws-sdk](https://www.npmjs.com/package/aws-sdk), and [many](https://github.com/s3tools/s3cmd) more.
111+
- Check out [the lifecycle of data in IPFS](../concepts/lifecycle.md) to learn more about how publishing by pinning fits into the full lifecycle of data in IPFS.
112+
- Try fetching the pinned file by following the [retrieval quickstart](./retrieve.md).
113+
- If you prefer to upload a file programmatically with JavaScript or Go, check out the [Web3.storage docs](https://web3.storage/docs/how-tos/store/).
114+
- Alternatively, try out [Filebase](https://filebase.com/), a pinning service that offers an [AWS S3-compatible API for pinning](https://docs.filebase.com/getting-started/s3-api-getting-started-guide), so you can use any S3-compatible SDK, e.g. [aws-sdk](https://www.npmjs.com/package/aws-sdk), and [many](https://github.com/s3tools/s3cmd) more.

0 commit comments

Comments
 (0)