You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/concepts/lifecycle.md
+30-12Lines changed: 30 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,23 +1,41 @@
1
1
---
2
2
title: The lifecycle of data in IPFS
3
-
description:
3
+
description: Learn about the lifecycle of data in IPFS.
4
4
---
5
5
6
6
# The lifecycle of data in IPFS
7
7
8
-
The lifecycle of data in IPFS can be summarised as follows:
8
+
The lifecycle of data in IPFS is summarized below.
9
9
10
-
1.**Content-addressable representation**: The file is transformed into a content-addressable representation with a CID. The basic idea is that this representation makes files and directories **content-addressable** via CIDs by chunking files into smaller blocks, calculating their hashes, and constructing a [Merkle DAG](./merkle-dag.md).
11
-
2.**Pinning:** refers to the stage where the blocks of the CID are saved on an IPFS node (or pinning service) and made retrievable to the network. Since saving is not enough for the CID to be retrievable, pinning typically refers to two things:
12
-
-**Advertising:** Making each CID discoverable to the IPFS network by advertising a record linking between the CID and the server's IP address to the [DHT](./dht.md). This advertising stage is a continuous process that repeats typically every 12 hours. The term **publishing** is also commonly used to refer to this step.
13
-
-**Providing:** The content-addressable representation of the CID is persisted on one of web3.storage's IPFS nodes (servers running an IPFS node) and made publicly available to the IPFS network.
14
-
3.**Retrieval:** refers to when an IPFS node fetches the blocks of the CID and constructs the Merkle DAG. This usually involves several steps:
15
-
-**Block fetching:** An IPFS node fetches the blocks of the Merkle DAG (of the file or folder) from providers.
16
-
-**Verifying** the IPFS node verifies the blocks fetched by hashing them and ensuring the resulting hash is correct. Note that blocks can come from any node in the network and this type of retrieval is _trustless_.
17
-
-**Local access:** once all the blocks are present, the Merkle DAG can be constructed making the file or directory underlying the CID successfully replicated and accessible.
18
-
4.**Deleting:** refers to the deletion of a CID from a node. Deletion is always a local operation. If a CID has been replicated by other nodes, it will continue to be available on the IPFS network.
10
+
## 1. Content-addressable representation
19
11
20
-
> **Note:** once the CID is replicated by another node, it is typically advertised by default, even if it isn't explicitly pinned.
12
+
The file is transformed into a content-addressable representation using a CID. The basic idea is that this representation makes files and directories **content-addressable** via CIDs by chunking files into smaller blocks, calculating their hashes, and constructing a [Merkle DAG](./merkle-dag.md).
13
+
14
+
## 2. Pinning
15
+
16
+
In this stage, the blocks of the CID are saved on an IPFS node (or pinning service) and made retrievable to the network. Simply saving the CID on the node does not mean the CID is retrievable, so pinning must be used. Pinning allows the node to advertise that it has the CID, and provide it to the network.
17
+
18
+
-**Advertising:** In this step, a CID is made discoverable to the IPFS network by advertising a record linking the CID and the server's IP address to the [DHT](./dht.md). Advertising is a continuous process that repeats typically every 12 hours. The term **publishing** is also commonly used to refer to this step.
19
+
20
+
-**Providing:** The content-addressable representation of the CID is persisted on one of web3.storage's IPFS nodes (servers running an IPFS node) and made publicly available to the IPFS network.
21
+
22
+
## 3. Retrieval
23
+
24
+
In this stage, an IPFS node fetches the blocks of the CID and constructs the Merkle DAG. This usually involves several steps:
25
+
26
+
-**Block fetching:** An IPFS node fetches the blocks of the Merkle DAG (of the file or folder) from providers.
27
+
28
+
-**Verifying:** The IPFS node verifies the blocks fetched by hashing them and ensuring that the resulting hash is correct. Note that this type of retrieval is _trustless_; that is, blocks can come from any node in the network.
29
+
30
+
-**Local access:** Once all blocks are present, the Merkle DAG can be constructed, making the file or directory underlying the CID successfully replicated and accessible.
31
+
32
+
## 4. Deleting
33
+
34
+
At this point, the CID is deleted from a node. Deletion is always a local operation. If a CID has been replicated to other nodes, it will continue to be available on the IPFS network.
35
+
36
+
:::callout
37
+
Once the CID is replicated by another node, it is typically advertised by default, even if it isn't explicitly pinned.
38
+
:::
21
39
22
40
Learn more about the lifecycle of data in IPFS in the following video:
title: 'Publishing a file with IPFS using a pinning service'
3
-
description: 'learn how to publish content with IPFS by pinning a file to a pinning service'
3
+
description: 'Learn how to publish content with IPFS by pinning a file to a pinning service.'
4
+
sidebarDepth: 0
4
5
---
5
6
6
7
# Publish a file with IPFS
7
8
8
-
In this quickstart guide, you will learn about [pinning services](../concepts/persistence.md#pinning-in-context) and how to use them to publish content-addressed data with IPFS. To do so, you will upload the file to a pinning service called [web3.storage](https://web3.storage/). By the end of this guide, you should have a better understanding of how content addressing and CIDs work from a high level.
9
+
In this quickstart guide, you will learn about [pinning services](../concepts/persistence.md#pinning-in-context) and how to use them to publish content-addressed data with IPFS. To learn the process, you will upload the file to a pinning service called [web3.storage](https://web3.storage/). By the end of this guide, you should have a better understanding of how content addressing and CIDs work from a high level.
9
10
10
-
> **Note:** The web3.storage pinning service was chosen purely for demonstration purposes, and is one of many [pinning services](../concepts/persistence.md#pinning-in-context) you can choose from. While each pinning services has different SDKs and APIs, their fundamental role is the same - to store files and make them available to the IPFS network. In fact, one of the main benefits of IPFS is that files can be pinned to multiple pinning services, thereby reducing vendor lock-in.
11
+
:::callout
12
+
The web3.storage pinning service was chosen purely for demonstration purposes, and is one of many [pinning services](../concepts/persistence.md#pinning-in-context) you can choose from. While each pinning services has different SDKs and APIs, their fundamental role is the same - to store files and make them available to the IPFS network. In fact, one of the main benefits of IPFS is that files can be pinned to multiple pinning services, thereby reducing vendor lock-in.
13
+
:::
11
14
12
15
## Contents <!-- omit from toc -->
13
16
14
-
-[What is pinning?](#what-is-pinning)
17
+
-[Pinning overview](#overview)
15
18
-[Prerequisites](#prerequisites)
16
19
-[Uploading and pinning a file](#uploading-and-pinning-a-file)
17
-
-[What's a CID?](#whats-a-cid)
20
+
-[CIDs explained](#cids-explained)
18
21
-[Summary and next steps](#summary-and-next-steps)
19
22
20
-
## What is pinning?
23
+
## Overview
21
24
22
-
Pinning refers to the process of ensuring that a particular piece of content is retrievable with IPFS. In other words, pinning is equivalent to storing a file on a computer or server that is connected to the internet, thereby making it available to the rest of the IPFS network.
25
+
_Pinning_ refers to the process of ensuring that a particular piece of content is retrievable with IPFS. In other words, pinning is equivalent to storing a file on a computer or server that is connected to the internet, thereby making it available to the rest of the IPFS network.
23
26
24
27
Pinning can be done at various levels, from individual files to entire directories that are addressed by a CID. You can also pin CIDs to multiple IPFS nodes to increase the redundancy and resilience of the file on the network.
25
28
26
29
### Pinning services
27
30
28
-
[Pinning services](../concepts/persistence/#pinning-services) are like hosting servicesthat run an IPFS node for you and ensure that your files are available to the IPFS network.
31
+
[Pinning services](../concepts/persistence/#pinning-services) are similar to hosting services, in that they run an IPFS node for you and ensure that your files are available to the IPFS network.
29
32
30
33
:::callout
31
-
Data pinned to the IPFS network is public by default and retrievable by anyone. Make sure to avoid publishing private data or take the necessary measures to adequately encrypt it before publishing.
34
+
Data pinned to the IPFS network is public by default and retrievable by anyone. Avoid publishing private data or adequately encrypt it before publishing.
32
35
:::
33
36
34
37
## Prerequisites
@@ -40,39 +43,37 @@ Data pinned to the IPFS network is public by default and retrievable by anyone.
1. Upload and pin the file by doing either dragging the [image file](/images/welcome-to-IPFS.jpg) to the dashed rectangle labeled **Drag and drop your files here**, or clicking on the dashed rectangle to select the [image](/images/welcome-to-IPFS.jpg) file.
1. Upload and pin the file by either dragging the [image file](/images/welcome-to-IPFS.jpg) to the dashed rectangle, or clicking on the dashed rectangle to select the [image](/images/welcome-to-IPFS.jpg) file.
56
-
57
58
Once the file has been successfully uploaded, the following displays:
If you close the upload tab, you should also be able to see a shortened **Content Identifier (CID)** of the uploaded image, **`bafyb...d32wm3q4`**:
62
+
If you close the upload tab, you should also be able to see a shortened **Content Identifier (CID)** of the uploaded image, **`bafyb...d32wm3q4`**, similar to the image below:
62
63
63
64

64
65
65
66
Congratulations, you have successfully **pinned** a file to IPFS! 🎉
66
67
67
68
Let's unpack what just happened, by looking at CIDs.
68
69
69
-
## What's a CID?
70
+
## CIDs explained
70
71
71
72
In IPFS, every file and directory is identified with a Content Identifier ([CID](../concepts/content-addressing.md)). The CID serves as the **permanent address** of the file and can be used by anyone to find it on the IPFS network.
72
73
73
-
When a file is first added to an IPFS node (like the image used in this guide), it's first transformed into a content-addressable representation by which the file is split into smaller chunks (if above ~1MB) which are linked and hashed to produce the CID.
74
+
When a file is first added to an IPFS node (like the image used in this guide), it's first transformed into a content-addressable representation in which the file is split into smaller chunks (if above ~1MB) which are linked together and hashed to produce the CID.
@@ -82,29 +83,32 @@ You can now share the CID with anyone and they can fetch the file using IPFS.
82
83
83
84
To dive deeper into the anatomy of the CID, check out the [CID inspector](https://cid.ipfs.tech/#bafybeicn7i3soqdgr7dwnrwytgq4zxy7a5jpkizrvhm5mv6bgjd32wm3q4)
84
85
85
-
> **Note:** the transformation into a content-addressable representation is a local operation that doesn't require any network connectivity. With web3.storage, this transformation happens on the client-side (in the browser.)
86
+
:::callout
87
+
The transformation into a content-addressable representation is a local operation that doesn't require any network connectivity. With web3.storage, this transformation happens client-side (in the browser).
88
+
:::
86
89
87
90
## Retrieving with a gateway
88
91
89
-
Now that your file is pinned to a pinning service, you will fetch it using an IPFS gateway. An [**IPFS Gateway**](/concepts/ipfs-gateway/) is an HTTP interface that serves as a bridge to the IPFS network. In other words, it allows fetching CIDs from IPFS with HTTP.
92
+
Now that your file is pinned to a pinning service, you will fetch it using an IPFS gateway. An [**IPFS Gateway**](/concepts/ipfs-gateway/) is an HTTP interface that serves as a bridge to the IPFS network. In other words, it allows you to fetch CIDs from IPFS using HTTP.
90
93
91
-
Pinning services typically offer an IPFS gateway as a way to easily retrieve your CIDs, for example, Web3.storage operates the [w3s.link](https://w3s.link) gateway, with which you can retrieve the uploaded CID.
94
+
Pinning services typically offer an IPFS gateway as a way to easily retrieve your CIDs. For example, Web3.storage operates the [w3s.link](https://w3s.link) gateway, from which you can retrieve the uploaded CID.
> **Note:** When pinning a file to IPFS, the filename is not stored by default. To ensure the filename is retained, it's common to wrap the file in a directory. In such instances, both the file and the directory will have unique CIDs. Web3.storage wraps files in a directory by default, which is why you see a directory listing with the file `welcome-to-IPFS.jpg` followed by a shorthand CID: `bafk…beom` of the file.
99
+
:::callout
100
+
When pinning a file to IPFS, the filename is not stored by default. To ensure the filename is retained, it's common to wrap the file in a directory. In such instances, both the file and the directory will have unique CIDs. Web3.storage wraps files in a directory by default, which is why you see a directory listing with the file `welcome-to-IPFS.jpg` followed by a shorthand CID: `bafk…beom` of the file.
101
+
:::
97
102
98
103
## Summary and next steps
99
104
100
105
In this quickstart guide, you learned about [pinning services](../concepts/persistence.md#pinning-in-context), and how to use them to publish content-addressed data with IPFS. You also learned how CIDs address files and directories in IPFS by uploading a file to a pinning service called [web3.storage](https://web3.storage/).
101
106
102
107
Pinning services provide a convenient alternative to running IPFS nodes and infrastructure. However, the two are not mutually exclusive; you can combine a pinning service with an IPFS node on your computer to increase the resilience of your CIDs.
103
108
104
-
Check out [the lifecycle of data in IPFS](../concepts/lifecycle.md) to learn more about how publishing by pinning fits into the full lifecycle of data in IPFS.
105
-
106
-
As a next step, try fetching the pinned file, by following the [retrieval quickstart](./retrieve.md).
107
-
108
-
If you prefer programmatically uploading with JavaScript or Go, check out the [Web3.storage docs](https://web3.storage/docs/how-tos/store/).
109
+
Possible next steps include:
109
110
110
-
Alternatively, try out [Filebase](https://filebase.com/), a pinning service that offers an [AWS S3-compatible API for pinning](https://docs.filebase.com/getting-started/s3-api-getting-started-guide), so you can use any S3-compatible SDK, e.g. [aws-sdk](https://www.npmjs.com/package/aws-sdk), and [many](https://github.com/s3tools/s3cmd) more.
111
+
- Check out [the lifecycle of data in IPFS](../concepts/lifecycle.md) to learn more about how publishing by pinning fits into the full lifecycle of data in IPFS.
112
+
- Try fetching the pinned file by following the [retrieval quickstart](./retrieve.md).
113
+
- If you prefer to upload a file programmatically with JavaScript or Go, check out the [Web3.storage docs](https://web3.storage/docs/how-tos/store/).
114
+
- Alternatively, try out [Filebase](https://filebase.com/), a pinning service that offers an [AWS S3-compatible API for pinning](https://docs.filebase.com/getting-started/s3-api-getting-started-guide), so you can use any S3-compatible SDK, e.g. [aws-sdk](https://www.npmjs.com/package/aws-sdk), and [many](https://github.com/s3tools/s3cmd) more.
0 commit comments