Skip to content

Commit e683942

Browse files
authored
Merge pull request #1714 from ipfs/add/distrib-compute
Add CoD concepts section
2 parents 5d80535 + d7705a5 commit e683942

File tree

2 files changed

+57
-0
lines changed

2 files changed

+57
-0
lines changed

docs/.vuepress/config.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,7 @@ module.exports = {
177177
]
178178
},
179179
'/concepts/ipfs-implementations',
180+
'/concepts/cod',
180181
'/concepts/comparisons',
181182
'/concepts/usage-ideas-examples',
182183
['/concepts/measuring', 'Measuring the network'],

docs/concepts/cod.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
---
2+
title: Compute-over-Data (CoD)
3+
description: Learn about compute-over-data with content-addressed data on IPFS
4+
---
5+
6+
# Compute-over-Data with content-addressed data
7+
8+
The term "Compute-over-data" (CoD) generally refers to a computing paradigm where processing of data is performed near the location of the data. This concept is particularly relevant in the context of big data and distributed computing, where the transfer of large volumes of data over a network can be inefficient and costly. By performing computations close to where the data is stored (compute-over-data), faster processing speeds and lower network bandwidth requirements are possible.
9+
10+
IPFS users can perform CoD on IPFS data with the [Bacalhau platform](#bacalhau) and the [InterPlanetary Virtual Machine (IPVM) specification](#ipvm), both of which natively support content-addressed data.
11+
12+
## Bacalhau
13+
14+
Bacalhau is a platform for fast, cost-efficient, secure, distributed computation. Bacalhau works by running jobs where the data is generated and stored, also referred to as Compute Over Data (or CoD). Using Bacalhau, you can streamline existing workflows without extensive refactoring by running arbitrary Docker containers and WebAssembly (Wasm) images as compute tasks. The name _Bacalhau_ was coined from the Portuguese word for "salted cod fish".
15+
16+
### Features
17+
18+
Bacalhau can:
19+
20+
- Simplify management of compute jobs by providing a **unified platform** for managing jobs across different regions, clouds, and edge devices.
21+
- Provide reliable and network-partition resistant orchestration, ensuring jobs will complete even if there are network disruptions.
22+
- Provide a complete and permanent audit log, so you can be confident that jobs are being executed securely.
23+
- Run [private workloads](https://docs.bacalhau.org/next-steps/private-cluster) to reduce the chance of leaked data outside of your organization.
24+
- Reduce ingress and egress costs since jobs are processed closer to the source.
25+
- Run against data [mounted anywhere](https://docs.bacalhau.org/#how-it-works) on your machine.
26+
- Integrate with services running on nodes to run jobs, such as [DuckDB](https://docs.bacalhau.org/examples/data-engineering/DuckDB/).
27+
- Operate at scale over parallel jobs and batch process petabytes of data.
28+
- Auto-generate art using a [Stable Diffusion AI model](https://www.waterlily.ai/) trained on the chosen artist’s original works.
29+
30+
### More Bacalhau resources
31+
32+
- [Getting started tutorial](https://docs.bacalhau.org/getting-started/installation/)
33+
- [Bacalhau platform architecture](https://docs.bacalhau.org/getting-started/architecture/)
34+
- [GitHub](https://github.com/bacalhau-project/bacalhau)
35+
36+
## IPVM
37+
38+
The InterPlanetary Virtual Machine (IPVM) specification defines the easiest, fastest, most secure, and open way to run decentralized compute jobs on IPFS. One way to describe IPVM would be as "an open, decentralized, and local-first competitor to AWS Lambda".
39+
40+
IPVM uses [WebAssembly (Wasm)](https://webassembly.org/), content addressing, [simple public key infrastructure (SPKI)](https://en.wikipedia.org/wiki/Simple_public-key_infrastructure), and object capabilities to liberate computation from specific, prenegotiated services, such as large cloud computing providers. By default, execution scales flexibly on-device, all the way up to edge points-of-presence (PoPs) and data centers.
41+
42+
The core, Rust-based implementation and runtime of IPVM is the [Homestar project](https://github.com/ipvm-wg/homestar/). IPVM supports interoperability with [Bacalhau](https://bacalhau.org) and [Web3Storage](https://web3.storage/)
43+
44+
### More IPVM resources
45+
46+
- [github.com/ipvm-wg/homestar/](https://github.com/ipvm-wg/homestar/)
47+
- [Seamless Services for an Open World](https://youtu.be/Kr3B3sXh_VA) by Brooklyn Zelenka
48+
- [Foundations for Open-World Compute](https://youtu.be/dRz5mau6fsY) by Zeeshan Lakhani
49+
- [Foundations for Open-World Compute: Homestar, an IPVM Tale](https://youtu.be/BFAMy5-VHak) by Zeeshan Lakhani
50+
- [IPVM: The Long-Fabled Execution Layer](https://www.youtube.com/watch?v=3y1RB8wt_YY) by Brooklyn Zelenka
51+
- [IPVM: Content Addressed Compute for an Open World](https://youtu.be/jhtEYr3ORfk) by Brooklyn Zelenka
52+
- [IPVM - IPFS and WASM](https://www.youtube.com/watch?v=rzJWk1nlYvs) by Brooklyn Zelenka
53+
- [IPVM: Use Cases & System Designs](https://www.youtube.com/watch?v=FhwzEKNZEIA) by Juan Benet
54+
- [IPVM: High-Level Spec](https://github.com/ipvm-wg/spec)
55+
- [IPVM Workflow Spec](https://github.com/ipvm-wg/workflow)
56+
- [UCAN Invocation Spec](https://github.com/ucan-wg/invocation)

0 commit comments

Comments
 (0)