Skip to content

Commit bf84e0a

Browse files
djdvdjdv
committed
doc: nascent documentation
1 parent b413c9e commit bf84e0a

File tree

11 files changed

+1771
-0
lines changed

11 files changed

+1771
-0
lines changed

‎doc/assets/filesystem.svg‎

Lines changed: 875 additions & 0 deletions
Loading

‎doc/assets/goda-short.svg‎

Lines changed: 256 additions & 0 deletions
Loading

‎doc/assets/interactions.svg‎

Lines changed: 113 additions & 0 deletions
Loading

‎doc/development.md‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# Architecture
2+
3+
![goda-cluster-short](assets/goda-short.svg)
4+
5+
## Package documentation
6+
7+
The Go project hosts a documentation server that can be accessed here:
8+
https://pkg.go.dev/github.com/djdv/go-filesystem-utils/internal
9+
The public `pkgsite` service hides a lot of information by default, so developers are encouraged
10+
to run their own local [`godoc`](https://pkg.go.dev/golang.org/x/tools/cmd/godoc) server (or any
11+
equivalent documentation server such as `golds`, etc.).
12+
`godoc` allows an `m=all` query string, which will render internal packages and unexported
13+
identifiers.
14+
E.g. `godoc -http:6060` then navigate to "http://127.0.0.1:6060/pkg/github.com/djdv/go-filesystem-utils/?m=all".
15+
16+
This project separates platform differences at compile time via build constraints, so other
17+
useful query strings are `GOOS` and `GOARCH` which work with both `pkgsite` and `godoc`.
18+
By default, the documentation servers will render documentation for the `$GOOS` and `$GOARCH` of
19+
the requester, but these can be specified explicitly to request documentation sets for other
20+
combinations.
21+
E.g. "https://pkg.go.dev/os?GOOS=windows", "https://pkg.go.dev/os?GOOS=darwin&GOARCH=amd64", etc.
22+
23+
As APIs stabilize, packages may be moved out of `/internal`, potentially into separate repos.
24+
25+
## `fs` command
26+
27+
### Command line interface
28+
29+
The `fs` command serves multiple roles, acting as both a client and server process.
30+
The package `command` facilitates multiplexing and dispatching between our subcommands.
31+
32+
### Command implementation
33+
34+
(Sub)commands are defined as Go functions that have a signature compatible with
35+
one of the `command.Make*Command` constructors.
36+
E.g. `func execute(ctx context.Context, arbitrary ExecuteType...)`
37+
which is compatible with the `command.MakeVariadicCommand` constructor.
38+
39+
The `ExecuteType` constraint may be any Go type which satisfies its interface.
40+
Specifically, this includes a `BindFlags(*flag.FlagSet)` method which registers the type with
41+
Go's standard `flag.FlagSet` type.
42+
43+
Package `command` will handle parsing the command line and resolving subcommands, before passing
44+
the expected parameters to the execute function.
45+
46+
For more details, you can read the examples in the `command` package, or read the
47+
implementations used by the `fs` commands.
48+
49+
### `fs` execute function implementations
50+
51+
The execute functions used by the `fs` commands typically accept a variadic list of "option"
52+
functions, which are applied to a "settings" structure if provided.
53+
This is the most complicated function type supported by the `command` package, but carries some
54+
benefits in regards to lazy evaluation, handling of default values, and chaining shared options
55+
across commands.
56+
57+
*Note: the actual implementations are more abstracted than these examples, but the principles
58+
remain the same.*
59+
60+
```go
61+
type (
62+
fooSettings struct {
63+
value int
64+
}
65+
fooOption func(*fooSettings)
66+
)
67+
68+
func fooExecute(ctx context.Context, options ...fooOption) error {
69+
settings := fooSettings{
70+
value: 1, // Default.
71+
}
72+
for _, apply := range options {
73+
apply(&settings) // Override defaults.
74+
}
75+
// Execute foo command with settings...
76+
return nil
77+
}
78+
```
79+
80+
This function signature can satisfy the `command.ExecuteVariadic` constraint, which allows the
81+
`ExecuteType` to be a pointer to any underlying slice type.
82+
To do this, we implement a method which binds to a `flag.FlagSet` on a pointer receiver that
83+
matches the variadic type.
84+
85+
```go
86+
type fooOptions []fooOption // Same underlying type as `...fooOption`.
87+
88+
// Satisfy the [command.ExecuteType] constraint.
89+
func (fo *fooOptions) BindFlags(flagSet *flag.FlagSet) {
90+
flagSet.Func("flag", "help text", func(parameter string) error {
91+
value, err := parse(parameter)
92+
// Handle err...
93+
*fo = append(*fo, func(settings *fooSettings) {
94+
settings.value = value
95+
})
96+
return nil
97+
})
98+
}
99+
```
100+
101+
We can then call the command constructor to create a formal command.
102+
103+
```go
104+
func Foo() command.Command {
105+
const (
106+
name = "foo"
107+
synopsis = "Frobnicates a bar."
108+
usage = "Foo command long help text..."
109+
)
110+
return command.MakeVariadicCommand[fooOptions](name, synopsis, usage, execute)
111+
}
112+
```
113+
114+
## Client <-> Server <-> OS interactions
115+
116+
![interactions](assets/interactions.svg)
117+
118+
### Client commands
119+
120+
Client commands are primarily only responsible for parsing the command line vector,
121+
instantiating a connection to a server, and passing parameters to Go interfaces which give names
122+
to common 9P operation sequences.
123+
124+
If an address is not specified via the `-api-server` flag, client commands will try to connect
125+
to a default local address. If the default server cannot initially be dialed, the client command
126+
will automatically spawn a daemon process that will exit when it becomes idle for some time.
127+
(See: help text for `-api-exit-after` flag)
128+
129+
### Server command
130+
131+
`fs daemon` is the server component which responds to requests made by client commands.
132+
The daemon is typically run in the background and persists to facilitate longstanding operations
133+
such as engaging with the host's file system APIs.
134+
135+
When the daemon starts, it first initializes its 9P file system, then tries to serve this system
136+
on an address. Afterwards, it launches a number of event monitors which handle tasks
137+
like IPC communication, shutdown requests, idle detection, and more.
138+
139+
140+
### Daemon IPC
141+
142+
The daemon can communicate with a parent process using the 9P2000.L protocol over standard
143+
input and output, as well as text over standard error.
144+
When the parent process is finished with the service, it must write the `EOT` byte to the file
145+
`/control/release`.
146+
147+
<!-- vi: set textwidth=96: -->

‎doc/generate.ps1‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
$moduleName = "go-fs-documentation"
2+
$modulePath = Join-Path . "$($moduleName).psm1"
3+
4+
Import-Module -Name $modulePath
5+
New-GoFSDocumentation
6+
Remove-Module -Name $moduleName

‎doc/go-fs-documentation.psm1‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
function Get-GoFSDocumentationCommandTable {
2+
$dependencies = @{
3+
'go' = 'https://go.dev/'
4+
'goda' = 'https://github.com/loov/goda'
5+
'dot' = 'https://graphviz.org/'
6+
'd2' = 'https://github.com/terrastruct/d2'
7+
}
8+
$commands = @{}
9+
foreach ($application in $dependencies.GetEnumerator()) {
10+
$name = $application.Name
11+
try {
12+
$command = Get-Command -Name $name -CommandType Application -ErrorAction Stop
13+
} catch {
14+
throw "Required command ``$name`` was not found. See: $($dependencies[$name])"
15+
}
16+
$commands[$name] = $command
17+
}
18+
return $commands
19+
}
20+
21+
function New-GoFSDocumentation {
22+
[CmdletBinding(SupportsShouldProcess)]
23+
param()
24+
25+
$commands = Get-GoFSDocumentationCommandTable
26+
$resultDirectory = Join-Path . assets
27+
[void](New-Item -ItemType Directory -ErrorAction Ignore -Path $resultDirectory)
28+
29+
# Include all of our modules, but exclude the build tool.
30+
# Note: Module separator '/', not host separator.
31+
$godaExpression = '../... - ../cmd/build'
32+
$dependencyGraph = 'goda-short.svg'
33+
$dependencyGraphPath = $(Join-Path $resultDirectory $dependencyGraph)
34+
if ($PSCmdlet.ShouldProcess($dependencyGraphPath, 'Generate dependency graph')) {
35+
Write-Information "Generating graph: `"$($dependencyGraphPath)`""
36+
& $commands['goda'] graph -short -f='{{.ID}}' $godaExpression | & $commands['dot'] -Tsvg -o $dependencyGraphPath
37+
}
38+
39+
$verbose = $InformationPreference -eq [System.Management.Automation.ActionPreference]::SilentlyContinue
40+
Get-ChildItem -Path $(Join-Path graphs *) -Include *.d2 | ForEach-Object {
41+
$vectorName = "$($_.BaseName).svg"
42+
$vectorPath = Join-Path $resultDirectory $vectorName
43+
if (!$PSCmdlet.ShouldProcess($vectorPath, 'Generate vector')) {
44+
return
45+
}
46+
Write-Information "Generating graph: `"$vectorPath`""
47+
# d2 uses stderr for all logging.
48+
# Redirect it and only print it conditionally.
49+
$startInfo = [System.Diagnostics.ProcessStartInfo]::new(
50+
$commands['d2'].Source,
51+
"`"$($_.FullName)`" `"$($vectorPath)`""
52+
)
53+
$startInfo.RedirectStandardError = $true
54+
$process = [System.Diagnostics.Process]::Start($startInfo)
55+
$stderr = $process.StandardError.ReadToEnd()
56+
$process.WaitForExit()
57+
if ($process.ExitCode) {
58+
$sourceRelative = Resolve-Path -Relative -Path $_.FullName
59+
Write-Error "`"$($sourceRelative)`"` -> `"$($vectorPath)`" $stderr"
60+
} elseif ($verbose) {
61+
Write-Information $stderr
62+
}
63+
}
64+
}

‎doc/graphs/filesystem.d2‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
file system {
2+
control/ {
3+
shutdown: |md
4+
### Shutdown file
5+
name: "shutdown"
6+
write: shutdown disposition value
7+
(string or decimal; e.g. "patient" or 1)
8+
|
9+
}
10+
listeners/ {
11+
/\.\.\.$maddr-components\.\.\./ {
12+
socket: |md
13+
### Socket file
14+
name: "listener"
15+
read: multiaddr string
16+
|
17+
connections/ {
18+
conn: |md
19+
### Connection file
20+
name: ${decimal number}
21+
read: connection metadata as JSON
22+
|
23+
link: schemas/connection.json
24+
}
25+
}
26+
}
27+
mounts/ {
28+
$host-API-name/ {
29+
$guest-API-name/ {
30+
mount-file: |md
31+
### Mount point file
32+
name: ${NanoID}.json
33+
read: mount point metadata as JSON
34+
|
35+
link: schemas/mountpoint.json
36+
}
37+
}
38+
}
39+
}

‎doc/graphs/interactions.d2‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
direction: right
2+
cli: Command Line
3+
fs-process: |
4+
1st-party client process
5+
(fs mount, etc.)
6+
|
7+
fs-process.shape: rectangle
8+
ext-process: |
9+
3rd-party client process
10+
(shell extension dll, etc.)
11+
|
12+
ext-process.shape: rectangle
13+
fs-daemon: |
14+
File system service
15+
(fs daemon)
16+
|
17+
fs-daemon.shape: rectangle
18+
19+
cli -> fs-process: Text parser
20+
fs-process <-> fs-daemon: 9P {
21+
style.animated:true
22+
}
23+
ext-process <-> fs-daemon: 9P {
24+
style.animated:true
25+
}
26+
host: |
27+
Host API
28+
(FUSE, 9P, et al.)
29+
|
30+
host.shape: rectangle
31+
guest: |
32+
Guest API
33+
(IPFS, PinFS, et al.)
34+
|
35+
guest.shape: rectangle
36+
os: Operating system
37+
fs-daemon <-> guest: Go API {
38+
style.animated: true
39+
}
40+
fs-daemon <-> host: Go API {
41+
style.animated: true
42+
}
43+
host <-> os: API|ABI {
44+
style.animated: true
45+
}

‎doc/schemas/connection.json‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"type": "object",
3+
"properties": {
4+
"lastRead": {
5+
"type": "string"
6+
},
7+
"lastWrite": {
8+
"type": "string"
9+
},
10+
"local": {
11+
"type": "string"
12+
},
13+
"remote": {
14+
"type": "string"
15+
},
16+
"#": {
17+
"type": "integer"
18+
}
19+
},
20+
"required": [
21+
"lastRead",
22+
"lastWrite",
23+
"local",
24+
"remote",
25+
"#"
26+
]
27+
}

‎doc/schemas/mount.json‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"type": "object",
3+
"properties": {
4+
"data": {
5+
"type": "object",
6+
"properties": {
7+
"host": {
8+
"type": "object"
9+
},
10+
"guest": {
11+
"type": "object"
12+
}
13+
}
14+
},
15+
"tag": {
16+
"type": "object",
17+
"properties": {
18+
"host": {
19+
"type": "string",
20+
"enum": [
21+
"FUSE"
22+
]
23+
},
24+
"guest": {
25+
"type": "string",
26+
"enum": [
27+
"IPFS",
28+
"IPNS",
29+
"PinFS",
30+
"KeyFS"
31+
]
32+
}
33+
},
34+
"required": [
35+
"host",
36+
"guest"
37+
]
38+
}
39+
},
40+
"required": [
41+
"data",
42+
"tag"
43+
]
44+
}

0 commit comments

Comments
 (0)