Init Design

[groundwater]

Init is a job-control service.

HTTP IPC

You communicate with init over http.

For example, to start a job you PUT to /job/:name where :name is the unique identifier for your job.

{
  "tasks": [
    {
      "exec": "node",
      "args": ["server.js"],
      "envs": {
        "PATH": "/bin:/root/lib/node_modules/myapp/node_modules/.bin"
      },
      "cwd": "/root/lib/node_modules/myapp",
      "fd": [...],
      "user": 2,
      "group": 3
    }
  ]
}

All of the above are required, aside from user and group.

File Descriptors

File descriptors are pointers to various IO conduits of a process. There are many things that can be represented by a file descriptor,

• a normal file on the file system
• an unnamed pipe between two processes
• a named pipe
• a network socket
• a unix domain socket
• a block or character device

Each of the above is represented by a file-descriptor, and various system calls can be made to each descriptor based on the thing it represents.

Going forward, we probably want init to work with all of the above.

{ // a file
  type: "file",
  path: "/root/docs/myfile.txt",
  mode: "rw"  
}

{ // a unix socket
  type: "unix socket",
  path: "/root/var/myapp.sock"
}

{ // a network socket
  type: "network socket",
  bind: "127.0.0.1",
  port: 8080
}

{ // a named pipe
  type: "pipe",
  path: "..."
}

I don't want to support all of these yet, but it's something to keep in mind. I'm also not sure how to represent unnamed pipes between processes.

Init should respond with 501 Not Implemented, rather than a 400/500 error when using an unsupported file type.

Authentication

HTTP is secure. If you think otherwise, you should stop using all websites.

HTTP security can be poorly implemented however. We actually punt on handling security init the server. Instead, we require that you secure the socket.

For simple cases, you can bind to localhost. Only processes on the same host can access init.

A better approach might be binding to a unix domain socket, for example /root/var/init.sock. You can use file-system permissions to restrict access to the http server. This is how Docker works.

Permissions

There are no permissions. Either you have access to everything, or nothing.

In a multi-user environment, each user will run their own init service.

-+ init (root)
  \
   +-+ init (bob)
   |  \
   |   server.js (bob's server)
   |
   +-+ init (tom)
      \
       server.js (tom's server)

Restart Semantics

There are no restart semantics. You can restart a job by defining a second restart task, e.g.

{
  tasks: [
    {...}, // job to run 
    {...}  // command to restart job
  ]
}

Restart semantics are difficult

• A process that exits immediately due to an error can peg the CPU at 100%.
• Generally we do not want to ever stop trying to restart a process.
  Do you care more if your CPU gets pegged for a while, or your website goes down?
• Sometimes we want notifications when processes exit.

We let the caller to init decide how they want to handle restart.

• Tasks are always run serially.
  The previous task always exist before the next one starts.
• The exit code of the previous process is available via the
  environment variable TASK_EXIT.
  This will either be the number or signal name that
  caused the previous task to exit.

Review

• Init is simple, yet flexible.
• Init is focused on running tasks, and jobs.
• Init should export a well-defined API, to allow multiple products interact with it.

cc @piranna

[groundwater]

I've been thinking about the actual role of the above init, and the process with pid 1. The pid 1 process really only has two duties:

1. reap zombie children
2. begin the user-land boot process

Also pid 1 cannot ever exit, or that will cause a kernel panic.

If a parent process exists before it's children, then init automatically becomes the new parent. When those children exit, it's init's duty to call wait or waitpid to free the pid and exit status in the kernel's process table.

So maybe the true pid 1 process should just do that, and then start the above init server. Our init server will no longer be init init, but that might be correct design.

[piranna]

Seems good to have a minimal init that only works as safenet for orphan process, 👍. Otherwise, what would happens? Kernel catch them? Kernel panic dispatched? Is mandatory to be pid 1 or could be pid 0?

Your init specification is also ok, just two comments:

• the type of the file-description can be inferred easily from a plain string
• probably the unix socket or the named pipe (are they the same?) use less resources

[groundwater]

The way linux works, and most *nix systems work, is the kernel assigns orphan processes to be the child of pid 1. The kernel only panics if pid 1 exits. There isn't a user-space pid 0.

the type of the file-description can be inferred easily from a plain string

The type can only be inferred for an existing file, which might be an okay requirement. I don't expect people to call this API manually, they will use a command like npkg to start/stop processes, so explicit beats implicit. The npkg command can take care of making assumptions about what the user wants.

For example

$ npkg start --stdout=out.log ./myapp

The npkg command can create out.log if it doesn't already exist, and tell init that it should open out.log as a writable file. Init should not infer, but npkg absolutely can.

probably the unix socket or the named pipe (are they the same?) use less resources

A unix socket is not the same as a named pipe. A unix domain socket is a full socket, capable of duplex communication. A named pipe is just a one-way stream of bytes.

[piranna]

The way linux works, and most *nix systems work, is the kernel assigns orphan processes to pid 1. The kernel only panics if pid 1 exits. There isn't a user-space pid 0.

Thanks for the info, didn't know how it works. Ok, then this confirms
init should be as minimal as possible, and only work as a
safenet for orphan processes. I think this would be the safest to do
here.

the type of the file-description can be inferred easily from a plain string

The type can only be inferred for an existing file, which might be an okay requirement. I don't expect people to call this API manually, they will use a command like npkg to start/stop processes, so explicit beats implicit. The npkg command can take care of making assumptions about what the user wants.

For example

$ npkg start --stdout=out.log ./myapp

The npkg command can create out.log if it doesn't already exist, and tell init that it should open out.log as a writable file. Init should not infer, but npkg absolutely can.

Ok, I agree.

probably the unix socket or the named pipe (are they the same?) use less resources

A unix socket is not the same as a named pipe. A unix domain socket is a full socket, capable of duplex communication. A named pipe is just a one-way stream of bytes.

Ok, thanks. In Plan9 (it's what we studied in Design of Operating
Systems classes) pipes are bi-directional, maybe my confussion came
from here.

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

[groundwater]

I believe @jbenet also suggested following some of the design around Plan 9.

init should be as minimal as possible, and only work as a
safenet for orphan processes

I agree here. If you wanted to take a stab at it, please add me and I'll contribute.

[piranna]

I believe @jbenet https://github.com/jbenet also suggested following
some of the design around Plan 9.

I've always though Python file-like objects would fits nice with the
file-oriented interfaces of Plan9, but Node.js stream objects would also do
the trick well... :-)

init should be as minimal as possible, and only work as a
safenet for orphan processes

I agree here. If you wanted to take a stab at it, please add me and I'll
contribute.

I'm currently just doing comments as a way to dis-stress of exams, but I
could start contributing code on July Until then, I will be only here :-)

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton
de sitios diferentes, simplemente escribe un sistema operativo Unix."
– Linus Tordvals, creador del sistema operativo Linux

[jbenet]

There are no permissions. Either you have access to everything, or nothing.

I think aiming for Least Privilege semantics is TRTTD. This gets a lot easier with capabilities.

(Without looking at how v8 does things and what's exploitable...) JS has a good isolation model in theory. Nothin except vars in local context. You could return "views" of the OS resources that limit (or virtualize) access. Think chroot jails/containers/docker but at the js level. So like, a parent module could cause fs = require('fs') within a child module to look a certain way (i.e. isolate it).

Sandboxing might be a killer app reason for nodeOS.

I believe @jbenet also suggested following some of the design around Plan 9

In particular, super cheap process startup. In node, modules are imported into the same process. As @groundwater mentioned in jbenet/random-ideas#9 v8 isolates might be a really good way to do this. (not sure how cheap they are).

That could be another killer app reason to use nodeOS.

[groundwater]

I think aiming for Least Privilege semantics is TRTTD. This gets a lot easier with capabilities.

Each copy of init spawns jobs requested by a particular user, so that user needs access to all jobs supervised by a single init process. We however don't want jobs to have access to each other. If jobs are spawned under the current user, then jobs will be able to call init on behalf of the user.

I agree that one of the killer features of node-os might be the ability to sandbox everything. If you're installing a daemon from npm you probably want some form of isolation, since the package could literally do anything.

On linux, the clone system call governs all new process (and thread) creation. Clone lets you decide what to share, and what not to share. If we require that all modules must contain all their dependencies, we could isolate each service in its own file system. This would cut the service off from having access to init.

I think we should isolate processes from one and other, but I think putting access control in the init process is the wrong place (for now).

(Without looking at how v8 does things and what's exploitable...) JS has a good isolation model in theory. Nothin except vars in local context. You could return "views" of the OS resources that limit (or virtualize) access. Think chroot jails/containers/docker but at the js level. So like, a parent module could cause fs = require('fs') within a child module to look a certain way (i.e. isolate it).

This is a whole other topic, but it's a really interesting one. I want to come back to it later, perhaps by opening a new ticket on NodeOS/NodeOS. I'll be sure to ping ya'll when I do.

[groundwater]

Here is my first pass at a minimal init module century.

It contains a compiled module, because node has no mechanism for waiting on children it didn't spawn. You can test it with the demo.js file included.

[piranna]

Seems fairly minimal :-) Seems would need to compile it by hand, isn't it? You could use https://github.com/TooTallNate/node-gyp as dependency so it's build automatically from inside NodeOS, and also there's the posibility to distribute binary builds (on Node-WebRTC) we are talking about it.

Regarding your code, on https://github.com/groundwater/node-century/blob/master/century.js#L64 would be good to stop the setInterval() function, seems to be cleaner and also probably it would exit automatically the process since the events queue is already empty.

[groundwater]

Seems would need to compile it by hand, isn't it?

It will compile automatically on npm install

also there's the posibility to distribute binary

This is a general problem for node-os and I'm still working on a good general solution 👍

Regarding your code, on https://github.com/groundwater/node-century/blob/master/century.js#L64 would be good to stop the setInterval() function

I want to exit with the same code as the child process.