Advanced usage

Preamble

ZettelGeist makes use of many tools to do its job. We embrace the Unix philosophy by using the best available tools to make a full notetaking, outlining, and writing system come to full realization. We are grateful to these projects, without which our own efforts would not be possible.

Prerequisites

Please Install ZettelGeist before spending any more time reading this page.

Install these Tools

Pandoc

Pandoc is among our favorite tools at Team ZettelGeist. We embrace the Markdown format for writing and use it for Zettels that have ZettelGeist metadata and Markdown documents as the document body.

Installing Pandoc is easy:

• For OS X, just brew install pandoc.
• For Linux, go to the Linux installation page and just download the .deb file and install with dpkg.
• For Windows/WSL2, you should follow the Linux instructions as well. ZettelGeist is only supported on Windows via WSL 2.

yq

Beyond the native capabilities ZettelGeist offers for working with our zettel format, there are times when we want to inspect the raw YAML metadata and extract it for other purposes. The yq (YAML Query) project is a go-to project for achieving this.

• For OS X, just brew install yq
• For Ubuntu Linux, you can use sudo snap install yq.
• For Windows/WSL2, unfortunately, snaps are not supported out of box. The easiest way is to grab a link to the latest release and put it in /usr/local/bin, which should be in your PATH.

$ wget -O yq https://github.com/mikefarah/yq/releases/download/v4.8.0/yq_linux_amd64
$ chmod +x yq
$ sudo mv yq /usr/local/bin/yq

Zettel Database Use Cases

Overview

Because it will be an ongoing process to make all of the needed command-line tools for various scenarios, we are working to understand the use cases by working directly with our ephemeral database. The database is nothing more than a single table that indexes all of YAML fields. Hereafter, when we are working with the index of Zettels, we will use the term ZDB (Zettel database).

Match the filename

This allows us to figure out what Zettels come from a given imported file. In this case, I had imported a file 01-Numeration.yaml that contains about 30 or so entries.

sqlite> select rowid,title from zettels where filename match 'Numeration';

2|Numbers
3|Readings
4|Symbols
5|Clay Tablet
6|Symbols
7|Numeration
8|Tally Stick
9|Tokens
10|Concept of Number
11|Hieroglyphic numbers
12|Two examples
13|12,425 Birds
14|Egyptians Hieroglyphics (pictographic symbols)
15|Hieroglyphics Addition

Count number of Zettels in the database

(zenv) thirteen:zettelgeist gkt$ sqlite3 zettels.db

sqlite> select count(rowid) from zettels;
477

Find context where a term is used

This query shows how to do a contextual search. The output is formatted such that it would be rendered in Markdown. When a term is found, we surround it with brackets and place emphasis on it.

sqlite> select filename, snippet(zettels, '*[', ']*', ' ... ', -25, 25) as context from zettels where text match "Castells";

20170411170205-01-Castels1-BIBnote.yaml|BibTex first note for [Castells]

201704171604-01-Syllabi--Sigsis-HPSS_S689.yaml| ... the Virtual Life: The Culture and Politics of Information (City Lights, 1995)

[Castells], Manual. The Information Age: Economy, Society and Culture, 3v. (Blackwell, 1996).

Dreyfus ... 201704171604-01-Syllabi--Sigsis-hssc515syllabus.yaml| ... Internet

Using Query at Command Line

Any query can be done at the command line, too.

This example is a bit crazier. We are doing an FTS match on the text and filename fields (from our Zettels). The output will be written in "line" mode. (Query results not shown here.)

sqlite3 -line zettels.db 'select filename from zettels where zettels match "text:Antikythera -filename:computing";'

Principles of Development

The initial idea is to build a command line interface for ALL of the Zettelkasten ideas. The hope is to build something, primarily, from existing open source tools. There would be some scaffolding that comes in the form of Python scripts, but there are many existing tools that can reduce the amount of code we need to write--substantially.

Here are a few:

• sqlite3 + fts4/fts5. This embedded database software can be used to support indexing of all of the notes. With the fts extensions, we can get full-text search, so we can search the plaintext to identify patterns in the data that may not have been directly classified when a note was created.

• Markdown - This is a great format that preserves the advantages of minimal formatting but allows for the occasional emphasis that might be desired when creating a note. Much as Evernote and others allow simple formatting, there is no reason for us to stop folks from having some formatting, especially if we want to render somewhat “pretty” summary pages or present the notes via the web.

• Pandoc - Pandoc is an amazing tool that can convert from one format to another. We’ll especially want this for rendering various output formats. For example, we can transform a collection of notes into a Word document (or HTML, or whatever) from the command line.

• Jekyll - Although not directly related, Jekyll is a tool for creating web sites from plaintext, Markdown or HTML page fragments. You can use this to generate an entire web site that looks like someone spent years preparing it. My own web site uses it, and the format is similar to what we’ll use for our own notes. Jekyll also has a “data” framework that could allow us to build a web viewer for our collection of Zettels without too much pain.

• YAML - Is a great way to represent metadata. It is what XML could have been, if only the creators had thought more about the things people need to do with markup. We’ll use this format for metadata at the beginning of a document.