Skip to content

Shell

Jump to bottom Edit New page
David Foster edited this page Jan 27, 2026 · 19 revisions

Crystal can be launched with an interactive Python shell. This allows you to perform advanced manipulations on a project manually, and to manipulate projects with automated scripts.

Quickstart

Tell Crystal to open a shell by passing the --shell CLI option:

$ crystal --shell
Crystal 2.1.0 (Python 3.14.2)
Type "help" for more information.
Variables "project" and "window" are available.
Use exit() or Ctrl-D (i.e. EOF) to exit.
>>>

The graphical Open/Create Project dialog will still appear and you can use it to open a project.

After a project is opened, you can manipulate it in the shell with the project variable:

>>> project
<crystal.model.Project object at 0x115a61bb0>
>>> help(project)  # display interactive help

>>> list(project.root_resources)
[RootResource('Home Page','https://xkcd.com/')]

>>> list(project.resource_groups)
[ResourceGroup('Comic','https://xkcd.com/#/')]

>>> len(project.resources)
16313
>>> list(project.resources)[0]
Resource('https://xkcd.com/')

>>> first_comic = project.get_resource('https://xkcd.com/1/')
>>> first_comic
Resource('https://xkcd.com/1/')

>>> list(first_comic.revisions())
[<ResourceRevision 14 for 'https://xkcd.com/1/'>]
>>> first_comic.default_revision()
<ResourceRevision 14 for 'https://xkcd.com/1/'>

>>> project.get_resource('https://xkcd.com/99/')
>>> # not in project
>>> from crystal.model import Resource
>>> some_comic = Resource(project, 'https://xkcd.com/99/')
>>> some_comic
Resource('https://xkcd.com/99/')

>>> list(some_comic.revisions())
[]
>>> some_comic.default_revision()
>>> # no default revision
>>> revision_future = some_comic.download()
>>> revision = revision_future.result()  # keep result() separate from download(); will deadlock otherwise
>>> revision
<ResourceRevision 18 for 'https://xkcd.com/99/'>
>>> import pprint
>>> pprint.pprint(revision.metadata)
{'headers': [['Connection', 'keep-alive'],
             ['Content-Length', '8297'],
             ['Server', 'nginx'],
             ['Content-Type', 'text/html; charset=UTF-8'],
             ['Last-Modified', 'Sat, 16 Jul 2022 01:52:42 GMT'],
             ['ETag', '"62d219ea-2069"'],
             ['Expires', 'Mon, 18 Jul 2022 15:18:54 GMT'],
             ['Cache-Control', 'max-age=300'],
             ['Via', '1.1 varnish, 1.1 varnish'],
             ['Accept-Ranges', 'bytes'],
             ['Date', 'Mon, 18 Jul 2022 15:13:54 GMT'],
             ['Age', '0'],
             ['X-Served-By', 'cache-dfw18663-DFW, cache-bfi-krnt7300049-BFI'],
             ['X-Cache', 'MISS, MISS'],
             ['X-Cache-Hits', '0, 0'],
             ['X-Timer', 'S1658157234.020196,VS0,VE58'],
             ['Vary', 'Accept-Encoding']],
 'http_version': 11,
 'reason_phrase': 'OK',
 'status_code': 200}
>>> with revision.open() as f:
...     body = f.read()
>>> body[:141]
b'<!DOCTYPE html>\n<html>\n<head>\n<link rel="stylesheet" type="text/css" href="/s/7d94e0.css" title="Default"/>\n<title>xkcd: Binary Heart</title>'

When you're done, exit the shell by pressing Ctrl-D (on macOS and Linux), Ctrl-Z and Return (on Windows), or by running the exit() command:

>>> ^D
now waiting for main window to close...

Headless Shell

You can also open a project directly with a headless shell:

$ crystal --shell --headless MyDownloadedWebsite.crystalproj
Crystal 2.1.0 (Python 3.14.2)
Type "help" for more information.
Variables "project" and "window" are available.
Use exit() or Ctrl-D (i.e. EOF) to exit.
>>>

No graphical dialogs or windows will be shown when Crystal is run as --headless.

The project variable can still be used to access the opened Project.

Shell Features

Colors, Multi-Line Editing, Copying and Pasting

Crystal's shell inherits most of the capabilities of PyREPL from Python 3.13+, including:

  • Colors
    • Prompts are colored.
    • Typed code has syntax highlighting.
    • Colors can be disabled using the standard NO_COLOR environment variable, or forced by using the standard FORCE_COLOR environment variable. A Python-specific environment variable is also available called PYTHON_COLORS.
  • Multi-line Editing
    • Editing multi-line blocks provides automatic indentation using four spaces, which is consistent with PEP 8 recommendations. When a line ending with a colon is encountered, the following line is automatically indented utilizing the indentation pattern that is inferred from the first line that contains indentation. Lines are indented with four spaces, and tabs are converted into spaces.
  • Copying and Pasting
    • In supported terminal emulators, bracketed pasting capability is discovered and used.
    • For terminal emulators that don’t support this mode, a dedicated paste mode is implemented to allow for easy insertion of multi-line code snippets without triggering immediate execution or indentation issues. Enter manual paste mode by hitting the F3 key. The prompt changes from >>> to (paste) where users can paste contents from their clipboard or manually type as desired. Once the content is ready, hitting F3 exits paste mode. Then, pressing Enter executes the block.

Users have the option to explicitly choose the old basic REPL by setting the environment variable PYTHON_BASIC_REPL to 1.

AI Agent Assistance

CRYSTAL_AI_AGENT=True

Crystal's shell has special support for being controlled by AI coding agents like GitHub Copilot Agent and Claude Code. To signal to Crystal that it is being controlled by an AI agent, configure/prompt your agent to set the CRYSTAL_AI_AGENT=True environment variable when executing the crystal command.

When an AI agent is detected:

  • Builtins for viewing/controlling Crystal's UI are advertised
  • UI changes are detected and printed automatically when caused by a command the agent ran
  • Warnings are printed for common agent mistakes
Example of running Crystal in agent mode, with the agent instructed to download the Home page of https://xkcd.daarchive.net/: 
$ CRYSTAL_AI_AGENT=True crystal --shell

Crystal 2.1.0 (Python 3.14.2)
Type "help" for more information.
Variables "project" and "window" are available.
AI agents:
- Use `T` to view/control the UI. Learn more with `help(T)`.
- Use `click(window)` to click a button.
- Use `await screenshot()` to capture the UI as an image.
- Run multi-line code with exec(): exec("for i in range(5):\n    print(i)")
- Use Python control flow (for/while loops, if statements, etc.) to batch operations.
Use exit() or Ctrl-D (i.e. EOF) to exit.

>>> T
🤖 T accessed but help(T) not read. Recommend reading help(T).
{
  (T[0].W := wx.Frame(Shown=False, Name='cr-menubar-frame')): {
    (T[0].MenuBar.M := wx.MenuBar()): {
      (T[0].MenuBar[0].M := wx.Menu(Title='&File')): {...},
      (T[0].MenuBar[1].M := wx.Menu(Title='Edit')): {...},
      (T[0].MenuBar[2].M := wx.Menu(Title='Help')): {...},
    },
  },
  (T[1].W := crystal.ui.dialog.BetterMessageDialog(Name='cr-open-or-create-project', Label='Select a Project')): {
    (T[1][0].W := wx.StaticText(Label='Create a new project or open an existing project?')): {},
    (T[1][1].W := wx.CheckBox(Name='cr-open-or-create-project__checkbox', Label='Open as &read only', Value=False)): {},
    (T[1][2].W := wx.Button(Id=wx.ID_NO, Label='&Open')): {},
    (T[1][3].W := wx.Button(Id=wx.ID_YES, Label='&New Project')): {},
  },
}

>>> help(T)
Help on WindowNavigator in module crystal.ui.nav:

{
  (T[0].W := wx.Frame(Name='cr-main-window', L...d=False, Id=wx.ID_NEW, Label='&New')): {},
  },
}
    The top navigator, pointing to the root of the wx.Window hierarchy.

    The repr() of the top navigator provides an snapshot of all visible
    windows in the wxPython application.

    The children of the top navigator correspond to all visible top-level windows.

    # Examples

    Look at the entire UI:
        >>> T

    Look at part of the UI:
        >>> T['cr-task-tree']  # lookup by Name (focused view!)
        >>> T(Id=wx.ID_YES)  # lookup by Id
        >>> T(Label='✏️')  # lookup by Label
        >>> T[0][0][1]  # lookup by index; prefer other less-brittle methods

    Click a button, checkbox, or radio button:
        >>> click(T(Id=wx.ID_YES).W)
        >>> click(T(Label='Open as &read only').W)
        >>> click(T['cr-preferences-dialog__no-proxy-radio'].W)

    Wait for UI changes:
        >>> await wait_for(lambda: len(T['cr-task-tree'].Tree.Children) == 0)
        >>> await wait_for(lambda: T['cr-download-button'].W.IsEnabled())

    Type in an input field:
        >>> T['cr-new-root-url-dialog__url-field'].W.Value = 'https://xkcd.com/'
        >>> T['cr-new-root-url-dialog__url-field'].W.Value
        'https://xkcd.com/'

    Manipulate a TreeItem:
        >>> T['cr-entity-tree'].Tree[0].I.Expand()
        >>> T['cr-entity-tree'].Tree[0][0].I.SelectItem()
        >>> T['cr-entity-tree'].Tree[0].I.Collapse()
        >>> help(TreeItem)  # for more methods and properties

    Obtain a query for a wx.Window/TreeItem for use in production code:
        >>> T['cr-entity-pane'].Q
        wx.FindWindowByName('cr-entity-pane')
        >>> T(Id=wx.ID_YES).Q
        wx.FindWindowById(wx.ID_YES)
        >>> T[0][0][0][0][1].Tree[0].Q
        TreeItem.GetRootItem(wx.FindWindowByName('cr-entity-tree')).Children[0]

>>> click(T(Label='&New Project'))
🤖 UI changed at: S := T
  S[0] - wx.Frame(Shown=False, Name='cr-menubar-frame')
  S[0][0] - More(Count=1)
  S[0] + wx.Frame(Name='cr-main-window', Label='Untitled Project')
  S[0][0] + wx.MenuBar()
  S[0][0][0] + wx.Menu(Title='File')
  S[0][0][1] + wx.Menu(Title='Edit')
  S[0][0][2] + wx.Menu(Title='Entity')
  S[0][0][3] + wx.Menu(Title='Help')
  S[0][1] + _
  S[0][1][0] + wx.SplitterWindow()
  S[0][1][0][0] + wx.Panel(Name='cr-entity-pane')
  S[0][1][0][0][0] + wx.StaticText(Label='Root URLs and Groups')
  S[0][1][0][0][1] + _
  S[0][1][0][0][1][0] + wx.StaticText(Label='Download your first page by defining a root URL for the page.')
  S[0][1][0][0][1][1] + wx.Button(Name='cr-empty-state-new-root-url-button', Label='New Root URL...')
  S[0][1][0][0][2] + wx.Button(Name='cr-add-url-button', Label='New Root URL...')
  S[0][1][0][0][3] + wx.Button(Name='cr-add-group-button', Label='New Group...')
  S[0][1][0][0][4] + wx.Button(Enabled=False, Name='cr-edit-button', Label='✏️ Edit...')
  S[0][1][0][0][5] + wx.Button(Enabled=False, Name='cr-forget-button', Label='✖️ Forget')
  S[0][1][0][0][6] + wx.Button(Enabled=False, Name='cr-update-members-button', Label='🔎 Update Members')
  S[0][1][0][0][7] + wx.Button(Enabled=False, Name='cr-download-button', Label='⬇ Download')
  S[0][1][0][0][8] + wx.Button(Enabled=False, Name='cr-view-button', Label='👀 View')
  S[0][1][0][1] + wx.Panel(Name='cr-task-pane')
  S[0][1][0][1][0] + wx.StaticText(Label='Tasks')
  S[0][1][0][1][1] + crystal.ui.tree._OrderedTreeCtrl(Name='cr-task-tree')
  S[0][1][0][1][1][0] + TreeItem(IsRoot=True, Visible=False, IsSelected=True)
  S[0][1][1] + wx.Panel(Name='cr-status-bar')
  S[0][1][1][0] + wx.Panel(Name='cr-branding-area')
  S[0][1][1][0][0] + wx.StaticBitmap(Name='cr-branding-area__icon')
  S[0][1][1][0][1] + _
  S[0][1][1][0][1][0] + _
  S[0][1][1][0][1][0][0] + wx.StaticBitmap(Name='cr-branding-area__program-name-bitmap')
  S[0][1][1][0][1][0][1] + wx.StaticText(Name='cr-branding-area__version', Label='v2.1.0')
  S[0][1][1][0][1][1] + _
  S[0][1][1][0][1][1][0] + wx.StaticText(Name='cr-branding-area__authors-1', Label='By David Foster and ')
  S[0][1][1][0][1][1][1] + crystal.ui.clickable_text.ClickableText(Name='cr-branding-area__authors-2', Label='contributors')
  S[0][1][1][1] + wx.Button(Name='cr-preferences-button', Label='⚙️ Settings...')
  S[0][1][1][2] + wx.StaticText(Name='cr-read-write-icon', Label='✏️')
  S[1] - crystal.ui.dialog.BetterMessageDialog(Name='cr-open-or-create-project', Label='Select a Project')
  S[1][0..3] - More(Count=4)

>>> click(T['cr-empty-state-new-root-url-button'])
🤖 UI changed at: S := T
  S[1] + wx.Dialog(IsModal=True, Name='cr-new-root-url-dialog', Label='New Root URL')
  S[1][0] + wx.StaticText(Label='New Root URL')
  S[1][1] + wx.StaticText(Label='URL:')
  S[1][2] + wx.TextCtrl(Name='cr-new-root-url-dialog__url-field', Value='')
  S[1][3] + wx.Button(Name='cr-new-root-url-dialog__url-copy-button', Label='📋')
  S[1][4] + wx.StaticText(Label='Name:')
  S[1][5] + wx.TextCtrl(Name='cr-new-root-url-dialog__name-field', Value='')
  S[1][6] + wx.StaticBox(Label='New URL Options')
  S[1][6][0] + wx.CheckBox(Name='cr-new-root-url-dialog__download-immediately-checkbox', Label='Download URL Immediately', Value=True)
  S[1][6][1] + wx.CheckBox(Name='cr-new-root-url-dialog__create-group-checkbox', Label='Create Group to Download Entire Site', Value=False)
  S[1][6][2] + wx.StaticText(Label='Only recommended for downloading simple sites')
  S[1][7] + wx.Button(Id=wx.ID_MORE, Label='Advanced Options')
  S[1][8] + wx.Button(Id=wx.ID_CANCEL, Label='&Cancel')
  S[1][9] + wx.Button(Enabled=False, Id=wx.ID_NEW, Label='&New')

>>> T['cr-new-root-url-dialog__url-field'].W.Value = 'https://xkcd.daarchive.net/'
🤖 UI changed at: S := T[1]
  S[2] ~ wx.TextCtrl(Name='cr-new-root-url-dialog__url-field', Value='{→https://xkcd.daarchive.net/}')
  S[9] ~ wx.Button({Enabled=False, →}Id=wx.ID_NEW, Label='&New')

>>> T['cr-new-root-url-dialog__name-field'].W.Value = 'Home'
🤖 UI changed at: S := T[1][5]
  S ~ wx.TextCtrl(Name='cr-new-root-url-dialog__name-field', Value='{→Home}')

>>> click(T(Id=wx.ID_NEW))
🤖 UI changed at: S := T
  S[0][1][0][0][1] - _
  S[0][1][0][0][1][0..1] - More(Count=2)
  S[0][1][0][0][1] + crystal.ui.tree._OrderedTreeCtrl(Name='cr-entity-tree')
  S[0][1][0][0][1][0] + TreeItem(IsRoot=True, Visible=False)
  S[0][1][0][0][1][0][0] + TreeItem(👁='▶︎ 📁 / - Home', IsSelected=True, IconTooltip='Undownloaded root URL')
  S[0][1][0][0][4] ~ wx.Button({Enabled=False, →}Name='cr-edit-button', Label='✏️ Edit...')
  S[0][1][0][0][5] ~ wx.Button({Enabled=False, →}Name='cr-forget-button', Label='✖️ Forget')
  S[0][1][0][0][7] ~ wx.Button({Enabled=False, →}Name='cr-download-button', Label='⬇ Download')
  S[0][1][0][0][8] ~ wx.Button({Enabled=False, →}Name='cr-view-button', Label='👀 View')
  S[0][1][0][0][9] + crystal.ui.callout.Callout(Name='cr-view-button-callout')
  S[0][1][0][0][9][0] + wx.StaticText(Label='View your first downloaded page in a\nbrowser by pressing "View"')
  S[0][1][0][0][9][1] + wx.Button(Name='cr-view-button-callout__close-button', Label='✕')
  S[0][1][0][0][9][2] + wx.CheckBox(Name='cr-view-button-callout__dont-show-again', Label="Don't show this message again", Value=False)
  S[0][1][0][1][1][0][0] + TreeItem(👁='▶︎ 📁 Downloading: https://xkcd.daarchive.net/ -- Waiting for response...')
  S[1] - wx.Dialog(IsModal=True, Name='cr-new-root-url-dialog', Label='New Root URL')
  S[1][0..9] - More(Count=10)

CRYSTAL_MCP_SHELL_SERVER=True

There is additional special support for using mako10k's mcp-shell-server and its terminal_operate tool. Configure the MCP server to additionally set CRYSTAL_MCP_SHELL_SERVER=True to enable additional behaviors and warnings specific to this tool's limitations. In VS Code's mcp.json, use this configuration for mcp-shell-server:

"shell-server": {
    "type": "stdio",
    "command": "mcp-shell-server",
    "args": [],
    "env": {
        "CRYSTAL_AI_AGENT": "True",
        "CRYSTAL_MCP_SHELL_SERVER": "True"
    }
},

$PYTHONSTARTUP

If you start Crystal with the PYTHONSTARTUP environment variable set to the filepath of a Python .py file, that file will be run in the shell when Crystal starts. That can be useful for automatically importing certain modules or defining certain variables.

For example, if the file startup.py exists in the current directory that looks like:

# startup.py
from crystal.model import *
EXCLUDED_URLS = ['a', 'b', 'c']

And you start Crystal using:

$ PYTHONSTARTUP=startup.py crystal --shell

You can access items imported from crystal.model and the EXCLUDED_URLS variable defined by startup.py:

>>> Resource
<class 'crystal.model.Resource'>
>>> EXCLUDED_URLS
['a', 'b', 'c']
>>>

Async Support

Crystal's shell supports top-level await expressions that use Crystal's internal awaitable testing utilities in the crystal.tests.util package. Example:

from crystal.tests.util.wait import wait_for

await wait_for(lambda: len(T['cr-task-tree'].Tree.Children) == 0)

Awaitables that rely on the asyncio event loop are NOT supported.

Shell API

There is no officially supported API on the shell at the moment. If you want to write automated scripts that run in the shell, avoid using methods/attributes whose name starts with an underscore (_) because those methods/attributes are private or unstable.

Add a custom footer
Add a custom sidebar

Clone this wiki locally