v2.0.2

v2.0.0 (September 26, 2025)

Crystal 2.0 is a huge release with many new features and all-new tutorials!
It's now possible to download a website entirely through a web browser,
by navigating through already-downloaded pages and incrementally adding URLs
and groups for any links that weren't already downloaded. Inline help messages
guide first-time users through the process of downloading their first website.
Projects are now created as untitled by default, no longer requiring you to
immediately save & title them upon creation. Many bugs related to browsing
read-only projects are now fixed. Documentation is great. UI is stable.
New contributors are actively welcome to help enhance Crystal going forward!

• First-time-run experience improvements

  • After creating an empty project a call-to-action appears to create
    the first root URL.
  • After creating the first root URL a call-to-action appears to view
    the downloaded root URL in a web browser.
  • There are new "Root URLs and Groups" and "Tasks" headings in the
    main window to clearly identify what is in each part of the window.
  • The "View" button no longer appears squished on Linux.
  • A project can be opened by double-clicking it in the macOS Finder
    after Crystal is already running.

• Workflow improvements

  • A website can be effectively downloaded entirely by browsing a partially
    downloaded site via a web browser, rather than through Crystal's traditional UI:
    • Missing pages can be immediately downloaded with 1 click when browsing
      a partialy downloaded website.
    • Groups can be immediately added with 2 clicks when browsing
      a partialy downloaded website.
  • Untitled projects are automatically reopened after an unexpected quit.
  • Untitled projects are now saved to a permanent hidden directory instead of
    the system temporary directory, preventing data loss after unexpected
    logout, restart, or shutdown.
  • The Download button is now closer to the View button,
    to make them easier to use together.
  • A copy button is added beside the URL and URL Pattern fields when editing
    root URLs and groups.

• Branding changes

  • Crystal is now known as "Crystal: A Website Archiver" in long form.
  • Crystal's logo and contributor information is now displayed in the
    main window, alongside its version number.
  • Browser error pages are now appropriately styled and include
    Crystal's branding elements.
  • Served pages include a footer banner with Crystal's branding elements.
  • Licensing terms have been added: PolyForm Noncommercial 1.0.0

• Serving improvements

  • When serving from a remote --host, projects are opened in --readonly
    mode by default so that remote users cannot modify a served project.
    Remote modifications can be explicitly enabled with --no-readonly.
  • Caching is disabled for revisions served from the root directory
    to avoid seeing stale HTML pages when browsing multiple Crystal projects
    around the same time.
  • Served revisions always have a Date header, to conform with RFC 7231 §7.1.1.2.
  • Experimental: Resource group pattern prediction. When enabled with
    CRYSTAL_PREDICT_GROUPS=1, Crystal will analyze the current URL and
    referrer when visiting non-archived pages to predict suitable URL patterns
    and sources for creating resource groups. This is intended to support
    future dynamic browsing features. Predictions are printed to the terminal
    with special [PREDICT] formatting.

• Read-only project fixes

  • Fix inability to browse the entity tree of some readonly projects.
    Impact was limited primarily to very old readonly projects.
  • Fix inability to open preferences/settings dialog of readonly projects
    created with Crystal <=1.2.0.
  • Project-specific preferences in a read-only project now appear as disabled.
  • Menuitems related to changing the default URL prefix in a read-only project
    now appear as disabled.

• Parsing improvements

  • Recognizes pages containing <frameset> tags more accurately.
    In particular a page using a DOCTYPE DTD that mentions "frameset"
    is no longer assumed to necessarily actually contain a <frameset>.

• Downloading improvements

  • While long downloads are running the system will be kept awake
    ("caffeinated") on Linux. macOS and Windows support for caffeination
    already existed.

• Documentation improvements

  • Contributing guidelines added as CONTRIBUTING.md.
  • License FAQ drafted.

• API changes

  • Project.get_root_resource, Project.get_resource_group, and
    Project.get_resource now support kwargs for easily filtering by
    name, URL pattern, and ID.
  • Resource.bulk_get_or_create is added as a replacement for
    Resource.bulk_create. The former function returns resources that were
    looked up in addition to resources that were created, which is usually the
    behavior that a caller wants. The latter function is deprecated.

• Minor fixes

  • Most dialogs are now shown as window-modal rather than modeless to
    prevent certain types of concurrent edits to a project that Crystal
    isn't designed to handle.
  • Dark mode fixes:
    • Task tree appears with a dark background on macOS when in dark mode
      rather than a pure white background.
    • Pure black icons in the task tree and entity tree appear as off-white
      when in dark mode
  • Adhere to OS-native conventions more closely:
    • Save changes dialog now uses OS-native button labels ("Save" and "Don't Save")
      instead of generic "Yes" and "No" buttons, following platform conventions.
    • The "Preferences..." / "Settings..." button in the main window now uses
      the correct title based on OS conventions. And the dialog which appears
      after pressing the button uses a consistent title.
  • Fixed the "Dismiss" action on a crashed task to not itself crash!
  • Altered the prefilled URL prefix on the server Welcome page to
    be https:// rather than http://, which is more appropriate for most sites.

• Testing improvements

  • If the default server port is in use while tests are running, a warning
    message will be printed suggesting to look for other running copies
    of Crystal.

v2.0.1 (October 14, 2025)

• Documentation improvements

  • Add video tutorials.
  • Fix anchor formatting in README on PyPI.

• Downloading improvements

  • Crawl speed improvements
    • For URLs downloaded interactively - by browsing to an undownloaded URL or
      expanding a entity tree node for an undownloaded URL - download at
      high priority, before other tasks. Mark such tasks with a ⚡️ icon.
    • Adjust default (non-interactive) crawl speed to be friendlier to website hosts.

• Serving improvements

  • Wait for a page to fully download (including embedded resources)
    before serving it.
  • Not in Archive Page fixes:
    • Fix race condition where the Download button might not actually wait
      for URL to download before trying to navigate to it.
    • Fix cache configuration so that refreshing the page will never
      display a stale cached Not in Archive page.

• Minor fixes

  • Quit and OS logout improvements
    • When OS logout occurs, Crystal will actually finish quitting normally.
    • When Quit is selected from Crystal's Dock icon on macOS,
      Crystal will actually quit.

v2.0.2 (October 19, 2025)

• Workflow improvements

  • When reopening a project, restore download tasks correctly.

• Testing improvements

  • Increased determinism and speed of tests that perform downloads.

• Development improvements

  • Manually bring app to front on macOS when run from source,
    to workaround a wxPython 4.2.3 bug.
  • Show correct Dock icon on macOS when run from source.
  • Code coverage statistics can now be gathered using the coverage tool.

• Minor fixes

  • Fix rare IndexError that could occur when starting new downloads at the
    same time as other downloads are running.