-
Notifications
You must be signed in to change notification settings - Fork 8
Roadmap
David Foster edited this page Mar 17, 2024
·
157 revisions
For smaller items, please see the open issue list.
Easy to install for early adopter developers, on macOS and Windows: ✅ (with binary and pipx)
Stabilize open experience from Finder/Explorer/Nautilus: ✅ (with icons and double-click behavior)
Consider additional issues with the topic-firstrun label:
- Download: Don't crash when downloading an empty group (#170)
Harder to make mistakes. Easier to correct mistakes.
- Add URL: Accept common types of loosely formatted URLs like "xkcd.com" (which rounds to "https://xkcd.com/") (#172)
- Add URL: Default to "https://" rather than "http://". Ideally as a placeholder rather than as regular text. (#172)
- Add URL: Encourage use of Default URL Prefix when adding root URL (#46)
- Avoids various serving errors on more modern websites
- Edit URL/Group: Allow edit of Root URLs and Groups after creation (#98)
- Allows user to correct a mistake when defining an URL or Group
- Edit URL: Right-Click non-URL: Still shows a contextual menu (#94)
- Add URL/Group: Make titles for Root URLs and Groups be optional (#37)
- Add URL/Group: (Bonus) Wider and adjustable "Add Root URL" and "Add Group" dialogs (#38)
Hangnails:
- Add ⚓ and 📁 icons everywhere that URLs and Groups mentioned in UI (#138)
Entice to install:
- [▶︎] README: Add intro/sizzle video showing how to download various sites (#140)
- Alter quickstart to use a mirrored copy of xkcd as an example, rather than the real xkcd.
- Does reduce bandwidth load on real xkcd.
- Does prevent instructions from breaking.
- But does require ability to easily export sites in format amenable to hosting on GitHub Pages.
- Alter quickstart to use a mirrored copy of xkcd as an example, rather than the real xkcd.
- README: Provide more info about how Crystal's downloaded sites are in a "file format suitable for archival". Suggest expand on the existing bullet with that text in the current README. (NEEDS ISSUE)
- Document .crystalproj package format, and anything else that a digital archivist would care about (NEEDS ISSUE)
Legal:
- Add license that allows others to distribute widely but NOT to sell commercially. Source-Available Shareware. (#16)
- Provide way for early adopters to contribute to development financially (#147)
Easy to install for early adopters, on Linux:
Follow-ons to recently introduced features:
- Do not Download: Persist this option after reopening project (#191)
High-value features (Choose 1 or 2):
- Sequential groups (#110)
- Already have initial UI for this, behind the
NewGroupDialog._SHOW_SEQUENTIAL_OPTIONfeature flag.
- Already have initial UI for this, behind the
- Ability to search for URLs that did not download at all or did not download successfully (with an error) (#84)
- Support resume of (download) tasks when project reopened (#115)
Branding during README:
- Branding: Consider changing name from "Crystal Web Archiver" to "Crystal" (#158)
- Branding: Change PyPI package name from "crystal-web" to "crystal" (#159)
Easy to install for end users (Part 1):
- Binary install steps work on all platforms:
- —
- Easy to install/run on Mac, from disk image:
- Mac Installation: Streamline process to install Crystal to Applications folder (#156)
- Easy to install/run on Windows, from installer:
- Windows Installer: Offer to open Crystal after install. Offer to add desktop shortcut. (#155)
Easy to install for end users (Part 2):
- Easy to run on Mac, from Mac App Store:
- Build/submit .app for Mac App Store (with M1 support too) (#42)
- 💰 Requires an Apple Developer ID certificate ($99/year)
- Build/submit .app for Mac App Store (with M1 support too) (#42)
- Easy to run on Windows, from Windows Store:
- [#] Investigate localization steps to make Crystal feel more like a native Windows app, if any...
- Submit .exe to Windows App Store (#181)
- 💰 Probably requires some kind of paid certificate from Microsoft
- Easy to run on Linux, from local package managers:
Branding during first run:
- Better start dialog/window (#113)
- Show Crystal logo and icon in first dialog
- Main Window: Improve Crystal branding in lower-left corner (#99)
- Icons:
- Main Window: Add icon for Crystal that appears in top-left corner, for Windows (#151)
- Main Window: Add dock icon/title and activity icon/title, for Linux (#152)
- Main Window: Only show file extension if shown in Finder, for macOS (#160)
- [#] Main Window: Use same dock icon when run from source as when run from .app or .exe, for macOS and Windows (#153)
Easy to create first project:
- Support untitled projects (#102)
- Eliminate requirement to name & save a project early
- Launch: Long delay opening the Save/Load dialog (#148)
Easy to create first URL:
- One/both of:
- Better error messages for common error types (#180)
Easy to create first group:
- Add Group: Warn when trying to create group using pattern with no wildcards (#77)
Easy to serve first URL:
- View: Long delay opening web browser (#149)
- Log Drawer: Animate open/close (#134)
- Strengthen mental association between pressing View and the Log Drawer that appears
Easy/possible to export:
- Support export of project to filesystem (NEEDS ISSUE)
- A user asked about this in #119
- Support export of project to static file server (like GitHub Pages) (#41)
Easy to quit:
- Respond to Quit event from OS correctly (#150)
Quality:
- Automated test coverage of core workflows:
- Download Site Requiring Login:
- test_can_download_and_serve_a_site_requiring_cookie_authentication (related to #68)
- ... (probably more core workflows missing test coverage)
- Download Site Requiring Login:
Any of the following items individually are major enough to warrant a release on their own (releaseworthy):
- Support dynamic download of site entirely through browser (#69)
- Support rewriting URLs on the frontend in JavaScript, which is necessary to archive more-modern sites that load many resources using JavaScript that cannot be fully statically analyzed to extract URLs in advance. 🧠 (#8)
...as an advanced contribution-required feature
- Support export of downloaded site to plain files that can be deployed to a static file server, either to a custom domain (best) or to a subdirectory within an existing domain (worse, because some URLs may not be as straightforward to rewrite). (#41)
- Does round out final piece of end-to-end preservation lifecycle: capture, retain, & share
- Does enable removing real xkcd from quickstart instructions.
Separate from the release of Crystal itself:
- Do use the export feature to export & host various notable sites I've archived that have fallen off the internet:
- TMFFA -> tmffa.daarchive.net (#60)
- PRFI -> rakhal.daarchive.net (#61)
- (??) Apple Support Area -> ???.daarchive.net
- ⚖️ Has special legal considerations: Would Apple disapprove of distributing the old files from this area?
- –––
- xkcd -> xkcd.daarchive.net
- 🎗 Needs special robots.txt to prevent indexing, because original site still online
- (??) OtakuWorld + The Big KiSS Page + Jenniverse -> (multiple!)
- 🎗 Needs special robots.txt to prevent indexing, because original site still online
- –––
- -> *.otakuworld.daarchive.net
- -> jenniverse.daarchive.net
- -> *.unicornjelly.daarchive.net
- -> pasteldefender.daarchive.net
- -> transsexual.daarchive.net
- -> kamishibai.daarchive.net
...as an advanced contribution-required feature
- Distributed web crawling: Separate out crawling logic such that a centralized "orchestrator" can direct a remote fleet of "crawlers" to quickly capture a large domain that is scheduled to fall off the public internet soon (ex: Geocities, DeviantArt Portfolios, etc).