All notable changes to this project will be documented in this file.
- Added BGP community filtering support to
monocle parseandmonocle search- New CLI option:
-C, --community(with--communitiesalias) - Supports repeated flags and comma-separated values
- Supports standard communities (
A:B) and large communities (A:B:C) - Supports wildcard matching per position with
*(for example*:100,1299:*,57866:*:*) - Uses strict positional matching with exact colon-count semantics
A:Bpatterns match only standard communitiesA:B:Cpatterns match only large communities- Example:
1299:*does not match1403:1299
- New CLI option:
- Added CLI compatibility aliases for time filters in
monocle parseandmonocle search--ts-startis now accepted as an alias for--start-ts--ts-endis now accepted as an alias for--end-ts
- Fixed panic when truncating names containing non-ASCII UTF-8 characters
- Used character-based truncation instead of byte-based slicing
- Affects RPKI ASPA display and inspect lens name truncation
- Changed config, data, and cache paths to follow XDG base directory specification
- Config file default is now
$XDG_CONFIG_HOME/monocle/monocle.toml(fallback:~/.config/monocle/monocle.toml) - Data directory default is now
$XDG_DATA_HOME/monocle(fallback:~/.local/share/monocle) - Cache directory default is now
$XDG_CACHE_HOME/monocle(fallback:~/.cache/monocle) - Existing SQLite data under
~/.monocleis no longer used by default and will be rebuilt in the new data location- Legacy config migration: when the new config directory is empty, monocle copies
~/.monocle/monocle.tomlto the new config path - Old database file will not be copied over. Once the updated monocle has been executed at least once, old
~/.monoclecan be safely deleted
- Legacy config migration: when the new config directory is empty, monocle copies
- Config file default is now
- Added
--use-cacheflag tomonocle searchto use the default XDG cache path ($XDG_CACHE_HOME/monocle)- Value set by
--cache-diroverrides--use-cachewhen both are provided
- Value set by
- Updated README command help examples to match current CLI help output from the release binary
- Moved
utilsmodule fromlens::utilsto crate-levelutils- Eliminates misleading module structure since utilities are used throughout the codebase
- Updated all imports from
crate::lens::utilsandmonocle::lens::utilstocrate::utilsandmonocle::utils
- Switched directory resolution library from
dirstoetcetera
-
Simplified feature flags: Replaced 6-tier feature system with 3 clear features
- Old:
database,lens-core,lens-bgpkit,lens-full,display,cli - New:
lib,server,cli - Quick guide:
- Need CLI binary? Use
cli(includes everything) - Need WebSocket server without CLI? Use
server(includes lib) - Need only library/data access? Use
lib(database + all lenses + display)
- Need CLI binary? Use
- Display (tabled) now always included with
libfeature
- Old:
-
CLI flag renamed:
--no-refreshrenamed to--no-updatefor consistency with "update" terminology- Old:
monocle --no-refresh <command> - New:
monocle --no-update <command>
- Old:
-
Config subcommands renamed: Removed
db-prefix from config subcommands for cleaner syntaxmonocle config db-refreshβmonocle config updatemonocle config db-backupβmonocle config backupmonocle config db-sourcesβmonocle config sources
-
Configurable TTL for all data sources: All data sources now have configurable cache TTL with 7-day default
- Added
asinfo_cache_ttl_secsconfig option (default: 7 days) - Added
as2rel_cache_ttl_secsconfig option (default: 7 days) - Changed
rpki_cache_ttl_secsdefault from 1 hour to 7 days - Changed
pfx2as_cache_ttl_secsdefault from 24 hours to 7 days - Configure via
~/.monocle/monocle.tomlor environment variables (MONOCLE_ASINFO_CACHE_TTL_SECS, etc.)
- Added
-
Standardized database refresh API: Consistent interface for all data sources
- New
RefreshResultstruct withrecords_loaded,source,timestamp,details - Renamed methods for consistency:
bootstrap_asinfo()βrefresh_asinfo()(with deprecated alias)update_as2rel()βrefresh_as2rel()(with deprecated alias)
- Added missing methods:
refresh_asinfo_from(path)- Load ASInfo from custom pathrefresh_rpki()- Load RPKI data from recordsrefresh_pfx2as()- Load Pfx2as data from records
- All repositories now use consistent
needs_*_refresh(ttl)pattern - Removed hardcoded TTL methods (
should_update()from AS2Rel) - All repositories have both URL and path loading methods
- New
-
Reorganized examples: One example per lens with
_lenssuffix- Flat directory structure:
examples/time_lens.rs,examples/rpki_lens.rs, etc. - Added new examples for IpLens, Pfx2asLens, As2relLens
- Removed verbose multi-example files
- All examples use
libfeature exclusively
- Flat directory structure:
-
ParseFilters: Changed filter field types to support multiple values with OR logic
origin_asn:Option<u32>βVec<String>prefix:Option<String>βVec<String>peer_asn:Option<u32>βVec<String>- Empty
Vecis equivalent to no filter (previousNone) - Values can be prefixed with
!for negation (exclusion) - Library users will need to update code:
Some(13335)βvec!["13335".to_string()]
-
RTR protocol support: Added support for fetching ROAs via RTR (RPKI-to-Router) protocol
- Configure RTR endpoint in
~/.monocle/monocle.toml:rpki_rtr_host = "rtr.rpki.cloudflare.com" rpki_rtr_port = 8282 rpki_rtr_timeout_secs = 10 rpki_rtr_no_fallback = false
- Or use environment variables:
MONOCLE_RPKI_RTR_HOST,MONOCLE_RPKI_RTR_PORT,MONOCLE_RPKI_RTR_TIMEOUT_SECS,MONOCLE_RPKI_RTR_NO_FALLBACK - Or use CLI flag for one-time override:
monocle config update --rpki --rtr-endpoint rtr.rpki.cloudflare.com:8282 - ROAs are fetched via RTR, ASPAs always from Cloudflare (RTR v1 per RFC 8210 doesn't support ASPA)
- Automatic fallback to Cloudflare if RTR connection fails, with warning message (set
rpki_rtr_no_fallback = trueto disable fallback and error out instead) - Connection timeout defaults to 10 seconds
- Supports RTR protocol version negotiation (v1 with v0 fallback)
- Configure RTR endpoint in
-
--cache-dir: Added local caching support to thesearchcommand- Download MRT files to a local directory before parsing
- Files are cached as
{cache-dir}/{collector}/{path}(e.g.,cache/rrc00/2024.01/updates.20240101.0000.gz) - Cached files are reused on subsequent runs, avoiding redundant downloads
- Uses
.partialextension during downloads to handle interrupted transfers - Cache directory access is validated upfront before processing begins
- Broker query caching: When
--cache-diris specified, broker API query results are cached in SQLite- Cache stored at
{cache-dir}/broker-cache.sqlite3 - Only queries with end time >2 hours in the past are cached (recent data may still change)
- Subsequent identical queries use cached results, enabling offline operation
- Tested: run search once, disable network, run same search again - results returned from cache
- Cache stored at
- Example:
monocle search -t 2024-01-01 -d 1h --cache-dir /tmp/mrt-cache
-
Multi-value filters:
parseandsearchcommands now support filtering by multiple values with OR logic- Example:
-o 13335,15169,8075matches elements from ANY of the specified origin ASNs - Example:
-p 1.1.1.0/24,8.8.8.0/24matches ANY of the specified prefixes - Example:
-J 174,2914matches elements from ANY of the specified peer ASNs
- Example:
-
Negative filters: Support for exclusion filters using
!prefix- Example:
-o '!13335'excludes elements from AS13335 - Example:
-o '!13335,!15169'excludes elements from AS13335 AND AS15169 - Note: Cannot mix positive and negative values in the same filter
- Example:
-
Added validation for ASN format, prefix CIDR notation, and negation consistency
-
--time-format: Added timestamp output format option toparseandsearchcommands--time-format unix(default): Output timestamps as Unix epoch (integer/float)--time-format rfc3339: Output timestamps in ISO 8601/RFC3339 format (e.g.,2023-10-11T17:00:00+00:00)- Applies to non-JSON output formats (table, psv, markdown)
- JSON output always uses numeric Unix timestamps for backward compatibility
- Example:
monocle parse file.mrt --time-format rfc3339 - Example:
monocle search -t 2024-01-01 -d 1h -p 1.1.1.0/24 --time-format rfc3339
-
Added
--fields(-f) option toparseandsearchcommands for selecting output fields (#99, #101)- Choose which columns to display with comma-separated field names
- Available fields:
type,timestamp,peer_ip,peer_asn,prefix,as_path,origin,next_hop,local_pref,med,communities,atomic,aggr_asn,aggr_ip,collector - Parse command defaults exclude
collectorfield (not applicable) - Search command defaults include
collectorfield - Example:
monocle search -t 2024-01-01 -d 1h -f prefix,as_path,collector
-
Added proper table formatting with borders using
tabledcrate for--format table(#99, #101)- Table output now uses rounded borders instead of tab-separated values
- Markdown format includes proper header row with separator
-
Added
--order-byand--orderparameters toparseandsearchcommands (#98)- Sort output by:
timestamp,prefix,peer_ip,peer_asn,as_path, ornext_hop - Direction:
asc(ascending, default) ordesc(descending) - When ordering is specified, output is buffered and sorted before display
- Example:
monocle parse file.mrt --order-by timestamp --order asc - Example:
monocle search -t 2024-01-01 -d 1h -p 1.1.1.0/24 --order-by timestamp --order desc
- Sort output by:
-
monocle config sources: Shows staleness status based on TTL for all data sources- "Stale" column shows whether each source needs updating based on its configured TTL
- Configuration section shows current TTL values for all sources
- Avoid creating a new SQLite database when
monocle config sourcesinspects staleness
- Data refresh logging: CLI now shows specific reason for data refresh ("data is empty" vs "data is outdated") instead of generic "empty or outdated" message
- AS name display: ASN names are now displayed using a preferred source hierarchy:
- Priority order: PeeringDB
akaβ PeeringDBname_longβ PeeringDBnameβ AS2Orgorg_nameβ AS2Orgnameβ Corename - This provides more recognizable, commonly-used AS names from PeeringDB when available
- Affects all commands that display AS names:
inspect,as2rel,rpki,pfx2as
- Priority order: PeeringDB
- Feature gate cleanup: Simplified feature gating for the
databasemodule- The entire
databasemodule is now gated atlib.rslevel with#[cfg(feature = "lib")] - Removed redundant feature gates from internal submodules
- Added detailed feature documentation to
ARCHITECTURE.mdwith use case scenarios
- The entire
-
Added new
monocle pfx2ascommand for prefix-to-ASN mapping lookups- Search by prefix: Query prefixes to find their origin ASNs
- Example:
monocle pfx2as 1.1.1.0/24
- Example:
- Search by ASN: Query an ASN to find all its announced prefixes
- Example:
monocle pfx2as 13335ormonocle pfx2as AS13335
- Example:
- RPKI validation: Shows RPKI validation status (valid/invalid/not_found) for each prefix-ASN pair
--show-name: Display AS organization name for each origin ASN--include-sub: Include sub-prefixes (more specific) in results- Example:
monocle pfx2as 8.0.0.0/8 --include-sub --limit 20
- Example:
--include-super: Include super-prefixes (less specific) in results- Example:
monocle pfx2as 1.1.1.0/24 --include-super
- Example:
--limit: Limit the number of results- Supports all standard output formats (
--format table/json/psv/etc.)
- Search by prefix: Query prefixes to find their origin ASNs
-
Enhanced
monocle as2relcommand with advanced filtering and multi-ASN support--min-visibility <PERCENT>: Filter results by minimum visibility percentage (0-100)- Available for all as2rel queries
- Filters out relationships seen by fewer than the specified percentage of peers
--single-homed: Find ASNs that are single-homed to the queried ASN- Shows only ASNs where the queried ASN is their ONLY upstream provider
- Useful for identifying customers with no redundancy
- Example:
monocle as2rel 2914 --single-homed
--is-upstream: Filter to show only downstream customers of the queried ASN- Shows relationships where the queried ASN is the upstream (provider)
--is-downstream: Filter to show only upstream providers of the queried ASN- Shows relationships where the queried ASN is a downstream (customer)
--is-peer: Filter to show only peer relationships (settlement-free interconnection)- Multi-ASN support: Query relationships among multiple ASNs at once
- When more than two ASNs are provided, shows all pair combinations
- Results sorted by asn1, with asn1 < asn2 for each pair
- Example:
monocle as2rel 174 2914 3356shows all three pair relationships
-
Added global
--no-refreshflag to disable automatic data refresh- Use
monocle --no-refresh <command>to skip all automatic data loading/refresh - Useful when you want to use existing cached data only
- Shows warnings when data is missing or stale instead of auto-refreshing
- Use
-
Added Docker support with multi-stage build
Dockerfilewith two-stage build process for minimal image size (~176MB final image)- Uses Rust 1.92 and Debian trixie-slim as runtime base
docker-compose.ymlfor easy container orchestration.dockerignoreto optimize build context- Runs as non-root user for security
- Persistent data volume at
/data - Default server mode with port 8080 exposed
- Fixed "database is locked" error in
monocle config db-refreshcommand (Issue #90)- The
do_refreshfunction was opening redundant database connections for ASInfo and AS2Rel data sources - Now correctly uses the already-passed database connection parameter
- The
-
Added visual
...row indicator in tables when results are truncated- Search results table now shows a
...row when more matches exist - RPKI ROA tables show truncation indicator
- Announced prefixes table shows truncation indicator
- Connectivity section (upstreams/peers/downstreams) tables show truncation indicator
- Makes it much more visible that additional results are available
- Search results table now shows a
-
Added
[monocle]prefix to all auto-refresh log messages- Makes it easier to distinguish monocle's internal logging from main output
- Especially useful when refresh operations run automatically during commands
-
RPKI ASPA command now ensures ASInfo data is available for AS name enrichment
- Automatically loads ASInfo data before showing ASPA output
- AS names and countries are displayed in ASPA results
-
Added comprehensive tests for database initialization with mock data
- Tests for all repositories being accessible after initialization
- Tests for schema version verification
- Tests for RPKI and Pfx2as mock data storage/retrieval
- Fixed cross-compilation issue on Linux platforms caused by OpenSSL dependency
- Updated
bgpkit-commonsto v0.10.1 which usesrustlsinstead ofnative-tls - All TLS operations now use
rustls, eliminating the need for OpenSSL development packages
- Updated
This is a major release with significant architectural changes, new commands, and breaking changes.
- Command Removals & Renames:
- Removed
brokercommand (usesearch --broker-filesinstead). - Removed
radarcommand (access Cloudflare Radar directly via their API). - Removed
rpki listandrpki summarycommands (userpki roasinstead). - Renamed
rpki checktorpki validate. - Renamed
whoistoinspect(unified AS/prefix lookup command).
- Removed
- Library API:
- All public functions are now accessed through lens structs (e.g.,
InspectLens,RpkiLens).
- All public functions are now accessed through lens structs (e.g.,
- Output:
- Default output format changed from markdown to table (pretty borders).
monocle inspect: Unified AS/prefix information lookup.- Replaces
whoisandpfx2as. - Auto-detects query type (ASN, prefix, IP address, or name).
- Combines data from ASInfo, AS2Rel, RPKI, and Pfx2as.
- Replaces
monocle server: WebSocket API Server.- JSON-RPC style protocol with streaming support.
- Endpoints for all major monocle operations.
monocle config: Consolidated configuration and database management.- Manage data sources, refresh data, and backup database.
monocle as2rel: AS Relationship lookup.- Query relationships, peers, and upstreams.
- Enrichment:
- Enriched customer/provider names and countries via SQL JOINs.
- Unified provider structure in JSON output (
providersarray with{asn, name}objects).
- Unified Output Format: Global
--formatoption for all commands (table,markdown,json,json-pretty,json-line,psv). - SQLite Integration:
- ASInfo: Unified AS information stored in SQLite (replaces as2org).
- Pfx2as: Prefix-to-ASN mappings cached in SQLite for fast range lookups.
- RPKI: ROAs and ASPAs cached in SQLite.
- Progress Tracking: Library support for callback-based progress reporting in
ParseLensandSearchLens. - Feature Flags: Reorganized into tiers (
database,lens-core,lens-bgpkit,lens-full,display,cli).
monocle inspect:- Progress messages during data loading.
- Improved output formatting with section headers.
- Performance optimization (lazy loading of data sources).
- General:
- Name truncation: Long names in tables are truncated to 20 chars (disable with
--show-full-name). - Database performance: Optimized batch insert operations.
- Broken pipe handling: Graceful exit when piping output (e.g., to
head).
- Name truncation: Long names in tables are truncated to 20 chars (disable with
- Handle SIGPIPE gracefully to prevent panics when piping output.
- Added WebSocket server documentation.
- Updated all documentation references and examples.
- Lens-based architecture: Centralized logic in
src/lens/. - Refactoring: Improved CLI command organization.
- Examples: Added comprehensive examples for all feature tiers.
- Added
bgpkit-commons(asinfo, rpki, countries). - Added server dependencies (
axum,tokio, etc.). - Added
libc(used for SIGPIPE handling on Unix systems). - Removed
rpkicrate.
- update dependencies
oneio-> v0.20.0bgpkit-parser-> v0.12.0bgpkit-broker-> v0.9.1
- Fix an issue where monocle fails to locate the latest CAIDA as2org dataset file
- Added retry mechanism for failed search operations with exponential backoff
- Implemented real-time success/failure progress tracking during search
- Added paginated search processing for large time ranges to handle memory efficiently
- Database bootstrap performance improvements
- Added proper transaction management for bulk inserts
- Replaced string-based SQL with prepared statements
- Added database indexes for common query patterns
- Enabled SQLite performance optimizations (WAL mode, cache tuning)
- Impact: BGP data insertion ~10x faster, as2org bootstrap ~100x faster (3+ minutes β 1-2 seconds)
- Fixed network error handling in multi-file processing to prevent thread panics
- Replaced unwrap/expect calls with proper error handling
- Added clippy lints to deny unsafe unwrap_used and expect_used patterns
- Updated CI workflow to include formatting and clippy checks
- Enhanced database operations with proper Result types
- Improved RPKI validator error handling
- added
monocle pfx2assubcommand to allow bulk prefix-to-asn mapping using BGPKIT dataset- it takes a list of prefixes or prefix files (one prefix per line)
Example:
monocle pfx2as 1.1.1.0/24 8.8.8.0/24 --json
[
{
"origin": 13335,
"prefix": "1.1.1.0/24"
},
{
"origin": 15169,
"prefix": "8.8.8.0/24"
}
]- update dependencies
- note that we upgraded to
bgpkit-parserv0.11 and community values are now without prefixes such aslg:ecv6
- note that we upgraded to
- support searching data from RIB dumps by specifying
--dump-typeargument--dump-type updates: search updates files only--dump-type rib: search RIB files only--dump-type rib-updates: search RIB dump and updates
- improved internal handling of filters and time string parsing
- improved documentation
- add back
Cargo.lockfile to reproducible builds
Add a new monocle ip command to retrieve information for the current IP of the machine or any specified IP address,
including location, network (ASN, network name) and the covering IP prefix of the given IP address.
The command triggers an API call to BGPKIT API, and it retrieves the information based on the incoming requester IP address with additional BGP information for the enclosing IP prefixes.
β ~ monocle ip
+----------+--------------------------+
| ip | 104.48.0.0 |
+----------+--------------------------+
| location | US |
+----------+---------+----------------+
| network | asn | 7018 |
| +---------+----------------+
| | country | US |
| +---------+----------------+
| | name | AT&T US - 7018 |
| +---------+----------------+
| | prefix | 104.48.0.0/12 |
| +---------+----------------+
| | rpki | valid |
+----------+---------+----------------+
β ~ monocle ip 1.1.1.1
+----------+----------------------+
| ip | 1.1.1.1 |
+----------+----------------------+
| location | US |
+----------+---------+------------+
| network | asn | 13335 |
| +---------+------------+
| | country | US |
| +---------+------------+
| | name | Cloudflare |
| +---------+------------+
| | prefix | 1.1.1.0/24 |
| +---------+------------+
| | rpki | valid |
+----------+---------+------------+
β ~ monocle ip 1.1.1.1 --json
{
"ip": "1.1.1.1",
"location": "US",
"network": {
"asn": 13335,
"country": "US",
"name": "Cloudflare",
"prefix": "1.1.1.0/24",
"rpki": "valid"
}
}
The monocle parse command now supports
exporting filtered BGP messages into MRT files by supplying an MRT file path with --mrt-path argument.
The parsing of time strings in monocle time and monocle search now utilizes dateparser for natural
date strings like May 6 at 9:24 PM or 2019-11-29 08:08-08.
It now also allows specifying a duration like 1h or "2 hours" to replace --start-ts or --end-ts.
- Updated documentation for various commands
- Cleaned up dependencies in the
Cargo.tomlfile
bgpkit-brokerto v0.7.0 -> v0.7.5bgpkit-parserto v0.10.9 -> v0.10.11
bgpkit-parser version v0.10.11 fixes the improper handling of AS23456 (AS_TRANS). If you previously see
AS23456 incorrectly showing on the path, it should no-longer showing up after this patchshould no-longer show up after
this patch.
- fixed a bug where
psvformat output does not actually print out content.
This is a maintenance release that updates the following dependencies.
bgpkit-brokerto v0.7.0 -> v0.7.1bgpkit-parserto v0.10.9 -> v0.10.10oneioto v0.16.7 -> v0.17.0
With the updated dependencies, monocle now supports using ONEIO_ACCEPT_INVALID_CERTS=true env variable
to run search within a network that uses self-signed certificates.
monocle timenow supports querying multiple time strings in a single querymonocle searchwith--sqlite-pathnow adds found messages to the progress bar during searchmonocle searchnow shows the collector IDs in the results, included in the plaintext, json output as well as the sqlite databasemonocle searchnow supports exporting to MRT files using--mrt-pathparameter
- update
bgpkit-parserto v0.10.5 andoneioto v0.16.7- not depends on
lzandxzfeatures anymore - this change allows
monocleto work on fresh systems with no xz library installed (e.g. more recent macOS)
- not depends on
- update
bgpkit-parserto v0.10.1, which includes a non-trivial performance boost for processing gzip compressed MRT files. - added a new
--simpleoption tomonocle timecommand to allow simple time conversion, suitable for use in scripts.
- remove openssl dependency, switching to rustls as TLS backend
- support installation via
cargo-binstall
- add GitHub actions config to build
monoclebinary for macOS (Universal), and linux (arm and amd64) - add
vendored-openssloptional feature flag to enable GitHub actions builds for different systems. - move
monoclebinary tobindirectory - install
monoclewithbrew install bgpkit/tap/monocle