Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
SuperDB's first GA release is expected in the Summer of 2025. Ongoing changes
SuperDB's first GA release is expected in the 2026. Ongoing changes
will be documented here once that release is made available. To try out
SuperDB while it's still under development we recommend starting from the
[`super` command doc](https://superdb.org/docs/commands/super/).
[`super` command doc](https://superdb.org/command/super.html).

The SuperDB codebase originates from the [Zed project](https://zed.brimdata.io/)
which is no longer under active development. While the full Zed commit history
Expand Down
27 changes: 13 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ SuperDB is a new analytics database that supports relational tables and JSON
on an equal footing. It shines when it comes to data wrangling where
you need to explore or process large eclectic data sets. It's also pretty
decent at analytics and
[search use cases](https://superdb.org/docs/language/search-expressions).
[search use cases](https://superdb.org/super-sql/operators/search.html).

Unlike other relational systems that do performance-fragile "schema inference" of JSON,
SuperDB won't fall over if you throw a bunch of eclectic JSON at it.
You can easily do
[schema inference if you want](https://superdb.org/docs/language/operators/fuse),
[schema inference if you want](https://superdb.org/super-sql/operators/fuse.html),
but data is ingested by default in its natural form no matter how much heterogeneity
it might have. And unlike systems based on the document data model,
every value in SuperDB is strongly and dynamically typed thus providing the
Expand All @@ -27,7 +27,7 @@ way to do things" and a different "JSON way to do things". Instead of having
a relational type system for structured data and completely separate JSON type
system for semi-structured data,
all data handled by SuperDB (e.g., JSON, CSV, Parquet files, Arrow streams, relational tables, etc) is automatically massaged into
[super-structured data](https://superdb.org/docs/formats/#2-a-super-structured-pattern)
[super-structured data](https://superdb.org/intro.html#super-structured-data)
form. This super-structured data is then processed by a runtime that simultaneously
supports the statically-typed relational model and the dynamically-typed
JSON data model in a unified compute engine.
Expand All @@ -36,7 +36,7 @@ JSON data model in a unified compute engine.

SuperDB uses SQL as its query language, but it's a SQL that has been extended
with [pipe syntax](https://research.google/pubs/sql-has-problems-we-can-fix-them-pipe-syntax-in-sql/)
and [lots of fun shortcuts](https://superdb.org/docs/language/pipeline-model/#implied-operators).
and [lots of fun shortcuts](https://superdb.org/super-sql/operators/intro.html#shortcuts).
This extended SQL is called SuperSQL.

Here's a SuperSQL query that fetches some data from GitHub Archive,
Expand All @@ -63,11 +63,11 @@ FROM 'https://data.gharchive.org/2015-01-01-15.json.gz'
Super-structured data is strongly typed and "polymorphic": any value can take on any type
and sequences of data need not all conform to a predefined schema. To this end,
SuperDB extends the JSON format to support super-structured data in a format called
[Super (SUP)](https://superdb.org/docs/formats/sup) where all JSON values
[Super (SUP)](https://superdb.org/formats/sup.html) where all JSON values
are also SUP. Similarly,
the [Super Binary (BSUP)](https://superdb.org/docs/formats/bsup) format is an efficient
the [Super Binary (BSUP)](https://superdb.org/formats/sup.html) format is an efficient
binary representation of SUP (a bit like Avro) and the
[Super Columnar (CSUP)](https://superdb.org/docs/formats/csup) format is a columnar
[Super Columnar (CSUP)](https://superdb.org/formats/csup.html) format is a columnar
representation of SUP (a bit like Parquet).

Even though SuperDB is based on these super-structured data formats, it can read and write
Expand All @@ -90,14 +90,14 @@ Our areas of active development include:
## Try It

As SuperDB is still under construction, GA releases are not yet available.
However, you can [install](https://superdb.org/docs/getting_started/install) a build of the
[`super`](https://superdb.org/docs/commands/super) command-line tool based on
However, you can [install](https://superdb.org/getting-started/install.html) a build of the
[`super`](https://superdb.org/command/super.html) command-line tool based on
code that's under active development to start tinkering. Detailed documentation
for the SuperDB system and its piped SQL syntax is available on the
[SuperDB docs site](https://superdb.org/docs).
[SuperDB docs site](https://superdb.org).

As the code and docs are evolving, we recommend focusing first on what's in the
[`super` command doc](https://superdb.org/docs/commands/super). Feel free to
[`super` command doc](https://superdb.org/command/super.html). Feel free to
explore other docs and try things out, but please don't be shocked if you hit
speedbumps in the near term, particularly in areas like performance and full
SQL coverage. We're working on it! :wink:
Expand All @@ -107,9 +107,8 @@ Once you've tried it out, we'd love to hear your feedback via our

>**NOTE:** The SuperDB query engine can run locally without a storage engine by accessing
>files, HTTP endpoints, or S3 paths using the `super` command. While
>[earlier in its development](https://superdb.org/docs/commands/super-db/#status),
>SuperDB can also run on a
>[super-structured database](https://superdb.org/docs/commands/super-db/#the-lake-model)
>earlier in its development, SuperDB can also run on a
>[super-structured database](https://superdb.org/command/db)
>using the `super db` sub-commands.

### SuperDB Desktop - Coming Soon
Expand Down
5 changes: 0 additions & 5 deletions book/src/command/db-auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,3 @@ Additional options of the [db sub-command](db.md#options)

> **TODO: rename this command. it's really about connecting to a database.
> authenticating is something you do to connect.**
Access to a lake can be secured with [Auth0 authentication](https://auth0.com/).
A [guide](../dev/integrations/auth.md) is available with example configurations.
Please reach out to us on our [community Slack](https://www.brimdata.io/join-slack/)
if you have feedback on your experience or need additional help.
Comment on lines -21 to -25
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mccanne: I saw that the article this was linking to had a !! prepended to it in the SUMMARY.md, so it's not currently alive to link to:

!! - [Authentication](dev/integrations/auth.md)

I took that to mean you were intentionally disabling it and hence figured I'd drop this paragraph rather than leave orphan text describing an absent article. Let me know if I misunderstood your intent though.

2 changes: 1 addition & 1 deletion book/src/getting-started/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
> **TODO: upon release, update this first paragraph.**

Because SuperDB is still under construction, GA releases are not yet available.
However, you can install a build of the [`super`](https://superdb.org/docs/commands/super)
However, you can install a build of the [`super`](../command/super.md)
command-line tool based on code that's under active development to start
tinkering.

Expand Down
2 changes: 1 addition & 1 deletion book/src/tutorials/jq.md
Original file line number Diff line number Diff line change
Expand Up @@ -608,7 +608,7 @@ super -f json \
> empty records no longer appear if the download is repeated today using the same
> URL shown above. But taming glitchy data is a big part of data discovery, so to
> relive the magic of our original experience, you can download
> [this archived copy](https://superdb.org/docs/tutorials/prs.json) of the
> [this archived copy](prs.json) of the
> `prs.json` we originally saw._

Now that you have this JSON file on your local file system, how would you query it
Expand Down
2 changes: 1 addition & 1 deletion cmd/super/root/command.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Output is written to one or more files or to standard output.

A query is comprised of one or more operators interconnected
into a pipeline using the pipe symbol "|" or the alternate "|>".
See https://superdb.org/docs/
See https://superdb.org
for details. The "select" and "from" operators provide backward
compatibility with SQL. In fact, you can use SQL exclusively and
avoid pipeline operators altogether if you prefer.
Expand Down
2 changes: 1 addition & 1 deletion python/superdb/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# `superdb` Python Package

Visit <https://superdb.org/docs/libraries/python/> for installation
Visit <https://superdb.org/dev/libraries/python.html> for installation
instructions and example usage.
5 changes: 2 additions & 3 deletions scripts/super-cmd-perf/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# Query Performance From `super` Command Doc
# Query Performance From Tutorial Doc

These scripts were used to generate the results in the
[Performance](https://superdb.org/docs/commands/super/#performance)
section of the [`super` command doc](https://superdb.org/docs/commands/super).
[Performance Tutorial doc](https://superdb.org/tutorials/performance.html).
The scripts have been made available to allow for easy reproduction of the
results under different conditions and/or as tested systems evolve.

Expand Down