Skip to content

Commit 8f3ebdc

Browse files
derrickstoleederrickstolee
committed
Merge pull request git-for-windows#377: Update README to include Scalar CLI details
I created these as `fixup!` commits so they get squashed in the next major release rebase, but also so they can be reviewed commit-by-commit. * The first is a doozy, adjusting line widths and otherwise updating some minor things here and there. * The rest are small and explained by the body of the commit messages. * One non-`fixup!` commit adds the `docs/` directory from `microsoft/scalar`, with some edits due to the C port of Scalar.
2 parents 60e6b64 + ab42168 commit 8f3ebdc

File tree

5 files changed

+336
-0
lines changed

5 files changed

+336
-0
lines changed

‎contrib/scalar/docs/faq.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
Frequently Asked Questions
2+
==========================
3+
4+
Using Scalar
5+
------------
6+
7+
### I don't want a sparse clone, I want every file after I clone!
8+
9+
Run `scalar clone --full-clone <url>` to initialize your repo to include
10+
every file. You can switch to a sparse-checkout later by running
11+
`git sparse-checkout init --cone`.
12+
13+
### I already cloned without `--full-clone`. How do I get everything?
14+
15+
Run `git sparse-checkout disable`.
16+
17+
Scalar Design Decisions
18+
-----------------------
19+
20+
There may be many design decisions within Scalar that are confusing at first
21+
glance. Some of them may cause friction when you use Scalar with your existing
22+
repos and existing habits.
23+
24+
> Scalar has the most benefit when users design repositories
25+
> with efficient patterns.
26+
27+
For example: Scalar uses the sparse-checkout feature to limit the size of the
28+
working directory within a large monorepo. It is designed to work efficiently
29+
with monorepos that are highly componentized, allowing most developers to
30+
need many fewer files in their daily work.
31+
32+
### Why does `scalar clone` create a `<repo>/src` folder?
33+
34+
Scalar uses a file system watcher to keep track of changes under this `src` folder.
35+
Any activity in this folder is assumed to be important to Git operations. By
36+
creating the `src` folder, we are making it easy for your build system to
37+
create output folders outside the `src` directory. We commonly see systems
38+
create folders for build outputs and package downloads. Scalar itself creates
39+
these folders during its builds.
40+
41+
Your build system may create build artifacts such as `.obj` or `.lib` files
42+
next to your source code. These are commonly "hidden" from Git using
43+
`.gitignore` files. Having such artifacts in your source tree creates
44+
additional work for Git because it needs to look at these files and match them
45+
against the `.gitignore` patterns.
46+
47+
By following the `src` pattern Scalar tries to establish and placing your build
48+
intermediates and outputs parallel with the `src` folder and not inside it,
49+
you can help optimize Git command performance for developers in the repository
50+
by limiting the number of files Git needs to consider for many common
51+
operations.

‎contrib/scalar/docs/getting-started.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
Getting Started
2+
===============
3+
4+
Registering existing Git repos
5+
------------------------------
6+
7+
To add a repository to the list of registered repos, run `scalar register [<path>]`.
8+
If `<path>` is not provided, then the "current repository" is discovered from
9+
the working directory by scanning the parent paths for a path containing a `.git`
10+
folder, possibly inside a `src` folder.
11+
12+
To see which repositories are currently tracked by the service, run
13+
`scalar list`.
14+
15+
Run `scalar unregister [<path>]` to remove the repo from this list.
16+
17+
Creating a new Scalar clone using the GVFS Protocol
18+
---------------------------------------------------
19+
20+
The `clone` verb creates a local enlistment of a remote repository using the
21+
[GVFS protocol](https://github.com/microsoft/VFSForGit/blob/HEAD/Protocol.md),
22+
such as Azure Repos.
23+
24+
```
25+
scalar clone [options] <url> [<dir>]
26+
```
27+
28+
Create a local copy of the repository at `<url>`. If specified, create the `<dir>`
29+
directory and place the repository there. Otherwise, the last section of the `<url>`
30+
will be used for `<dir>`.
31+
32+
At the end, the repo is located at `<dir>/src`. By default, the sparse-checkout
33+
feature is enabled and the only files present are those in the root of your
34+
Git repository. Use `git sparse-checkout set` to expand the set of directories
35+
you want to see, or `git sparse-checkout disable` to expand to all files. You
36+
can explore the subdirectories outside your sparse-checkout specification using
37+
`git ls-tree HEAD`.
38+
39+
### Sparse Repo Mode
40+
41+
By default, Scalar reduces your working directory to only the files at the
42+
root of the repository. You need to add the folders you care about to build up
43+
to your working set.
44+
45+
* `scalar clone <url>`
46+
* Please choose the **Clone with HTTPS** option in the `Clone Repository` dialog in Azure Repos, not **Clone with SSH**.
47+
* `cd <root>\src`
48+
* At this point, your `src` directory only contains files that appear in your root
49+
tree. No folders are populated.
50+
* Set the directory list for your sparse-checkout using:
51+
1. `git sparse-checkout set <dir1> <dir2> ...`
52+
2. `git sparse-checkout set --stdin < dir-list.txt`
53+
* Run git commands as you normally would.
54+
* To fully populate your working directory, run `git sparse-checkout disable`.
55+
56+
If instead you want to start with all files on-disk, you can clone with the
57+
`--full-clone` option. To enable sparse-checkout after the fact, run
58+
`git sparse-checkout init --cone`. This will initialize your sparse-checkout
59+
patterns to only match the files at root.
60+
61+
If you are unfamiliar with what directories are available in the repository,
62+
then you can run `git ls-tree -d --name-only HEAD` to discover the directories
63+
at root, or `git ls-tree -d --name-only HEAD <path>` to discover the directories
64+
in `<path>`.
65+
66+
### Options
67+
68+
These options allow a user to customize their initial enlistment.
69+
70+
* `--full-clone`: If specified, do not initialize the sparse-checkout feature.
71+
All files will be present in your `src` directory. This behaves very similar
72+
to a Git partial clone in that blobs are downloaded on demand. However, it
73+
will use the GVFS protocol to download all Git objects.
74+
75+
* `--cache-server-url=<url>`: If specified, set the intended cache server to
76+
the specified `<url>`. All object queries will use the GVFS protocol to this
77+
`<url>` instead of the origin remote. If the remote supplies a list of
78+
cache servers via the `<url>/gvfs/config` endpoint, then the `clone` command
79+
will select a nearby cache server from that list.
80+
81+
* `--branch=<ref>`: Specify the branch to checkout after clone.
82+
83+
* `--local-cache-path=<path>`: Use this option to override the path for the
84+
local Scalar cache. If not specified, then Scalar will select a default
85+
path to share objects with your other enlistments. On Windows, this path
86+
is a subdirectory of `<Volume>:\.scalarCache\`. On Mac, this path is a
87+
subdirectory of `~/.scalarCache/`. The default cache path is recommended so
88+
multiple enlistments of the same remote repository share objects on the
89+
same device.
90+
91+
### Advanced Options
92+
93+
The options below are not intended for use by a typical user. These are
94+
usually used by build machines to create a temporary enlistment that
95+
operates on a single commit.
96+
97+
* `--single-branch`: Use this option to only download metadata for the branch
98+
that will be checked out. This is helpful for build machines that target
99+
a remote with many branches. Any `git fetch` commands after the clone will
100+
still ask for all branches.
101+
102+
* `--no-prefetch`: Use this option to not prefetch commits after clone. This
103+
is not recommended for anyone planning to use their clone for history
104+
traversal. Use of this option will make commands like `git log` or
105+
`git pull` extremely slow and is therefore not recommended.
106+
107+
Removing a Scalar Clone
108+
-----------------------
109+
110+
Since the `scalar clone` command sets up a file-system watcher (when available),
111+
that watcher could prevent deleting the enlistment. Run `scalar delete <path>`
112+
from outside of your enlistment to unregister the enlistment from the filesystem
113+
watcher and delete the enlistment at `<path>`.

‎contrib/scalar/docs/index.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
Scalar: Enabling Git at Scale
2+
=============================
3+
4+
Scalar is a tool that helps Git scale to some of the largest Git repositories.
5+
It achieves this by enabling some advanced Git features, such as:
6+
7+
* *Partial clone:* reduces time to get a working repository by not
8+
downloading all Git objects right away.
9+
10+
* *Background prefetch:* downloads Git object data from all remotes every
11+
hour, reducing the amount of time for foreground `git fetch` calls.
12+
13+
* *Sparse-checkout:* limits the size of your working directory.
14+
15+
* *File system monitor:* tracks the recently modified files and eliminates
16+
the need for Git to scan the entire worktree.
17+
18+
* *Commit-graph:* accelerates commit walks and reachability calculations,
19+
speeding up commands like `git log`.
20+
21+
* *Multi-pack-index:* enables fast object lookups across many pack-files.
22+
23+
* *Incremental repack:* Repacks the packed Git data into fewer pack-file
24+
without disrupting concurrent commands by using the multi-pack-index.
25+
26+
By running `scalar register` in any Git repo, Scalar will automatically enable
27+
these features for that repo (except partial clone) and start running suggested
28+
maintenance in the background using
29+
[the `git maintenance` feature](https://git-scm.com/docs/git-maintenance).
30+
31+
Repos cloned with the `scalar clone` command use partial clone or the
32+
[GVFS protocol](https://github.com/microsoft/VFSForGit/blob/HEAD/Protocol.md)
33+
to significantly reduce the amount of data required to get started
34+
using a repository. By delaying all blob downloads until they are required,
35+
Scalar allows you to work with very large repositories quickly. The GVFS
36+
protocol allows a network of _cache servers_ to serve objects with lower
37+
latency and higher throughput. The cache servers also reduce load on the
38+
central server.
39+
40+
Documentation
41+
-------------
42+
43+
* [Getting Started](getting-started.md): Get started with Scalar.
44+
Includes `scalar register`, `scalar unregister`, `scalar clone`, and
45+
`scalar delete`.
46+
47+
* [Troubleshooting](troubleshooting.md):
48+
Collect diagnostic information or update custom settings. Includes
49+
`scalar diagnose` and `scalar cache-server`.
50+
51+
* [The Philosophy of Scalar](philosophy.md): Why does Scalar work the way
52+
it does, and how do we make decisions about its future?
53+
54+
* [Frequently Asked Questions](faq.md)

‎contrib/scalar/docs/philosophy.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
The Philosophy of Scalar
2+
========================
3+
4+
The team building Scalar has **opinions** about Git performance. Scalar
5+
takes out the guesswork by automatically configuring your Git repositories
6+
to take advantage of the latest and greatest features. It is difficult to
7+
say that these are the absolute best settings for every repository, but
8+
these settings do work for some of the largest repositories in the world.
9+
10+
Scalar intends to do very little more than the standard Git client. We
11+
actively implement new features into Git instead of Scalar, then update
12+
Scalar only to configure those new settings. In particular, we are porting
13+
features like background maintenance to Git to make Scalar simpler and
14+
make Git more powerful.
15+
16+
Scalar ships inside [a custom version of Git][microsoft-git], but we are
17+
working to make it available in other forks of Git. The only feature
18+
that is not intended to ever reach the standard Git client is Scalar's use
19+
of [the GVFS Protocol][gvfs-protocol], which is essentially an older
20+
version of [Git's partial clone feature](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/)
21+
that was available first in Azure Repos. Services such as GitHub support
22+
only partial clone instead of the GVFS protocol because that is the
23+
standard adopted by the Git project. If your hosting service supports
24+
partial clone, then we absolutely recommend it as a way to greatly speed
25+
up your clone and fetch times and to reduce how much disk space your Git
26+
repository requires. Scalar will help with this!
27+
28+
If you don't use the GVFS Protocol, then most of the value of Scalar can
29+
be found in the core Git client. However, most of the advanced features
30+
that really optimize Git's performance are off by default for compatibility
31+
reasons. To really take advantage of Git's latest and greatest features,
32+
you either need to study the [`git config` documentation](https://git-scm.com/docs/git-config)
33+
and regularly read [the Git release notes](https://github.com/git/git/tree/master/Documentation/RelNotes).
34+
Even if you do all that work and customize your Git settings on your machines,
35+
you likely will want to share those settings with other team members.
36+
Or, you can just use Scalar!
37+
38+
Using `scalar register` on an existing Git repository will give you these
39+
benefits:
40+
41+
* Additional compression of your `.git/index` file.
42+
* Hourly background `git fetch` operations, keeping you in-sync with your
43+
remotes.
44+
* Advanced data structures, such as the `commit-graph` and `multi-pack-index`
45+
are updated automatically in the background.
46+
* If using macOS or Windows, then Scalar configures Git's builtin File System
47+
Monitor, providing faster commands such as `git status` or `git add`.
48+
49+
Additionally, if you use `scalar clone` to create a new repository, then
50+
you will automatically get these benefits:
51+
52+
* Use Git's partial clone feature to only download the files you need for
53+
your current checkout.
54+
* Use Git's [sparse-checkout feature][sparse-checkout] to minimize the
55+
number of files required in your working directory.
56+
[Read more about sparse-checkout here.][sparse-checkout-blog]
57+
* Create the Git repository inside `<repo-name>/src` to make it easy to
58+
place build artifacts outside of the Git repository, such as in
59+
`<repo-name>/bin` or `<repo-name>/packages`.
60+
61+
We also admit that these **opinions** can always be improved! If you have
62+
an idea of how to improve our setup, consider
63+
[creating an issue](https://github.com/microsoft/scalar/issues/new) or
64+
contributing a pull request! Some [existing](https://github.com/microsoft/scalar/issues/382)
65+
[issues](https://github.com/microsoft/scalar/issues/388) have already
66+
improved our configuration settings and roadmap!
67+
68+
[gvfs-protocol]: https://github.com/microsoft/VFSForGit/blob/HEAD/Protocol.md
69+
[microsoft-git]: https://github.com/microsoft/git
70+
[sparse-checkout]: https://git-scm.com/docs/git-sparse-checkout
71+
[sparse-checkout-blog]: https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/

‎contrib/scalar/docs/troubleshooting.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
Troubleshooting
2+
===============
3+
4+
Diagnosing Issues
5+
-----------------
6+
7+
The `scalar diagnose` command collects logs and config details for the current
8+
repository. The resulting zip file helps root-cause issues.
9+
10+
When run inside your repository, creates a zip file containing several important
11+
files for that repository. This includes:
12+
13+
* Configuration files from your `.git` folder, such as the `config` file,
14+
`index`, `hooks`, and `refs`.
15+
16+
* A summary of your Git object database, including the number of loose objects
17+
and the names and sizes of pack-files.
18+
19+
As the `diagnose` command completes, it provides the path of the resulting
20+
zip file. This zip can be sent to [the support team](mailto:[email protected])
21+
for investigation.
22+
23+
Modifying Configuration Values
24+
------------------------------
25+
26+
The Scalar-specific configuration is only available for repos using the
27+
GVFS protocol.
28+
29+
### Cache Server URL
30+
31+
When using an enlistment cloned with `scalar clone` and the GVFS protocol,
32+
you will have a value called the cache server URL. Cache servers are a feature
33+
of the GVFS protocol to provide low-latency access to the on-demand object
34+
requests. This modifies the `gvfs.cache-server` setting in your local Git config
35+
file.
36+
37+
Run `scalar cache-server --get` to see the current cache server.
38+
39+
Run `scalar cache-server --list` to see the available cache server URLs.
40+
41+
Run `scalar cache-server --set=<url>` to set your cache server to `<url>`.
42+
43+
### Scalar Config
44+
45+
The `scalar config` command is used for customizing the feed used for
46+
Scalar upgrades. This is so large teams can bundle a custom installer
47+
or other tools along with Scalar upgrades.

0 commit comments

Comments
 (0)