Merge branch 'main' of github.com:git/git

* 'main' of github.com:git/git: (45 commits)
  A bit more of remaining topics before -rc1
  t1800: correct test to handle Cygwin
  chainlint: colorize problem annotations and test delimiters
  ls-files: fix black space in error message
  list-objects-filter: convert filter_spec to a strbuf
  list-objects-filter: add and use initializers
  list-objects-filter: handle null default filter spec
  list-objects-filter: don't memset after releasing filter struct
  builtin/mv.c: fix possible segfault in add_slash()
  Documentation/technical: include Scalar technical doc
  t/perf: add 'GIT_PERF_USE_SCALAR' run option
  t/perf: add Scalar performance tests
  scalar-clone: add test coverage
  scalar: add to 'git help -a' command list
  scalar: implement the `help` subcommand
  git help: special-case `scalar`
  scalar: include in standard Git build & installation
  scalar: fix command documentation section header
  t: retire unused chainlint.sed
  t/Makefile: teach `make test` and `make prove` to run chainlint.pl
  ...

Author: jiangxin

.gitignore

@@ -185,6 +185,7 @@
 /git-whatchanged
 /git-worktree
 /git-write-tree
+/scalar
 /git-core-*/?*
 /git.res
 /gitweb/GITWEB-BUILD-OPTIONS

Documentation/Makefile

@@ -21,6 +21,7 @@ MAN1_TXT += $(filter-out \
 MAN1_TXT += git.txt
 MAN1_TXT += gitk.txt
 MAN1_TXT += gitweb.txt
+MAN1_TXT += scalar.txt
 
 # man5 / man7 guides (note: new guides should also be added to command-list.txt)
 MAN5_TXT += gitattributes.txt
@@ -116,6 +117,7 @@ TECH_DOCS += technical/parallel-checkout
 TECH_DOCS += technical/partial-clone
 TECH_DOCS += technical/racy-git
 TECH_DOCS += technical/reftable
+TECH_DOCS += technical/scalar
 TECH_DOCS += technical/send-pack-pipeline
 TECH_DOCS += technical/shallow
 TECH_DOCS += technical/trivial-merge

Documentation/RelNotes/2.38.0.txt

@@ -6,7 +6,7 @@ UI, Workflows & Features
  * "git remote show [-n] frotz" now pays attention to negative
    pathspec.
 
- * "git push" sometimes perform poorly when reachability bitmaps are
+ * "git push" sometimes performs poorly when reachability bitmaps are
    used, even in a repository where other operations are helped by
    bitmaps.  The push.useBitmaps configuration variable is introduced
    to allow disabling use of reachability bitmaps only for "git push".
@@ -27,7 +27,7 @@ UI, Workflows & Features
    what locale they are in by sending Accept-Language HTTP header, but
    this was done only for some requests but not others.
 
- * Introduce a discovery.barerepository configuration variable that
+ * Introduce a safe.barerepository configuration variable that
    allows users to forbid discovery of bare repositories.
 
  * Various messages that come from the pack-bitmap codepaths have been
@@ -79,12 +79,15 @@ UI, Workflows & Features
 
  * "git format-patch --from=<ident>" can be told to add an in-body
    "From:" line even for commits that are authored by the given
-   <ident> with "--force-in-body-from"option.
+   <ident> with "--force-in-body-from" option.
 
  * The built-in fsmonitor refuses to work on a network mounted
    repositories; a configuration knob for users to override this has
    been introduced.
 
+ * The "scalar" addition from Microsoft is now part of the core Git
+   installation.
+
 
 Performance, Internal Implementation, Development Support etc.
 
@@ -127,7 +130,7 @@ Performance, Internal Implementation, Development Support etc.
 
  * The way "git multi-pack" uses parse-options API has been improved.
 
- * A coccinelle rule (in contrib/) to encourage use of COPY_ARRAY
+ * A Coccinelle rule (in contrib/) to encourage use of COPY_ARRAY
    macro has been improved.
 
  * API tweak to make it easier to run fuzz testing on commit-graph parser.
@@ -172,6 +175,12 @@ Performance, Internal Implementation, Development Support etc.
  * Share the text used to explain configuration variables used by "git
    <subcmd>" in "git help <subcmd>" with the text from "git help config".
 
+ * "git mv A B" in a sparsely populated working tree can be asked to
+   move a path from a directory that is "in cone" to another directory
+   that is "out of cone".  Handling of such a case has been improved.
+
+ * The chainlint script for our tests has been revamped.
+
 
 Fixes since v2.37
 -----------------
@@ -297,7 +306,7 @@ Fixes since v2.37
  * "git fsck" reads mode from tree objects but canonicalizes the mode
    before passing it to the logic to check object sanity, which has
    hid broken tree objects from the checking logic.  This has been
-   corrected, but to help exiting projects with broken tree objects
+   corrected, but to help existing projects with broken tree objects
    that they cannot fix retroactively, the severity of anomalies this
    code detects has been demoted to "info" for now.
 
@@ -306,33 +315,27 @@ Fixes since v2.37
 
  * An earlier optimization discarded a tree-object buffer that is
    still in use, which has been corrected.
-   (merge 1490d7d82d jk/is-promisor-object-keep-tree-in-use later to maint).
 
  * Fix deadlocks between main Git process and subprocess spawned via
    the pipe_command() API, that can kill "git add -p" that was
    reimplemented in C recently.
-   (merge 716c1f649e jk/pipe-command-nonblock later to maint).
 
  * The sequencer machinery translated messages left in the reflog by
    mistake, which has been corrected.
 
  * xcalloc(), imitating calloc(), takes "number of elements of the
    array", and "size of a single element", in this order.  A call that
    does not follow this ordering has been corrected.
-   (merge c4bbd9bb8f sg/xcalloc-cocci-fix later to maint).
 
  * The preload-index codepath made copies of pathspec to give to
    multiple threads, which were left leaked.
-   (merge 23578904da ad/preload-plug-memleak later to maint).
 
  * Update the version of Ubuntu used for GitHub Actions CI from 18.04
    to 22.04.
-   (merge ef46584831 ds/github-actions-use-newer-ubuntu later to maint).
 
  * The auto-stashed local changes created by "git merge --autostash"
    was mixed into a conflicted state left in the working tree, which
    has been corrected.
-   (merge d3a9295ada en/merge-unstash-only-on-clean-merge later to maint).
 
  * Multi-pack index got corrupted when preferred pack changed from one
    pack to another in a certain way, which has been corrected.

Documentation/cmd-list.perl

@@ -10,7 +10,7 @@ sub format_one {
 	$state = 0;
 	open I, '<', "$name.txt" or die "No such file $name.txt";
 	while (<I>) {
-		if (/^git[a-z0-9-]*\(([0-9])\)$/) {
+		if (/^(git|scalar)[a-z0-9-]*\(([0-9])\)$/) {
 			$mansection = $1;
 			next;
 		}

contrib/scalar/scalar.txt renamed to Documentation/scalar.txt

@@ -161,6 +161,6 @@ SEE ALSO
 --------
 linkgit:git-clone[1], linkgit:git-maintenance[1].
 
-Scalar
+GIT
 ---
-Associated with the linkgit:git[1] suite
+Part of the linkgit:git[1] suite

Documentation/technical/scalar.txt

@@ -64,64 +64,3 @@ some "global" `git` options (e.g., `-c` and `-C`).
 Because `scalar` is not invoked as a Git subcommand (like `git scalar`), it is
 built and installed as its own executable in the `bin/` directory, alongside
 `git`, `git-gui`, etc.
-
-Roadmap
--------
-
-NOTE: this section will be removed once the remaining tasks outlined in this
-roadmap are complete.
-
-Scalar is a large enough project that it is being upstreamed incrementally,
-living in `contrib/` until it is feature-complete. So far, the following patch
-series have been accepted:
-
-- `scalar-the-beginning`: The initial patch series which sets up
-  `contrib/scalar/` and populates it with a minimal `scalar` command that
-  demonstrates the fundamental ideas.
-
-- `scalar-c-and-C`: The `scalar` command learns about two options that can be
-  specified before the command, `-c <key>=<value>` and `-C <directory>`.
-
-- `scalar-diagnose`: The `scalar` command is taught the `diagnose` subcommand.
-
-- `scalar-generalize-diagnose`: Move the functionality of `scalar diagnose`
-  into `git diagnose` and `git bugreport --diagnose`.
-
-- 'scalar-add-fsmonitor: Enable the built-in FSMonitor in Scalar
-  enlistments. At the end of this series, Scalar should be feature-complete
-  from the perspective of a user.
-
-Roughly speaking (and subject to change), the following series are needed to
-"finish" this initial version of Scalar:
-
-- Move Scalar to toplevel: Move Scalar out of `contrib/` and into the root of
-  `git`. This includes a variety of related updates, including:
-    - building & installing Scalar in the Git root-level 'make [install]'.
-    - builing & testing Scalar as part of CI.
-    - moving and expanding test coverage of Scalar (including perf tests).
-    - implementing 'scalar help'/'git help scalar' to display scalar
-      documentation.
-
-Finally, there are two additional patch series that exist in Microsoft's fork of
-Git, but there is no current plan to upstream them. There are some interesting
-ideas there, but the implementation is too specific to Azure Repos and/or VFS
-for Git to be of much help in general.
-
-These still exist mainly because the GVFS protocol is what Azure Repos has
-instead of partial clone, while Git is focused on improving partial clone:
-
-- `scalar-with-gvfs`: The primary purpose of this patch series is to support
-  existing Scalar users whose repositories are hosted in Azure Repos (which does
-  not support Git's partial clones, but supports its predecessor, the GVFS
-  protocol, which is used by Scalar to emulate the partial clone).
-
-  Since the GVFS protocol will never be supported by core Git, this patch series
-  will remain in Microsoft's fork of Git.
-
-- `run-scalar-functional-tests`: The Scalar project developed a quite
-  comprehensive set of integration tests (or, "Functional Tests"). They are the
-  sole remaining part of the original C#-based Scalar project, and this patch
-  adds a GitHub workflow that runs them all.
-
-  Since the tests partially depend on features that are only provided in the
-  `scalar-with-gvfs` patch series, this patch cannot be upstreamed.

Makefile

@@ -605,7 +605,9 @@ FUZZ_OBJS =
 FUZZ_PROGRAMS =
 GIT_OBJS =
 LIB_OBJS =
+SCALAR_OBJS =
 OBJECTS =
+OTHER_PROGRAMS =
 PROGRAM_OBJS =
 PROGRAMS =
 EXCLUDED_PROGRAMS =
@@ -819,10 +821,12 @@ BUILT_INS += git-switch$X
 BUILT_INS += git-whatchanged$X
 
 # what 'all' will build but not install in gitexecdir
-OTHER_PROGRAMS = git$X
+OTHER_PROGRAMS += git$X
+OTHER_PROGRAMS += scalar$X
 
 # what test wrappers are needed and 'install' will install, in bindir
 BINDIR_PROGRAMS_NEED_X += git
+BINDIR_PROGRAMS_NEED_X += scalar
 BINDIR_PROGRAMS_NEED_X += git-receive-pack
 BINDIR_PROGRAMS_NEED_X += git-shell
 BINDIR_PROGRAMS_NEED_X += git-upload-archive
@@ -2220,7 +2224,7 @@ profile-fast: profile-clean
 
 all:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS
 ifneq (,$X)
-	$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) git$X)), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)
+	$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) $(OTHER_PROGRAMS))), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)
 endif
 
 all::
@@ -2543,7 +2547,12 @@ GIT_OBJS += git.o
 .PHONY: git-objs
 git-objs: $(GIT_OBJS)
 
+SCALAR_OBJS += scalar.o
+.PHONY: scalar-objs
+scalar-objs: $(SCALAR_OBJS)
+
 OBJECTS += $(GIT_OBJS)
+OBJECTS += $(SCALAR_OBJS)
 OBJECTS += $(PROGRAM_OBJS)
 OBJECTS += $(TEST_OBJS)
 OBJECTS += $(XDIFF_OBJS)
@@ -2554,10 +2563,6 @@ ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
 endif
 
-SCALAR_SOURCES := contrib/scalar/scalar.c
-SCALAR_OBJECTS := $(SCALAR_SOURCES:c=o)
-OBJECTS += $(SCALAR_OBJECTS)
-
 .PHONY: objects
 objects: $(OBJECTS)
 
@@ -2683,7 +2688,7 @@ $(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)
 
-contrib/scalar/scalar$X: $(SCALAR_OBJECTS) GIT-LDFLAGS $(GITLIBS)
+scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
@@ -2739,8 +2744,7 @@ XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \
 XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \
 	--keyword=__ --keyword=N__ --keyword="__n:1,2"
 MSGMERGE_FLAGS = --add-location --backup=off --update
-LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(SCALAR_SOURCES) \
-	        $(GENERATED_H))
+LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(GENERATED_H))
 LOCALIZED_SH = $(sort $(SCRIPT_SH) git-sh-setup.sh)
 LOCALIZED_PERL = $(sort $(SCRIPT_PERL))
 
@@ -3054,7 +3058,7 @@ bin-wrappers/%: wrap-for-bin.sh
 	$(call mkdir_p_parent_template)
 	$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
 	     -e 's|@@BUILD_DIR@@|$(shell pwd)|' \
-	     -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \
+	     -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \
 	chmod +x $@
 
 # GNU make supports exporting all variables by "export" without parameters.
@@ -3268,14 +3272,14 @@ ifndef NO_TCLTK
 	$(MAKE) -C git-gui gitexecdir='$(gitexec_instdir_SQ)' install
 endif
 ifneq (,$X)
-	$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) git$X)), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)
+	$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) $(OTHER_PROGRAMS))), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)
 endif
 
 	bindir=$$(cd '$(DESTDIR_SQ)$(bindir_SQ)' && pwd) && \
 	execdir=$$(cd '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' && pwd) && \
 	destdir_from_execdir_SQ=$$(echo '$(gitexecdir_relative_SQ)' | sed -e 's|[^/][^/]*|..|g') && \
 	{ test "$$bindir/" = "$$execdir/" || \
-	  for p in git$X $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \
+	  for p in $(OTHER_PROGRAMS) $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \
 		$(RM) "$$execdir/$$p" && \
 		test -n "$(INSTALL_SYMLINKS)" && \
 		ln -s "$$destdir_from_execdir_SQ/$(bindir_relative_SQ)/$$p" "$$execdir/$$p" || \
@@ -3450,7 +3454,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) git.res
 	$(RM) $(OBJECTS)
 	$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)
-	$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X
+	$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)
 	$(RM) $(TEST_PROGRAMS)
 	$(RM) $(FUZZ_PROGRAMS)
 	$(RM) $(SP_OBJ)
@@ -3501,6 +3505,7 @@ ALL_COMMANDS += git-citool
 ALL_COMMANDS += git-gui
 ALL_COMMANDS += gitk
 ALL_COMMANDS += gitweb
+ALL_COMMANDS += scalar
 
 .PHONY: check-docs
 check-docs::

advice.c

@@ -261,3 +261,22 @@ void detach_advice(const char *new_name)
 
 	fprintf(stderr, fmt, new_name);
 }
+
+void advise_on_moving_dirty_path(struct string_list *pathspec_list)
+{
+	struct string_list_item *item;
+
+	if (!pathspec_list->nr)
+		return;
+
+	fprintf(stderr, _("The following paths have been moved outside the\n"
+			  "sparse-checkout definition but are not sparse due to local\n"
+			  "modifications.\n"));
+	for_each_string_list_item(item, pathspec_list)
+		fprintf(stderr, "%s\n", item->string);
+
+	advise_if_enabled(ADVICE_UPDATE_SPARSE_PATH,
+			  _("To correct the sparsity of these paths, do the following:\n"
+			    "* Use \"git add --sparse <paths>\" to update the index\n"
+			    "* Use \"git sparse-checkout reapply\" to apply the sparsity rules"));
+}

advice.h

@@ -74,5 +74,6 @@ void NORETURN die_conclude_merge(void);
 void NORETURN die_ff_impossible(void);
 void advise_on_updating_sparse_paths(struct string_list *pathspec_list);
 void detach_advice(const char *new_name);
+void advise_on_moving_dirty_path(struct string_list *pathspec_list);
 
 #endif /* ADVICE_H */

builtin/clone.c

@@ -73,7 +73,7 @@ static struct string_list option_optional_reference = STRING_LIST_INIT_NODUP;
 static int option_dissociate;
 static int max_jobs = -1;
 static struct string_list option_recurse_submodules = STRING_LIST_INIT_NODUP;
-static struct list_objects_filter_options filter_options;
+static struct list_objects_filter_options filter_options = LIST_OBJECTS_FILTER_INIT;
 static int option_filter_submodules = -1;    /* unspecified */
 static int config_filter_submodules = -1;    /* unspecified */
 static struct string_list server_options = STRING_LIST_INIT_NODUP;