Implement #:ref directive for file-based apps

documentation/general/dotnet-run-file.md

@@ -110,8 +110,8 @@ If a dash (`-`) is given instead of the target path (i.e., `dotnet run -`), the
 In this case, the current working directory is not used to search for other files (launch profiles, other sources in case of multi-file apps);
 the compilation consists solely of the single file read from the standard input.
 However, the current working directory is still used as the working directory for building and executing the program.
-To reference projects relative to the current working directory (instead of relative to the temporary directory the file is isolated in),
-you can use something like `#:project $(MSBuildStartupDirectory)/relative/path`.
+To reference projects or files relative to the current working directory (instead of relative to the temporary directory the file is isolated in),
+you can use something like `#:project $(MSBuildStartupDirectory)/relative/path` or `#:ref $(MSBuildStartupDirectory)/relative/lib.cs`.
 
 `dotnet path.cs` is a shortcut for `dotnet run --file path.cs` provided that `path.cs` is a valid [target path](#target-path) (`dotnet -` is currently not supported)
 and it is not a DLL path, built-in command, or a NuGet tool (e.g., `dotnet watch` invokes the `dotnet-watch` tool
@@ -178,11 +178,12 @@ which are [ignored][ignored-directives] by the C# language but recognized by the
 #:property LangVersion=preview
 #:package System.CommandLine@2.0.0-*
 #:project ../MyLibrary
+#:ref ../lib/lib.cs
 #:include ./**/*.cs
 ```
 
 Each directive has a kind (e.g., `package`), a name (e.g., `System.CommandLine`), a separator (e.g., `@`), and a value (e.g., the package version).
-The value is required for `#:property`, optional for `#:package`/`#:sdk`, and disallowed for `#:project`/`#:include`.  
+The value is required for `#:property`, optional for `#:package`/`#:sdk`, and disallowed for `#:project`/`#:ref`/`#:include`.  
 
 The name must be separated from the kind of the directive by whitespace
 and any leading and trailing white space is not considered part of the name and value.
@@ -209,6 +210,24 @@ The directives are processed as follows:
   (because `ProjectReference` items don't support directory paths).
   An error is reported if zero or more than one projects are found in the directory, just like `dotnet reference add` would do.
 
+- Each `#:ref` references another `.cs` file as a compiled library.
+  A virtual project with `OutputType=Library` is created for the referenced file (e.g., `lib.cs` produces a virtual `lib.cs.csproj`),
+  and a `<ProjectReference Include="lib.cs.csproj" SkipGetTargetFrameworkProperties="true" />` is injected in an `<ItemGroup>`.
+  It is an error if the name is empty or if the referenced file does not exist.
+  Unlike `#:project`, `#:ref` points to a `.cs` file (not a `.csproj` file or directory).
+
+  Because the referenced file is compiled as a separate assembly, internal members of the referenced file are not accessible from the referencing file.
+  The `#:ref` directive is transitive: a referenced file can itself contain `#:ref` directives.
+
+  Relative paths are resolved relative to the file containing the directive.
+  MSBuild variables (like `$(MSBuildProjectDirectory)`) can be used in the path.
+
+  During [conversion](#grow-up), each `#:ref` directive creates a separate library project in a sibling directory
+  and a corresponding `<ProjectReference>` entry is added to the converted project.
+  The conversion is recursive: any `#:ref` directives in the referenced files are also converted in the same way.
+
+  This directive is currently gated under a feature flag that can be enabled by setting the MSBuild property `ExperimentalFileBasedProgramEnableRefDirective=true`.
+
 - Each `#:include` is injected as `<{1} Include="{0}" />` in an `<ItemGroup>`
   where `{0}` is the directive's value and `{1}` is determined by its extension.
   The mapping can be customized by setting the MSBuild property `FileBasedProgramsItemMapping`
@@ -228,9 +247,9 @@ The directives are processed as follows:
 - Other directive kinds result in an error, reserving them for future use.
 
 Directive values support MSBuild variables (like `$(..)`) normally as they are translated literally and left to MSBuild engine to process.
-However, in `#:project` directives, variables might not be preserved during [grow up](#grow-up),
+However, in `#:project` and `#:ref` directives, variables might not be preserved during [grow up](#grow-up),
 because there is additional processing of those directives that makes it technically challenging to preserve variables in all cases
-(project directive values need to be resolved to be relative to the target directory
+(project and ref directive values need to be resolved to be relative to the target directory
 and also to point to a project file rather than a directory).
 Note that it is not expected that variables inside the path change their meaning during the conversion,
 so for example `#:project ../$(LibName)` is translated to `<ProjectReference Include="../../$(LibName)/Lib.csproj" />` (i.e., the variable is preserved).

src/Cli/Microsoft.DotNet.FileBasedPrograms/FileBasedProgramsResources.resx

@@ -165,6 +165,14 @@
     <value>The '#:project' directive is invalid: {0}</value>
     <comment>{0} is the inner error message.</comment>
   </data>
+  <data name="InvalidRefDirective" xml:space="preserve">
+    <value>The '#:ref' directive is invalid: {0}</value>
+    <comment>{Locked="#:ref"}{0} is the inner error message.</comment>
+  </data>
+  <data name="CouldNotFindRefFile" xml:space="preserve">
+    <value>Could not find file '{0}'.</value>
+    <comment>{0} is the file path.</comment>
+  </data>
   <data name="MissingDirectiveName" xml:space="preserve">
     <value>Missing name of '{0}'.</value>
     <comment>{0} is the directive name like 'package' or 'sdk'.</comment>

src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs

@@ -323,6 +323,7 @@ public void ReportError(TextSpan span, string message)
             case "property": return Property.Parse(context);
             case "package": return Package.Parse(context);
             case "project": return Project.Parse(context);
+            case "ref": return Ref.Parse(context);
             case "include" or "exclude": return IncludeOrExclude.Parse(context);
             default:
                 context.ReportError(string.Format(FileBasedProgramsResources.UnrecognizedDirective, context.DirectiveKind));
@@ -587,6 +588,105 @@ void ReportError(string message)
         public override string ToString() => $"#:project {Name}";
     }
 
+    /// <summary>
+    /// <c>#:ref</c> directive. References another file-based app as a library.
+    /// </summary>
+    public sealed class Ref : Named
+    {
+        public const string ExperimentalFileBasedProgramEnableRefDirective = nameof(ExperimentalFileBasedProgramEnableRefDirective);
+
+        [SetsRequiredMembers]
+        public Ref(in ParseInfo info, string name) : base(info)
+        {
+            Name = name;
+            OriginalName = name;
+        }
+
+        /// <summary>
+        /// Preserved across <see cref="WithName"/> calls, i.e.,
+        /// this is the original directive text as entered by the user.
+        /// </summary>
+        public string OriginalName { get; init; }
+
+        /// <summary>
+        /// This is the <see cref="OriginalName"/> with MSBuild <c>$(..)</c> vars expanded.
+        /// </summary>
+        public string? ExpandedName { get; init; }
+
+        /// <summary>
+        /// The resolved full path to the referenced <c>.cs</c> file.
+        /// </summary>
+        public string? ResolvedPath { get; init; }
+
+        public static new Ref? Parse(in ParseContext context)
+        {
+            var directiveText = context.DirectiveText;
+            if (directiveText.IsWhiteSpace())
+            {
+                context.ReportError(string.Format(FileBasedProgramsResources.MissingDirectiveName, context.DirectiveKind));
+                return null;
+            }
+
+            return new Ref(context.Info, directiveText);
+        }
+
+        public enum NameKind
+        {
+            /// <summary>
+            /// Change <see cref="Named.Name"/> and <see cref="ExpandedName"/>.
+            /// </summary>
+            Expanded = 1,
+
+            /// <summary>
+            /// Change <see cref="Named.Name"/> and <see cref="ResolvedPath"/>.
+            /// </summary>
+            Resolved = 2,
+
+            /// <summary>
+            /// Change only <see cref="Named.Name"/>.
+            /// </summary>
+            Final = 3,
+        }
+
+        public Ref WithName(string name, NameKind kind)
+        {
+            return new Ref(Info, name)
+            {
+                OriginalName = OriginalName,
+                ExpandedName = kind == NameKind.Expanded ? name : ExpandedName,
+                ResolvedPath = kind == NameKind.Resolved ? name : ResolvedPath,
+            };
+        }
+
+        /// <summary>
+        /// Resolves the path relative to the source file's directory.
+        /// </summary>
+        public Ref EnsureResolvedPath(ErrorReporter errorReporter)
+        {
+            var sourcePath = Info.SourceFile.Path;
+            var sourceDirectory = Path.GetDirectoryName(sourcePath)
+                ?? throw new InvalidOperationException($"Source file path '{sourcePath}' does not have a containing directory.");
+
+            var resolvedFilePath = Path.GetFullPath(Path.Combine(sourceDirectory, Name.Replace('\\', '/')));
+
+            if (!File.Exists(resolvedFilePath))
+            {
+                errorReporter(Info.SourceFile.Text, sourcePath, Info.Span,
+                    string.Format(FileBasedProgramsResources.InvalidRefDirective,
+                        string.Format(FileBasedProgramsResources.CouldNotFindRefFile, resolvedFilePath)));
+            }
+
+            return new Ref(Info, Name)
+            {
+                OriginalName = OriginalName,
+                ExpandedName = ExpandedName,
+                ResolvedPath = resolvedFilePath,
+            };
+        }
+
+        public override string ToString() => $"#:ref {Name}";
+    }
+
     public enum IncludeOrExcludeKind
     {
         Include,

src/Cli/Microsoft.DotNet.FileBasedPrograms/InternalAPI.Unshipped.txt

@@ -3,6 +3,7 @@ const Microsoft.DotNet.FileBasedPrograms.CSharpDirective.IncludeOrExclude.Experi
 const Microsoft.DotNet.FileBasedPrograms.CSharpDirective.IncludeOrExclude.ExperimentalFileBasedProgramEnableItemMapping = "ExperimentalFileBasedProgramEnableItemMapping" -> string!
 const Microsoft.DotNet.FileBasedPrograms.CSharpDirective.IncludeOrExclude.ExperimentalFileBasedProgramEnableTransitiveDirectives = "ExperimentalFileBasedProgramEnableTransitiveDirectives" -> string!
 const Microsoft.DotNet.FileBasedPrograms.CSharpDirective.IncludeOrExclude.MappingPropertyName = "FileBasedProgramsItemMapping" -> string!
+const Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.ExperimentalFileBasedProgramEnableRefDirective = "ExperimentalFileBasedProgramEnableRefDirective" -> string!
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective.CSharpDirective(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseInfo info) -> void
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective.IncludeOrExclude
@@ -71,6 +72,20 @@ Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Property
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Property.Property(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseInfo info) -> void
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Property.Value.get -> string!
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Property.Value.init -> void
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.EnsureResolvedPath(Microsoft.DotNet.FileBasedPrograms.ErrorReporter! errorReporter) -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref!
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.ExpandedName.get -> string?
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.ExpandedName.init -> void
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.NameKind
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.NameKind.Expanded = 1 -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.NameKind
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.NameKind.Final = 3 -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.NameKind
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.NameKind.Resolved = 2 -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.NameKind
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.OriginalName.get -> string!
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.OriginalName.init -> void
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.Ref(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseInfo info, string! name) -> void
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.ResolvedPath.get -> string?
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.ResolvedPath.init -> void
+Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.WithName(string! name, Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.NameKind kind) -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref!
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Sdk
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Sdk.Sdk(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseInfo info) -> void
 Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Sdk.Version.get -> string?
@@ -123,6 +138,7 @@ override Microsoft.DotNet.FileBasedPrograms.CSharpDirective.IncludeOrExclude.ToS
 override Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Package.ToString() -> string!
 override Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Project.ToString() -> string!
 override Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Property.ToString() -> string!
+override Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.ToString() -> string!
 override Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Sdk.ToString() -> string!
 override Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Shebang.ToString() -> string!
 override Microsoft.DotNet.FileBasedPrograms.SourceFile.GetHashCode() -> int
@@ -134,6 +150,7 @@ static Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Package.Parse(in Micro
 static Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Parse(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseContext context) -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Named?
 static Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Project.Parse(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseContext context) -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Project?
 static Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Property.Parse(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseContext context) -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Property?
+static Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref.Parse(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseContext context) -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Ref?
 static Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Sdk.Parse(in Microsoft.DotNet.FileBasedPrograms.CSharpDirective.ParseContext context) -> Microsoft.DotNet.FileBasedPrograms.CSharpDirective.Sdk?
 static Microsoft.DotNet.FileBasedPrograms.ErrorReporters.CreateCollectingReporter(out System.Collections.Immutable.ImmutableArray<Microsoft.DotNet.FileBasedPrograms.SimpleDiagnostic!>.Builder! builder) -> Microsoft.DotNet.FileBasedPrograms.ErrorReporter!
 static Microsoft.DotNet.FileBasedPrograms.ExternalHelpers.CombineHashCodes(int value1, int value2) -> int