[Proposal] Ignored Directives Support

[333fred]

Ignored Directives

• Proposed: https://github.com/dotnet/csharplang/blob/main/proposals/ignored-directives.md
• Prototype
• Implementation
• Specification

Summary

As we move to allow C# statements in the top level of a file, it's reasonable to assume that we might someday get support for dotnet runing a .cs file. For further use of C# as a scripting language in this vein, I propose that we introduce a new preprocessor directive who's purpose is to exist solely to be ignored by the language: the hashbang directive. Linux and Unix scripts will often use #!<path to interpreter> in order to direct the program loader to the interpreter that can run them. We should add a similar ignore to the C# language.

Detailed design

We introduce a new directive:

pp_directive
    : ... // Existing directives
    | pp_hashbang
    | pp_reference
    ;

pp_hashbang
    : whitespace? '#' '!' input_character*
    ;

pp_reference
    : whitespace? '#' 'r' whitespace input_character*
    ;

The language ignores the contents of the hashbang and reference directives and moves to the next line.

Drawbacks

Any change adds complication to the language. If we implement support for this and it starts appearing in files without actual scripting support, then it could be confusing for users expecting it to work.

Alternatives

N/A

Unresolved questions

1. If we use these directives to support tooling such as nuget references, we will need to consider how they interact with other pragmas, such as #if. We have a few options for this:
   1. Require that ignored directives are the first directives in a file. We could even go further and require they're all at the beginning of the file.
   2. Less restrictive than 1, but forbid ignored directives inside conditional directives.
   3. Least restrictive, require a true preprocessing step in the compiler. Today, we can process directives up front without having to do a full multi-step preprocessing pass before lexing, but if we want to support nuget directives inside conditional directives we may need the full two-step process.

Design meetings

• https://github.com/dotnet/csharplang/blob/main/meetings/2021/LDM-2021-05-12.md#simple-c-programs
• https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-08-31.md#ignored-directives-support
• https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-09-26.md#ungrouped

[ufcpp]

I'm worried about BOM. The presence of the BOM before the shebang will prevent the script interpreter.

Visual Studio for Win uses Encoding.Default (usually not UTF8) if the BOM doesn't present.

[333fred]

Sure, and we could certainly consider warning if the file uses anything other than UTF-8 w/o BOM.

[yaakov-h]

I thought shebang was already supported... is it only in the .csx variant?

[yaakov-h]

ah, looks like its only currently supported in the scripting mode.

https://github.com/dotnet/roslyn/blob/1b266a1e67434578ec09737a4e9427e52f44a420/src/Compilers/CSharp/Portable/Parser/DirectiveParser.cs#L108-L111

[svick]

1. As @yaakov-h mentions, shebang is already supported in the scripting dialect. Is this effectively formalizing that support and making it work in both dialects?
2. If I understand it correctly, the shebang that's supported by the scripting mode has to be literally the first thing in the file: no whitespace before it, no other lines before it. Since that's the only way shebang works, does it make sense to support whitespace before it and to support it on lines other than the first line in the file?

[HaloFour]

I'm a little confused. Do "simple programs" not require a project file? How would a single file "script" in this manner establish references or change any compiler properties?

This feels like trying to bridge a gap between C# "simple programs" and CSX, which already has this support plus the directive support to make it actually useful. Short of adding that directive support this feels extremely limited.

Again, C# would benefit so much more from embracing CSX and improving the tooling support than it ever will by chasing CSX through some separate dialect that will always be much less capable.

[dersia]

I agree with @HaloFour without directives for assembly imports this might not be very useful. On the other hand having a script that downloads nuget packages or loading complex and somewhat big assemblies doesn't feel right for scripting.

The only thing that I kind of see as a big plus is that you can just take the cs file to any machine architecture and call dotnet run on it so you don't have to compile aot.

What is still unclear for me is if the purpose of this proposal is that the cs-file is being interpreted or is it going to be compiled on running it.

If the purpose is to "just" interpreted on run, than I agree even more with @HaloFour we should focus on making csx a better experience.

[hez2010]

I don't think it should introduce a new #! directive to C#, for C# doesn't use # for comment syntax.

How about:

/// <interpreter path="" />

Which will allow more complex definition, for example:

/// <project target="net5.0">
///   <properties>
///     <nullable>enable</nullable>
///   </properties>
///   <items>
///     <package include="Newtonsoft.Json" version="12.0.3" />
///   </items>
/// </project>

[HaloFour]

The point is to use a syntax that existing shells already recognize, otherwise there would be no benefit. Unix shells recognize shebang syntax and it is widely used to enable a script file to instruct the shell in how that file is to be interpreted. No shell will understand a C# comment or a blob of XML.

…

On Thu, May 28, 2020, 10:31 PM Steve ***@***.***> wrote: How about: /// <interpreter path="" /> I don't think it should introduce a new #! directive to C#, for C# doesn't use # for comment syntax. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3507 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACNCMEWBCWZ6OAC346MHQHTRT4M6VANCNFSM4NMYL2BQ> .

[aka-nse]

I summarize my understanding of this proposal at followings. Is it right?

• C# compiler will simply ignore a shebang in the first line of the source. Because the lines start with # are preprocessor directives even on conventional spec, this modification looks not to impact other features.

• You can write inline csproj XML on and after 2nd line, but the interpretation of them should be done by build tool (e.g. dotnet-cli), not compiler. Such an inline csproj will be written under the conventional comment syntax, and the extension spec will be defined by tools.

#!/bin/dotnet run
//+ <Project Sdk="Microsoft.NET.Sdk">
//+     <PropertyGroup>
//+         <TargetFramework>netcoreapp3.1</TargetFramework>
//+     </PropertyGroup>
//+ </Project>

/*
 * In this example, it is assumed that dotnet-cli will interprets the comments start with `//+` as inline csproj
 */

using System;

Console.WriteLine("Hello, world!");

[333fred]

The first point is correct, and is the only thing this proposal covers. This proposal makes no statement on how a program would communicate dependencies to a tool.

[rolfbjarne]

One of the design meetings says:

Would be nice to have, but only in conjunction with the rest of the simple programs feature. Implementation might be one of the simplest ever features in the compiler, but without the tooling to accompany it we don't know that this is the right design.

I'm not sure I agree, because this doesn't actually need official tooling to be useful, it's entirely possible to extend by anyone.

Consider this:

#!/bin/my-custom-program

using System;

Console.WriteLine("Hello, world!");

where /bin/my-custom-program can be a program that takes the script and does whatever it wants to - which could be creating a project file for the file and running dotnet run, but it an also be anything else.

One example could be:

#!/bin/run-unit-test

[TestFixture]
public class Test {
    [Test]
    public void IsValid (int number)
    {
        Assert.AreEqual (42, number, "everything");
    }
}

where run-unit-test would create a project, add the corresponding NUnit references and usings, compile & run the tests.

[CyrusNajmabadi]

it's entirely possible to extend by anyone.

The point is that without that tooling we don't know if this will be good enough to actually solve those cases.

[rolfbjarne]

it's entirely possible to extend by anyone.

The point is that without that tooling we don't know if this will be good enough to actually solve those cases.

I agree that the reference directive (#r) might be in this category, but the shebang directive (#!) can stand on its own and would be useful by itself, both without the reference directive, or any other tooling.

[CyrusNajmabadi]

it's entirely possible to extend by anyone.

The point is that without that tooling we don't know if this will be good enough to actually solve those cases.

I agree that the reference directive (#r) might be in this category, but the shebang directive (#!) can stand on its own and would be useful by itself, both without the reference directive, or any other tooling.

What is it's not good enough? What if we do it, and a month later people try to use it and say it's not sufficient and it needs to change or be redone.

We can't add features without some way of actually validating that it solves the problems we want it to solve. We need some actual tooling adoption to demonstrate it is sufficient.

[jkrejcha]

Yes they add source as a code object but this is 100% internal to the compilation process. There is no reaching out of the compilation and grabbing new artifacts off of disk. The source generator, in combination with the other compiler inputs, perfectly describes the inputs to the compilation.

There's no fundamental difference here.

This claim is simply incorrect.

I'm kinda unsure what you're thinking here, given that most references to pragma, in general, say something to the effect of it being implementation defined. Given that .NET uses it outside of diagnostic control as well, I'd say that tracks with usage of the term both in C# as well as other programming languages and tools like C, C++, SQLite, etc.

The language has nothing to say about it, just as the language has nothing to say about #region.

This is much more fundamentally different than region and even so, @HaloFour made a pretty good point regarding this which bears repeating

You could make that argument, and honestly if someone had suggested #region much more recently it'd probably be a tougher sell. But at least it does have a specific meaning and use case, plus a known syntax (e.g. #endregion required).

The language can't know what these other directives mean, or if they have a specific structure, unless the language becomes much more bound to this specific tooling. It can't even know what directives are possible, or might be possible in a future release. Supporting a pattern for those directives and treating them like comments is probably the safest bet. But at the same time is there a reason that the tooling must use new directives like this?

[jkrejcha]

(Regardless, the current proposal seems to have been changed. The current version (#: prefix) seems reasonable enough.)

[RokeJulianLockhart]

#discussioncomment-12497852

@jkrejcha, could you link to that? I'm having trouble locating it.

[jjonescz]

I've just merged the update in #9215. So the new version is at https://github.com/dotnet/csharplang/blob/main/proposals/ignored-directives.md

[HaloFour]

Here's the PR where they reversed the order to have a known preprocessor prefix that the compiler always ignores as the primary proposal where known directives could be an alternative:

#9215

A few alternative sigils to #: have also been proposed.

[jkrejcha]

To be honest, I don't really seem the advantage this has in the language over just using pragma since this is already implementation specific.

The idea of sdk and package is effectively specific only to the .NET implementation (the tooling)

[jkrejcha]

The only weird one is #! but even that could just be an extension used by CSX or something (it already has its own implementation with #r).

Interestingly of note, it does seem that .NET effectively just ignores compiler directives it doesn't know about. (See below)

[jjonescz]

it does seem that .NET effectively just ignores compiler directives it doesn't know about.

If by "ignore" you mean "warns about", that's true but only for pragma directives, not arbitrary directives, i.e.:

#pragma test // warning CS1633: Unrecognized #pragma directive
#test // error CS1024: Preprocessor directive expected

[jkrejcha]

Ah ya, I checked again and it seemed I had a different file open when I tested the latter. Whoops.

[egvijayanand]

I would suggest adding another directive along with the sdk, property, and package to reference other C# source files as mentioned below.

The proposed #:include directive would add multiple C# source files to the compilation unit.

The file provided as a parameter to the dotnet run command would be the entry point, and other files are supporting files.

The source file path can be mentioned as an absolute path or a relative path to the current file.

#:include Foo.cs

// Other statements