Skip to content

Commit 43d8ece

Browse files
authored
Refactor C# Generator to make adding source generators easier (#828)
This PR is a refactor of the C# Telemetry Generator. ## Problems 1. Some of the core fields ("global metadata properties") are generated types instead of primitive types (the "Result" property is one example). In order to add these into the base classes, we need the generated code. But the generated code is only applied during the package publishing automation. 2. The test classes awkwardly require the generator to be run, and the generated output is compiled into the test projects. This doesn't allow for a clean "build the repo", "test the repo" flow. Currently, we have a hack that force builds and runs the generator as a pre-build step for the test project. 3. There is a test that validates the generator output by placing the entire generated file in the repo. This file becomes very large, and falls out of date quickly, since most repo changes focus on updating the definitions json. Updates to this file are usually only checked in when changes are made to the generator. When this file is updated, the PR's change count is inflated by 17k+ lines. ## Approach We can take an _incremental_ approach to solving these problems, over a series of changes. The end goal is to use [C# Source Generators](https://learn.microsoft.com/en-us/dotnet/csharp/roslyn-sdk/source-generators-overview), which will auto-produce the generated code whenever the Events package is compiled. This allows us to solve problem 1 because the required generated types will be available within the `AwsToolkit.Telemetry.Events` package, even while it is being developed. Problems 2 and 3 will be solved, because we will be able to delete the large "GeneratedCode.cs" file, and exercise tests directly against `AwsToolkit.Telemetry.Events`, which will now contain the types that are currently only available after the package is produced and published. Before we introduce the source generator, the generation code is being moved to a "core" package, which will serve the CLI Generator and the (future) source generator. That is all that this change is doing. In follow-up changes, the source generator will be introduced. Once the source generator is available, the CLI generator will still be used for producing the product specific metrics that are defined in the Toolkit repo, but it will not be used for the purpose of producing `AwsToolkit.Telemetry.Events`. After the source generator is available, a follow-up change will define `Result` as a global property to all C# events. The motivation for making these changes incrementally is to rely on existing tests/CI/infrastructure to ensure the generator remains stable throughout the changes. Before: ```mermaid flowchart TB subgraph PkgEvents[Telemetry.Events package] direction TB eventsCodeDto["Core DTOs"] eventsCodeBase["Base Event class"] genCode["Generated code\n(based on telemetry definitions)"] end subgraph PkgEventsGen[Telemetry.Events.Generator package] direction TB gCodeDto["definitions json\nloading models"] gCodeBuilder["Code Builder"] gCli["Generator CLI\nProgram"] end PkgEventsGen -.Generates code\nwhen CLI is run;\nrequired for packaging.-> PkgEvents ``` After: ```mermaid flowchart TB subgraph PkgEventsGenCore[Telemetry.Events.Generator.Core package] direction TB gcCodeDto["definitions json\nloading models"] gcCodeBuilder["Code Builder"] end subgraph PkgEventsGen[Telemetry.Events.Generator package] direction TB gCli["Generator CLI\nProgram"] end subgraph PkgEventsSrcGen["Telemetry.Events.SourceGenerator package (FUTURE CHANGE)"] direction TB sgCode["Source Generator"] end style PkgEventsSrcGen stroke:#0000FF,stroke-dasharray: 5 5 subgraph PkgEvents[Telemetry.Events package] direction TB eventsCodeDto["Core DTOs"] eventsCodeBase["Base Event class"] genCode["Generated code\n(based on telemetry definitions)"] end PkgEventsGen -.Depends on.-> PkgEventsGenCore PkgEventsSrcGen -.Depends on.-> PkgEventsGenCore PkgEventsSrcGen -.Generates code\nwhen Events pkg\nis compiled.-> PkgEvents ```
1 parent cfc676d commit 43d8ece

25 files changed

+288
-130
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
3+
<PropertyGroup>
4+
<!-- Source generators must target netstandard 2.0, and this project will be a dependency. -->
5+
<TargetFramework>netstandard2.0</TargetFramework>
6+
7+
<RootNamespace>Amazon.AwsToolkit.Telemetry.Events.Generator.Core</RootNamespace>
8+
<AssemblyName>Amazon.AwsToolkit.Telemetry.Events.Generator.Core</AssemblyName>
9+
10+
</PropertyGroup>
11+
12+
<ItemGroup>
13+
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
14+
<PackageReference Include="System.CodeDom" Version="8.0.0" />
15+
</ItemGroup>
16+
17+
</Project>
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Core
2+
{
3+
public static class Constants
4+
{
5+
public const string DefaultEventsNamespace = "Amazon.AwsToolkit.Telemetry.Events.Generated";
6+
}
7+
}

telemetry/csharp/AwsToolkit.Telemetry.Events.Generator/DefinitionsBuilder.cs renamed to telemetry/csharp/AwsToolkit.Telemetry.Events.Generator.Core/DefinitionsBuilder.cs

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
1-
using System;
1+
using Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Models;
2+
using Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Utils;
3+
using System;
24
using System.CodeDom;
35
using System.CodeDom.Compiler;
46
using System.Collections.Generic;
57
using System.Globalization;
68
using System.IO;
79
using System.Linq;
810
using System.Reflection;
9-
using Amazon.AwsToolkit.Telemetry.Events.Generator.Models;
10-
using Amazon.AwsToolkit.Telemetry.Events.Generator.Utils;
1111

12-
namespace Amazon.AwsToolkit.Telemetry.Events.Generator
12+
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Core
1313
{
1414
/// <summary>
1515
/// Generates code that allows programs like the Toolkit to instantiate and publish Telemetry Events
@@ -112,9 +112,9 @@ public string Build()
112112
blankNamespace.Imports.Add(new CodeNamespaceImport("System.Collections.Generic"));
113113
// All generated code is expected to be placed in, or somewhere with a dependency on,
114114
// AwsToolkit.Telemetry.Events.Generated
115-
if (_namespace != Options.DefaultEventsNamespace)
115+
if (_namespace != Constants.DefaultEventsNamespace)
116116
{
117-
blankNamespace.Imports.Add(new CodeNamespaceImport(Options.DefaultEventsNamespace));
117+
blankNamespace.Imports.Add(new CodeNamespaceImport(Constants.DefaultEventsNamespace));
118118
}
119119
blankNamespace.Imports.Add(new CodeNamespaceImport("Amazon.AwsToolkit.Telemetry.Events.Core"));
120120

telemetry/csharp/AwsToolkit.Telemetry.Events.Generator/Models/Metadata.cs renamed to telemetry/csharp/AwsToolkit.Telemetry.Events.Generator.Core/Models/Metadata.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
// ReSharper disable InconsistentNaming
2-
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Models
2+
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Models
33
{
44
public class Metadata
55
{

telemetry/csharp/AwsToolkit.Telemetry.Events.Generator/Models/Metric.cs renamed to telemetry/csharp/AwsToolkit.Telemetry.Events.Generator.Core/Models/Metric.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
// ReSharper disable InconsistentNaming
2-
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Models
2+
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Models
33
{
44
public class Metric
55
{

telemetry/csharp/AwsToolkit.Telemetry.Events.Generator/Models/MetricType.cs renamed to telemetry/csharp/AwsToolkit.Telemetry.Events.Generator.Core/Models/MetricType.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
// ReSharper disable InconsistentNaming
2-
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Models
2+
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Models
33
{
44
public class MetricType
55
{

telemetry/csharp/AwsToolkit.Telemetry.Events.Generator/Models/TelemetryDefinitions.cs renamed to telemetry/csharp/AwsToolkit.Telemetry.Events.Generator.Core/Models/TelemetryDefinitions.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
using System.Linq;
44
using Newtonsoft.Json;
55

6-
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Models
6+
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Models
77
{
88
public class TelemetryDefinitions
99
{

telemetry/csharp/AwsToolkit.Telemetry.Events.Generator/Utils/MetricTypeExtensionMethods.cs renamed to telemetry/csharp/AwsToolkit.Telemetry.Events.Generator.Core/Utils/MetricTypeExtensionMethods.cs

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
using System;
1+
using Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Models;
2+
using System;
23
using System.Collections.Generic;
3-
using Amazon.AwsToolkit.Telemetry.Events.Generator.Models;
44

5-
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Utils
5+
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Utils
66
{
77
public static class MetricTypeExtensionMethods
88
{

telemetry/csharp/AwsToolkit.Telemetry.Events.Generator/Utils/StringExtensionMethods.cs renamed to telemetry/csharp/AwsToolkit.Telemetry.Events.Generator.Core/Utils/StringExtensionMethods.cs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
using System.Linq;
22

3-
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Utils
3+
namespace Amazon.AwsToolkit.Telemetry.Events.Generator.Core.Utils
44
{
55
public static class StringExtensionMethods
66
{
@@ -16,7 +16,7 @@ public static string ToPascalCase(this string text)
1616
return text.ToUpper();
1717
}
1818

19-
var segments = text.Split("_");
19+
var segments = text.Split('_');
2020
if (segments.Length > 1)
2121
{
2222
return string.Concat(segments.Select(ToPascalCase));

telemetry/csharp/AwsToolkit.Telemetry.Events.Generator.Tests/AwsToolkit.Telemetry.Events.Generator.Tests.csproj

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<Project Sdk="Microsoft.NET.Sdk">
1+
<Project Sdk="Microsoft.NET.Sdk">
22

33
<PropertyGroup>
44
<TargetFramework>net8.0</TargetFramework>
@@ -11,8 +11,8 @@
1111
</PropertyGroup>
1212

1313
<ItemGroup>
14+
<PackageReference Include="FluentAssertions" Version="6.12.0" />
1415
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.10.0" />
15-
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
1616
<PackageReference Include="xunit" Version="2.9.0" />
1717
<PackageReference Include="xunit.runner.visualstudio" Version="2.8.2">
1818
<PrivateAssets>all</PrivateAssets>
@@ -21,14 +21,26 @@
2121
</ItemGroup>
2222

2323
<ItemGroup>
24+
<None Include="..\SampleData\Inputs\*.json">
25+
<Link>SampleData\Inputs\%(Filename)%(Extension)</Link>
26+
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
27+
</None>
28+
<None Include="..\SampleData\Outcomes\*.txt">
29+
<Link>SampleData\Outcomes\%(Filename)%(Extension)</Link>
30+
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
31+
</None>
32+
</ItemGroup>
33+
34+
<ItemGroup>
35+
<ProjectReference Include="..\AwsToolkit.Telemetry.Events.Generator.Core\AwsToolkit.Telemetry.Events.Generator.Core.csproj" />
2436
<ProjectReference Include="..\AwsToolkit.Telemetry.Events.Generator\AwsToolkit.Telemetry.Events.Generator.csproj" />
2537
</ItemGroup>
2638

2739
<ItemGroup>
28-
<None Update="test-data\expectedCode*.txt">
40+
<None Update="SampleData\Outcomes\sampleDefinitions*.txt">
2941
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
3042
</None>
31-
<None Update="test-data\sampleDefinitions.json">
43+
<None Update="SampleData\Inputs\sampleDefinitions.json">
3244
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
3345
</None>
3446
</ItemGroup>

0 commit comments

Comments
 (0)