Skip to content

Commit 8c27269

Browse files
javiercnjaviercn
authored
[Infrastructure] Move back to NPM (#52914)
This change removes Yarn from our infrastructure completely and replaces it with NPM. There is a package.json in the top folder of the respository that defines the workspace. The workspace lists all the paths to packages that are part of the repository. In addition to that, the top package.json defines scripts that can be run from the root of the repository to install, build test, etc. all the packages in the repository. Each package has its own package.json file that defines its dependencies and scripts. The scripts in the top package.json file are just wrappers around the scripts in the individual package.json files. ## Developer workflow ### Restore dependencies To install the dependencies, you can run npm ci from the root of the repository. This will install all the dependencies for all the packages in the repository and automatically link them together. This normally takes very little time once you've have downloaded the packages onto your machine (10s for me). ### Install or update new dependencies Run npm install ... on whatever node you want to install the dependencies. The package-lock.json will get updated accordingly. For installing new dependencies you might need to authenticate yourself with the NPM VSTS feed. To do that, run: * `npm install -g vsts-npm-auth` * `vsts-npm-auth -config .npmrc -F`. This will walk you through the auth process and provision the required credentials. * `npm install` ### Updating deps to fix Component Governance alerts * Run npm audit to see the list of issues. * Run npm audit fix to see if the issues can get addressed automatically. * If that's not the case: * If possible, update to a newer version of the package that doesn't include the vulnerable dependency. * If that's not possible, at the appropriate node, add an entry to the "overrides" property to force the transitive dependency into a non-vulnerable version. * run npm install from the root. * run npm audit again to see if the issues have disappeared, if they haven't continue making changes (update more packages, etc). Upgrading the main deps should always be the preferred route over using overrides. Overrides require that we periodically remove them to check if we can update the dependency later on. ### Building JS components You can run `npm run build` at the repo root and it will build all the projects (takes about 2 minutes). Or alternatively, you can run `npm run build` on an individual project.json to build only that `project.json`. If you do this, any dependent project should have been built previously. Dependencies won't get build automatically. ### Testing JS components There are two test categories: * Unit tests: They are self-contained and don't depend on any of the .NET Code. Run `npm run test`. * Integration tests: They test the JS in combination with the .NET Code, for which you first need to have run `eng\Build.cmd`, before you invoke the tests. To execute them, run `npm run integration-test`
1 parent 85e457c commit 8c27269

File tree

93 files changed

+20049
-23633
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

93 files changed

+20049
-23633
lines changed

‎.azure/pipelines/ci.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -750,9 +750,9 @@ stages:
750750
binlogPath: artifacts/log/Release/Build.binlog
751751
presteps:
752752
- task: NodeTool@0
753-
displayName: Install Node 18.x
753+
displayName: Install Node 20.x
754754
inputs:
755-
versionSpec: 18.x
755+
versionSpec: 20.x
756756
pool:
757757
name: $(DncEngInternalBuildPool)
758758
demands: ImageOverride -equals 1es-windows-2022

‎.azure/pipelines/components-e2e-tests.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,10 @@ jobs:
4141
displayName: Update submodules
4242
- script: ./restore.sh
4343
displayName: Run restore.sh
44-
- script: yarn install --frozen-lockfile --cwd ./src/Components/test/E2ETest || yarn install --frozen-lockfile --cwd ./src/Components/test/E2ETest
44+
- script: npm ci
4545
displayName: NPM install
46+
- script: npm run build
47+
displayName: Build JS
4648
- script: .dotnet/dotnet build ./src/Components/test/E2ETest -c $(BuildConfiguration) --no-restore
4749
displayName: Build
4850
- script: .dotnet/dotnet test ./src/Components/test/E2ETest -c $(BuildConfiguration) --no-build --filter 'Quarantined!=true|Quarantined=false'

‎.azure/pipelines/jobs/default-build.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -199,9 +199,9 @@ jobs:
199199
displayName: Start background dump collection
200200
- ${{ if eq(parameters.installNodeJs, 'true') }}:
201201
- task: NodeTool@0
202-
displayName: Install Node 18.x
202+
displayName: Install Node 20.x
203203
inputs:
204-
versionSpec: 18.x
204+
versionSpec: 20.x
205205
- ${{ if and(eq(parameters.installJdk, 'true'), eq(parameters.agentOs, 'Windows')) }}:
206206
- powershell: ./eng/scripts/InstallJdk.ps1
207207
displayName: Install JDK 11

‎Directory.Build.props‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -247,7 +247,6 @@
247247
<Import Project="eng\targets\Cpp.Common.props" Condition="'$(MSBuildProjectExtension)' == '.vcxproj'" />
248248
<Import Project="eng\targets\CSharp.Common.props" Condition="'$(MSBuildProjectExtension)' == '.csproj'" />
249249
<Import Project="eng\targets\Wix.Common.props" Condition="'$(MSBuildProjectExtension)' == '.wixproj'" />
250-
<Import Project="eng\targets\Npm.Common.props" Condition="'$(MSBuildProjectExtension)' == '.npmproj'" />
251250
<Import Project="eng\targets\Java.Common.props" Condition="'$(MSBuildProjectExtension)' == '.javaproj'" />
252251
<Import Project="eng\testing\linker\trimmingTests.props" Condition="'$(IsPublishedAppTestProject)' == 'true'" />
253252
<Import Project="eng\targets\Helix.props" Condition=" $(IsTestProject) " />

‎Directory.Build.targets‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,6 @@
179179
<Import Project="eng\targets\CSharp.Common.targets" Condition="'$(MSBuildProjectExtension)' == '.csproj'" />
180180
<Import Project="eng\targets\FSharp.Common.targets" Condition="'$(MSBuildProjectExtension)' == '.fsproj'" />
181181
<Import Project="eng\targets\Wix.Common.targets" Condition="'$(MSBuildProjectExtension)' == '.wixproj'" />
182-
<Import Project="eng\targets\Npm.Common.targets" Condition="'$(MSBuildProjectExtension)' == '.npmproj'" />
183182
<Import Project="eng\targets\Java.Common.targets" Condition="'$(MSBuildProjectExtension)' == '.javaproj'" />
184183
<Import Project="eng\testing\linker\trimmingTests.targets" Condition="'$(IsPublishedAppTestProject)' == 'true'" />
185184
<Import Project="eng\targets\Helix.targets" Condition=" $(IsTestProject) " />

‎eng/Build.props‎

Lines changed: 9 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -63,10 +63,10 @@
6363
<ItemGroup>
6464
<ProjectToBuild Include="$(ProjectToBuild)"
6565
Exclude="@(ProjectToExclude);$(RepoRoot)**\bin\**\*;$(RepoRoot)**\obj\**\*">
66-
<BuildInParallel Condition=" '%(Extension)' == '.npmproj' OR '%(Extension)' == '.vcxproj' ">false</BuildInParallel>
67-
<RestoreInParallel Condition=" '%(Extension)' == '.npmproj' ">false</RestoreInParallel>
66+
<BuildInParallel Condition=" '%(Extension)' == '.nodeproj' OR '%(Extension)' == '.vcxproj' ">false</BuildInParallel>
67+
<RestoreInParallel Condition=" '%(Extension)' == '.nodeproj' ">false</RestoreInParallel>
6868
<!-- Also do not build in parallel w/in npm projects. -->
69-
<AdditionalProperties Condition=" '%(Extension)' == '.npmproj' OR '%(Extension)' == '.vcxproj' ">BuildInParallel=false</AdditionalProperties>
69+
<AdditionalProperties Condition=" '%(Extension)' == '.nodeproj' OR '%(Extension)' == '.vcxproj' ">BuildInParallel=false</AdditionalProperties>
7070
</ProjectToBuild>
7171
</ItemGroup>
7272
</When>
@@ -124,25 +124,19 @@
124124
<ProjectToBuild Condition=" $(BuildNative) " Include="@(NativeProjects)" Exclude="@(ProjectToExclude)" />
125125
<ProjectToExclude Condition=" !$(BuildNative) " Include="@(NativeProjects)" />
126126

127-
<NodeJsProjects Include="
128-
$(RepoRoot)src\Components\Web.JS\Microsoft.AspNetCore.Components.Web.JS.npmproj;
129-
$(RepoRoot)src\SignalR\**\*.npmproj;
130-
$(RepoRoot)src\JSInterop\**\*.npmproj;
131-
"
127+
<NodeJsProjects Condition="'$(TargetArchitecture)' == 'x64'"
128+
Include="$(RepoRoot)eng\Npm.Workspace.nodeproj;
129+
$(RepoRoot)eng\Npm.Workspace.FunctionalTests.nodeproj;"
132130
AdditionalProperties="BuildInParallel=false"
133131
BuildInParallel="false"
134132
RestoreInParallel="false"
135133
Exclude="@(ProjectToExclude)" />
136134

135+
<ExplicitRequiresDelay Include="$(RepoRoot)eng\Npm.Workspace.FunctionalTests.nodeproj" />
136+
137137
<ProjectToBuild Condition=" '$(BuildNodeJS)' == 'true'" Include="@(NodeJsProjects)" Exclude="@(ProjectToExclude)" />
138138
<ProjectToExclude Condition=" '$(BuildNodeJS)' != 'true'" Include="@(NodeJsProjects)" />
139139

140-
<YarnMSBuildProjects Include="
141-
$(RepoRoot)src\Components\test\E2ETest\Microsoft.AspNetCore.Components.E2ETests.csproj;
142-
$(RepoRoot)src\Components\WebAssembly\Authentication.Msal\src\Microsoft.Authentication.WebAssembly.Msal.csproj;
143-
$(RepoRoot)src\Components\WebAssembly\WebAssembly.Authentication\src\Microsoft.AspNetCore.Components.WebAssembly.Authentication.csproj;
144-
"
145-
Exclude="@(ProjectToExclude)" />
146140
<ProjectToExclude Condition=" '$(DotNetBuildFromSource)' == 'true'" Include="@(YarnMSBuildProjects)" />
147141

148142
<JavaProjects Include="$(RepoRoot)src\SignalR\**\*.javaproj"
@@ -253,4 +247,5 @@
253247
</ItemGroup>
254248
</Otherwise>
255249
</Choose>
250+
256251
</Project>

‎eng/CodeGen.proj‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
<Target Name="GenerateProjectList">
1212
<Message Importance="High" Text="Analyzing @(ProjectToBuild->Count()) projects" />
1313

14-
<MSBuild Projects="@(ProjectToBuild)"
14+
<MSBuild Projects="@(ProjectToBuild);@(ExplicitRequiresDelay)"
1515
Targets="GetReferencesProvided"
1616
BuildInParallel="true"
1717
SkipNonexistentTargets="true"

‎eng/Npm.Workspace.FunctionalTests.nodeproj‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
<Project>
2+
3+
<!-- Import Directory.Build.Props -->
4+
<Import Project="$(MSBuildThisFileDirectory)..\Directory.Build.props" />
5+
6+
<PropertyGroup>
7+
<IsTestProject>true</IsTestProject>
8+
<IsUnitTestProject>true</IsUnitTestProject>
9+
<IsIntegrationTestProject>true</IsIntegrationTestProject>
10+
<TestDependsOnAspNetPackages>false</TestDependsOnAspNetPackages>
11+
<TestDependsOnAspNetAppPackages>false</TestDependsOnAspNetAppPackages>
12+
<TestDependsOnAspNetRuntime>false</TestDependsOnAspNetRuntime>
13+
<ContainsFunctionalTestAssets>false</ContainsFunctionalTestAssets>
14+
<BuildHelixPayload>false</BuildHelixPayload>
15+
<PackageVersion>$(VersionPrefix)$(VersionSuffix)</PackageVersion>
16+
<RequiresDelayedBuild>true</RequiresDelayedBuild>
17+
<TestGroupName>FunctionalTests</TestGroupName>
18+
<RuntimeIdentifier Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">win-x86</RuntimeIdentifier>
19+
</PropertyGroup>
20+
21+
<Target Name="Restore" />
22+
<Target Name="Build" />
23+
<Target Name="Pack" />
24+
<Target Name="Publish" />
25+
26+
<Target Name="_Test" BeforeTargets="Test">
27+
<Message Text="Testing NPM packages..." Importance="high" />
28+
<Exec Command="npm run integration-test" WorkingDirectory="$(MSBuildThisFileDirectory).." />
29+
</Target>
30+
31+
<Target Name="GetReferencesProvided" Returns="@(ProvidesReference)">
32+
<ItemGroup>
33+
<!-- Include File name without the extension -->
34+
<ProvidesReference Include="Npm.Workspace.FunctionalTests">
35+
<IsAspNetCoreApp>$([MSBuild]::ValueOrDefault($(IsAspNetCoreApp),'false'))</IsAspNetCoreApp>
36+
<IsPackable>$([MSBuild]::ValueOrDefault($(IsPackable),'false'))</IsPackable>
37+
<ProjectFileRelativePath>$([MSBuild]::MakeRelative($(RepoRoot), $(MSBuildProjectFullPath)))</ProjectFileRelativePath>
38+
<IsTrimmable>$([MSBuild]::ValueOrDefault($(IsTrimmable),'false'))</IsTrimmable>
39+
<IsShippingPackage>$([MSBuild]::ValueOrDefault($(IsShippingPackage),'false'))</IsShippingPackage>
40+
41+
<!-- True if the project may be referenced using a @(Reference) item. -->
42+
<IsProjectReferenceProvider>$([MSBuild]::ValueOrDefault($(IsProjectReferenceProvider),'false'))</IsProjectReferenceProvider>
43+
44+
<!-- True if project must be restored etc. after App.Ref and App.Runtime are fully built. -->
45+
<RequiresDelayedBuild>$([MSBuild]::ValueOrDefault($(RequiresDelayedBuild),'false'))</RequiresDelayedBuild>
46+
</ProvidesReference>
47+
</ItemGroup>
48+
</Target>
49+
50+
<!-- Import Directory.Build.targets -->
51+
<Import Project="$(MSBuildThisFileDirectory)..\Directory.Build.targets" />
52+
53+
</Project>

‎eng/Npm.Workspace.nodeproj‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
<Project>
2+
3+
<!-- Import Directory.Build.Props -->
4+
<Import Project="$(MSBuildThisFileDirectory)..\Directory.Build.props" />
5+
6+
<PropertyGroup>
7+
<IsTestProject>true</IsTestProject>
8+
<IsUnitTestProject>true</IsUnitTestProject>
9+
<TestDependsOnAspNetPackages>false</TestDependsOnAspNetPackages>
10+
<TestDependsOnAspNetAppPackages>false</TestDependsOnAspNetAppPackages>
11+
<TestDependsOnAspNetRuntime>false</TestDependsOnAspNetRuntime>
12+
<ContainsFunctionalTestAssets>false</ContainsFunctionalTestAssets>
13+
<BuildHelixPayload>false</BuildHelixPayload>
14+
<PackageVersion>$(VersionPrefix)$(VersionSuffix)</PackageVersion>
15+
</PropertyGroup>
16+
17+
<Target Name="Restore">
18+
<Message Text="Restoring NPM packages..." Importance="high" />
19+
<Exec Command="npm ci" WorkingDirectory="$(MSBuildThisFileDirectory).." />
20+
</Target>
21+
22+
<Target Name="Build">
23+
<Message Text="Building NPM packages..." Importance="high" />
24+
<Exec Command="npm run build" WorkingDirectory="$(MSBuildThisFileDirectory).." />
25+
</Target>
26+
27+
<Target Name="_Test" BeforeTargets="Test">
28+
<Message Text="Testing NPM packages..." Importance="high" />
29+
<Exec Command="npm run test" ContinueOnError="true" WorkingDirectory="$(MSBuildThisFileDirectory).." />
30+
</Target>
31+
32+
<Target Name="Pack">
33+
<Message Text="Packing NPM packages..." Importance="high" />
34+
<MakeDir Directories="$(PackageOutputPath)" Condition="!Exists('$(PackageOutputPath)')" />
35+
<Exec Command="node $(MSBuildThisFileDirectory)scripts\pack-workspace.js $(RepoRoot)package.json $(PackageVersion) $(PackageOutputPath)" />
36+
</Target>
37+
38+
<!-- Import Directory.Build.targets -->
39+
<Import Project="$(MSBuildThisFileDirectory)..\Directory.Build.targets" />
40+
41+
</Project>

‎eng/Publishing.props‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -43,11 +43,11 @@
4343
DependsOnTargets="_WriteProductVersionFile">
4444
<!--
4545
This target is defined in eng/targets/Packaging.targets and Npm.Common.targets and included in every C#, F#,
46-
and npm project. We use SignalR.Npm.FunctionalTests.npmproj because it is non-shipping (we need a non-stable
46+
and npm project. We use SignalR.Npm.FunctionalTests.nodeproj because it is non-shipping (we need a non-stable
4747
version string to use as our publish location), non-packed (won't be shipped in the future), and it is _not_ a
4848
C# or F# project. For now at least, C# and F# projects should not be referenced when using desktop msbuild.
4949
-->
50-
<MSBuild Projects="$(RepoRoot)src\SignalR\clients\ts\FunctionalTests\SignalR.Npm.FunctionalTests.npmproj"
50+
<MSBuild Projects="$(RepoRoot)src\SignalR\clients\ts\FunctionalTests\SignalR.Npm.FunctionalTests.nodeproj"
5151
Properties="DisableYarnCheck=true;ExcludeFromBuild=false"
5252
Targets="_GetPackageVersionInfo">
5353
<Output TaskParameter="TargetOutputs" ItemName="_ResolvedPackageVersionInfo" />
@@ -102,11 +102,11 @@
102102
Condition=" '$(PublishInstallerBaseVersion)' == 'true'">
103103
<!--
104104
This target is defined in eng/targets/Packaging.targets and Npm.Common.targets and included in every C#, F#,
105-
and npm project. We use Microsoft.JSInterop.JS.npmproj because it is shipping (we need a stable
105+
and npm project. We use Microsoft.JSInterop.JS.nodeproj because it is shipping (we need a stable
106106
version string to use for productVersion.txt), and because it won't break when the SDK requires a newer
107107
desktop MSBuild than exists on the build machine.
108108
-->
109-
<MSBuild Projects="$(RepoRoot)src\JSInterop\Microsoft.JSInterop.JS\src\Microsoft.JSInterop.JS.npmproj"
109+
<MSBuild Projects="$(RepoRoot)src\JSInterop\Microsoft.JSInterop.JS\src\Microsoft.JSInterop.JS.nodeproj"
110110
Properties="DisableYarnCheck=true;ExcludeFromBuild=false"
111111
Targets="_GetPackageVersionInfo">
112112
<Output TaskParameter="TargetOutputs" ItemName="_ResolvedProductVersionInfo" />

0 commit comments

Comments
 (0)