[infra] Shard dart2js modular tests

* Shard dart2js modular tests (and decrease the shards of the
  unit tests)
* Remove folders we look for dart files in --- the remove folders
  doesn't contain any anyway and would have to be copied if keeping
  these lines.
* Fix sharding, previously trying to shard in 2 shards would only allow
  you to run ~50% of the tests:

```
$ out/ReleaseX64/dart-sdk/bin/dart pkg/compiler/tool/modular_test_suite.dart -nweb-unittest-asserts-linux --verbose --use-sdk --shards=2 --shard=0
Error: shard should be between 0 and 1, but got 0

$ out/ReleaseX64/dart-sdk/bin/dart pkg/compiler/tool/modular_test_suite.dart -nweb-unittest-asserts-linux --verbose --use-sdk --shards=2 --shard=2
Error: shard should be between 0 and 1, but got 2
```

  This has been corrected to allow from `1..n` for `n` shards to fit
  with what the testing system sends when specifying `shards` in
  `tools/bots/test_matrix.json`.

For previous try runs I extracted this:

Build 60615:
Shard #1: --- Total time: 04:27 ---
Shard #2: --- Total time: 05:19 ---
Shard #3: --- Total time: 04:23 ---
Shard #4: --- Total time: 10:29 ---
 => A total of 24:35 --- combined finish of 10:29
Modular tests: 32:19
Total bot runtime: 35:48

Build 60614:
Shard #1: --- Total time: 03:57 ---
Shard #2: --- Total time: 05:18 ---
Shard #3: --- Total time: 05:21 ---
Shard #4: --- Total time: 05:34 ---
 => A total of 20:10 --- combined finish of 5:34
Modular tests: 29:41 secs
Total bot runtime: 35:55

Build: 60613
Shard #1: --- Total time: 03:56 ---
Shard #2: --- Total time: 05:15 ---
Shard #3: --- Total time: 04:32 ---
Shard #4: --- Total time: 05:34 ---
 => A total of 19:17 --- combined finish of 5:34
Modular tests: 33:39
Total bot runtime: 38:51

With the new sharding I'd estimate that the unit tests and modular tests
would have finished in less than 13 minutes, making the bots finish in
~17 minutes, ~20 minutes and ~19 minutes instead.
The try-run with this ran in 17:06

Possibly a follow-up could do more stuff on the "main bot".

Change-Id: Ie5c96206deb9c0c6db3385bbca04ae6f4eab4c3a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/448381
Reviewed-by: Nate Biggs <[email protected]>
Reviewed-by: Ivan Inozemtsev <[email protected]>
Commit-Queue: Jens Johansen <[email protected]>

Author: jensjoha

pkg/modular_test/lib/src/loader.dart

@@ -193,8 +193,6 @@ Future<Module> _createSdkModule(Uri root) async {
   // it doesn't list files that are transitively imported.
   var sdkLibrariesAndPatchesRoots = [
     'sdk/lib/',
-    'runtime/lib/',
-    'runtime/bin/',
   ];
   for (var path in sdkLibrariesAndPatchesRoots) {
     var dir = Directory.fromUri(root.resolve(path));

pkg/modular_test/lib/src/runner.dart

@@ -87,8 +87,8 @@ class Options {
           help: 'total number of shards a suite is going to be split into.',
           defaultsTo: '1')
       ..addOption('shard',
-          help: 'which shard this script is executing. This should be between 0'
-              ' and `shards - 1`.')
+          help: 'which shard this script is executing. This should be between 1'
+              ' and `shards`.')
       ..addOption('output-directory',
           help: 'location where to emit the jsonl result and log files')
       ..addOption('named-configuration',
@@ -99,8 +99,8 @@ class Options {
     int shard = 1;
     if (shards > 1) {
       shard = int.tryParse(argResults['shard']) ?? 1;
-      if (shard <= 0 || shard >= shards) {
-        print('Error: shard should be between 0 and ${shards - 1},'
+      if (shard <= 0 || shard > shards) {
+        print('Error: shard should be between 1 and $shards,'
             ' but got $shard');
         exit(1);
       }
@@ -112,7 +112,8 @@ class Options {
       ..useSdk = argResults['use-sdk']
       ..filter = argResults['filter']
       ..shards = shards
-      ..shard = shard
+      // Turn shard from [1..shards] into [0..shards-1]
+      ..shard = shard - 1
       ..configurationName = argResults['named-configuration']
       ..outputDirectory = toUri(argResults['output-directory']);
   }

tools/bots/test_matrix.json

@@ -34,6 +34,7 @@
       "tests/ffi/",
       "tests/language/",
       "tests/lib/",
+      "tests/modular/",
       "tests/web/",
       "third_party/d8/",
       "third_party/pkg/",
@@ -2493,7 +2494,7 @@
             "-nweb-unittest-asserts-linux",
             "pkg//compiler/"
           ],
-          "shards": 4,
+          "shards": 2,
           "fileset": "js_platform"
         },
         {
@@ -2505,7 +2506,9 @@
             "-nweb-unittest-asserts-linux",
             "--verbose",
             "--use-sdk"
-          ]
+          ],
+          "shards": 3,
+          "fileset": "js_platform"
         },
         {
           "name": "js_runtime unit tests",