|
| 1 | +# Dependecy Tracking |
| 2 | + |
| 3 | +## [Experimental] Dependency options |
| 4 | + |
| 5 | +There are a number of dependency options which can be set in the scala toolchain. These include |
| 6 | +`dependency_mode`, `strict_deps_mode`, `unused_dependency_checker_mode`, and |
| 7 | +`dependency_tracking_method`. |
| 8 | + |
| 9 | +### [Experimental] Recommended options |
| 10 | + |
| 11 | +We recommend one of the following sets of options |
| 12 | + |
| 13 | +**Option A** |
| 14 | +Accept the defaults, which might work well enough for you. The defaults are |
| 15 | +``` |
| 16 | + dependency_mode = "direct", |
| 17 | + strict_deps_mode = "off", |
| 18 | + unused_dependency_checker_mode = "off", |
| 19 | + dependency_tracking_method = "high-level", |
| 20 | +``` |
| 21 | +but you do not need to include this in the toolchain as they are the defaults. |
| 22 | + |
| 23 | +**Option B** |
| 24 | +``` |
| 25 | + dependency_mode = "plus-one", |
| 26 | + strict_deps_mode = "error", |
| 27 | + unused_dependency_checker_mode = "error", |
| 28 | + dependency_tracking_method = "ast", |
| 29 | +``` |
| 30 | + |
| 31 | +Should the first option result in too much effort in handling build files and the like due to |
| 32 | +confusing dependencies and you becoming confused as to why some specific dependency is needed when |
| 33 | +the code being compiled never references it, consider this set of options. It will include both |
| 34 | +dependencies and dependencies of dependencies, which in practice is enough to stop almost all |
| 35 | +strange missing dependency errors at the cost of somewhat more incremental compile cost in |
| 36 | +certain cases. |
| 37 | + |
| 38 | +With these settings, we also will error on dependencies which are unneeded, and dependencies which |
| 39 | +should be included in `deps` due to be directly referenced in the code, but are not. |
| 40 | + |
| 41 | +The dependency tracking method `ast` is experimental but so far proves to be better than the default |
| 42 | +for computing the direct dependencies for `plus-one` mode code. In the future we hope to make this |
| 43 | +the default for `plus-one` mode and remove the option altogether. |
| 44 | + |
| 45 | +To try it out you can use the following toolchain: |
| 46 | +`@io_bazel_rules_scala//scala:minimal_direct_source_deps`. |
| 47 | + |
| 48 | +### [Experimental] Dependency mode |
| 49 | + |
| 50 | +There are three dependency modes. The reason for the multiple modes is that often `scalac` depends |
| 51 | +on jars which seem unnecessary at first glance. Hence, in order to reduce the need to please |
| 52 | +`scalac`, we provide the following options. |
| 53 | +- `dependency_mode = "direct"` - only include direct dependencies during compiliation; that is, |
| 54 | +those in the `deps` attribute |
| 55 | +- `dependency_mode = "plus-one"` - only include `deps` and `deps` of `deps` during compiliation. |
| 56 | +- `dependency_mode = "transitive"` - all transitive dependencies are included during compiliation. |
| 57 | +That is, `deps`, `deps` of `deps`, `deps` of `deps` of `deps`, and so on. |
| 58 | + |
| 59 | +Note when a dependency is included, that means its jars are included on the classpath, along with |
| 60 | +the jars of any targets that it exports. |
| 61 | + |
| 62 | +When using `direct` mode, there can be cryptic `scalac` errors when one mistakenly depends on a |
| 63 | +transitive dependency or, as more often the case for some, a transitive dependency is needed to |
| 64 | +[please scalac](https://github.com/scalacenter/advisoryboard/blob/master/proposals/009-improve-direct-dependency-experience.md) |
| 65 | +itself. |
| 66 | + |
| 67 | +As one goes down the list, more dependencies are included which helps reduce confusing requirements |
| 68 | +to add `deps`, at the cost of increased incremental builds due to a greater number of dependencies. |
| 69 | +In practice, using `plus-one` deps results in almost no confusing `deps` entries required while |
| 70 | +still being relatively small in terms of the number of total dependencies included. |
| 71 | + |
| 72 | +**Caveats for `plus_one` and `transitive`:** |
| 73 | +<ul> |
| 74 | + <li>Extra builds- Extra dependencies are inputs to the compilation action which means you can |
| 75 | + potentially have more build triggers for changes the cross the ijar boundary </li> |
| 76 | + <li>Label propagation- since label of targets are needed for the clear message and since it's |
| 77 | + not currently supported by JavaInfo from bazel we manually propagate it. This means that the |
| 78 | + error messages have a significantly lower grade if you don't use one of the scala rules or |
| 79 | + `scala_import` (since they don't propagate these labels)</li> |
| 80 | + <li>javac outputs incorrect targets due to a problem we're tracing down. Practically we've |
| 81 | + noticed it's pretty trivial to understand the correct target (i.e. it's almost a formatting |
| 82 | + problem) </li> |
| 83 | + </ul> |
| 84 | + |
| 85 | +Note: the last two issues are bugs which will be addressed by |
| 86 | +[https://github.com/bazelbuild/rules_scala/issues/839]. |
| 87 | + |
| 88 | +### [Experimental] Strict deps mode |
| 89 | +We have a strict dependency checker which requires that any type referenced in the sources of a |
| 90 | +scala target should be included in that rule's deps. To learn about the motivation for this you can |
| 91 | +visit this Bazel blog [post](https://blog.bazel.build/2017/06/28/sjd-unused_deps.html) on the |
| 92 | +subject. |
| 93 | + |
| 94 | +The option `strict_deps_mode` can be set to `off`, `warn`, or `error`. We highly recommend setting |
| 95 | +it to `error`. |
| 96 | + |
| 97 | +In both cases of `warn` or `error` you will get the following text in the event of a violation: |
| 98 | +``` |
| 99 | +... |
| 100 | +Target '//some_package:transitive_dependency' is used but isn't explicitly declared, please add it to the deps. |
| 101 | +You can use the following buildozer command: |
| 102 | +buildozer 'add deps //some_package:transitive_dependency' //some_other_package:transitive_dependency_user |
| 103 | +``` |
| 104 | +Note that if you have `buildozer` installed you can just run the last line and have it automatically |
| 105 | +apply the fix for you. |
| 106 | + |
| 107 | +Note that this option only applies to scala code. Any java code, even that within `scala_library` |
| 108 | +and other rules_scala rules, is still controlled by the `--strict_java_deps` command-line flag. |
| 109 | + |
| 110 | +### [Experimental] Unused dependency checking |
| 111 | +To allow for better caching and faster builds we want to minimize the direct dependencies of our |
| 112 | +targets. Unused dependency checking makes sure that all targets specified as direct dependencies are |
| 113 | +actually used. If `unused_dependency_checker_mode` is set to either |
| 114 | +`error` or `warn` you will get the following message for any dependencies that are not used: |
| 115 | +``` |
| 116 | +error: Target '//some_package:unused_dep' is specified as a dependency to //target:target but isn't used, please remove it from the deps. |
| 117 | +You can use the following buildozer command: |
| 118 | +buildozer 'remove deps //some_package:unused_dep' //target:target |
| 119 | +``` |
| 120 | + |
| 121 | +Unused dependency checking can either be enabled globally for all targets using a scala toolchain or for individual targets using the |
| 122 | +`unused_dependency_checker_mode` attribute. |
| 123 | + |
| 124 | +The feature is still experimental and there can thus be cases where it works incorrectly, in these cases you can enable unused dependency checking globally through a toolchain and disable reports of individual misbehaving targets with `unused_dependency_checker_ignored_targets` which is a list of labels. |
| 125 | + |
| 126 | +### [Experimental] Dependency tracking method |
| 127 | + |
| 128 | +The strict dependency tracker and unused dependency tracker need to track the used dependencies of a scala compilation unit. This toggle allows one to pick which method of tracking to use. |
| 129 | + |
| 130 | +- `dependency_tracking_method = "high-level"` - This is the existing tracking method which has false positives and negatives but generally works reasonably well for `direct` dependency mode. |
| 131 | +- `dependency_tracking_method = "ast"` - This is a new tracking method which is being developed for `plus-one` and `transitive` dependency modes. It is still being developed and may have issues which need fixing. If you discover an issue, please submit a small repro of the problem. |
| 132 | + |
| 133 | +By default, `plus-one` and `transitive` dependency modes will use the `ast` dependency tracking method, while `direct` mode will use the `high-level` dependency tracking method. |
| 134 | + |
| 135 | +Note we intend to eventually remove this flag and make the defaults non-configurable. |
| 136 | + |
| 137 | +### [Experimental] Turning on strict_deps_mode/unused_dependency_checker_mode |
| 138 | + |
| 139 | +It can be daunting to turn on strict deps checking or unused dependency mode checking on a large codebase. However, it need not be so bad if this is done in phases |
| 140 | + |
| 141 | +1. Have a default scala toolchain `A` with the option of interest set to `off` (the starting state) |
| 142 | +2. Create a second scala toolchain `B` with the option of interest set to `warn` or `error`. Those who are working on enabling the flag can run with this toolchain as a command line argument to help identify issues and fix them. |
| 143 | +3. Once all issues are fixed, change `A` to have the option of interest set to `error` and delete `B`. |
| 144 | + |
| 145 | +We recommend turning on strict_deps_mode first, as rule `A` might have an entry `B` in its `deps`, and `B` in turn depends on `C`. Meanwhile, the code of `A` only uses `C` but not `B`. Hence, the unused dependency checker, if on, will request that `B` be removed from `A`'s deps. But this will lead to a compile error as `A` can no longer depend on `C`. However, if strict dependency checking was on, then `A`'s deps is guaranteed to have `C` in it. |
| 146 | + |
| 147 | +### Include/exclude filters |
| 148 | +Both strict and unused deps tracking scope can be controlled by configuring *prefixes* of |
| 149 | +included/excluded targets on the toolchain with attributes |
| 150 | +`dependency_tracking_strict_deps_patterns`, `dependency_tracking_unused_deps_patterns`. |
| 151 | +Filters can be used for gradual migration towards strict/unused deps error mode. In general, you |
| 152 | +should get strict deps working first before enabling unused deps mode. |
| 153 | + |
| 154 | +Patterns prefixed with "-" will exclude targets. |
| 155 | + |
| 156 | +Example patterns: |
| 157 | +- `""` includes everything - default setting |
| 158 | +- `"//"` includes all local targets |
| 159 | +- `"//foo/"` includes everything under package `//foo`, if trailing slash is omitted, it will match |
| 160 | +other packages, which start with "some", eg. `//foo_bar` |
| 161 | +- `"//foo:bar"` includes target under label `//foo:bar` |
| 162 | +- `@junit_junit` includes external targets, which start with `"@junit_junit"` |
| 163 | +- `"-//foo:baz"` exludes target `//foo:baz` |
| 164 | + |
| 165 | +Exclusions take higher precedence over inclusions. Empty list will not match any targets. |
0 commit comments