Migrate to BCR rules_scala and Bazel BSP#453
Migrate to BCR rules_scala and Bazel BSP#453MrAMS wants to merge 3 commits intoThe-OpenROAD-Project:mainfrom
Conversation
oharboe
left a comment
oharboe
left a comment
There was a problem hiding this comment.
will test on private repository, but we need bloop before I can take delivery
MrAMS
force-pushed
the
migrate-to-BCR-rules-scala
branch
from
a1820c3 to
bd98d24
Compare
MrAMS
changed the title
|
Fails: |
|
|
||
| This toolchain is adapted to work with Chisel and support vscode with bloop. | ||
| This directory contains Bazel rule wrappers for Chisel hardware design, built on top of [BCR rules_scala](https://github.com/bazelbuild/rules_scala). | ||
|
|
There was a problem hiding this comment.
There are so few users of bazel-orfs and Chisel, we don't need to have permanent record this migration, it is more than enough to have a pull request comment.
Better to have a longer term README.md that focuses on how it is after this migration.
There was a problem hiding this comment.
I am thinking make these chisel rules to a new chisel_rules BCR package in the future.
There was a problem hiding this comment.
Yes: they don't belong in bazel-orfs as such. bazel-orfs has become a staging ground for various and sundry needed for OpenROAD+Chisel work.
There was a problem hiding this comment.
I will try to export these rules to a single rules_chisel in future.
MrAMS
force-pushed
the
migrate-to-BCR-rules-scala
branch
3 times, most recently
from
abc6ed2 to
6019831
Compare
MrAMS
changed the title
|
I've decided to migrate to the standard Bazel BSP (via bazel-bsp) instead of maintaining our own Bloop configuration rules locally. While Bloop offers great performance, maintaining custom build rules to generate bloop.json files created significant maintenance overhead and technical debt. The modern Bazel BSP ecosystem has matured enough to provide a stable IDE experience without requiring us to manually manage the bridge between Bazel and the IDE. This aligns us better with the upstream community standards. |
|
@oharboe Hi, CI all GREEN 🟢 , you can review now 🥳 |
I have tested in |
|
What happens with bazelbsp and running bazel on the command line, are they fighting over the same server? I.e. does it lock up? The motivation for using bloop was the bazelbsp was blocking the command line bazel use and vice versa. |
|
Yes, they do contend for the same Bazel server lock. Since Bazel enforces a single-writer lock per output base to ensure consistency, running a build in the IDE (via bazel-bsp) will block CLI commands until it finishes, and vice-versa. However, this is usually acceptable or even desired because they share the analysis cache and build artifacts. This means if I build a target in the IDE, a subsequent CLI build of the same target is instant (cached). If we separated them, we would double the memory usage and disk space, and lose the shared caching benefits. |
|
I think bloop creates a second copy and avoids the lockup and conflict, but there's a cost. Will test. |
|
We can configure the BSP server to run with its own output base. This would allow the IDE and CLI to run completely in parallel (separate Bazel servers), at the cost of higher RAM usage and no shared build cache. For now, sharing the server seems like the better trade-off for resource efficiency. |
|
Sounds good. We should document the both use cases, I think it is a matter of preferences as much as constraints |
|
run bazelisk mod tidy, re. failure in CI. Testing more locally though. |
|
Closer... I need to make this change $ git diff
diff --git a/.bazelproject b/.bazelproject
index 6bf48fb..3210dc1 100644
--- a/.bazelproject
+++ b/.bazelproject
@@ -1,6 +1,6 @@
targets:
- //chisel:all
- //sby:all
+ //chisel:codegenlib
+ //chisel:applicationlib
allow_manual_targets_sync: false
But still not quite there... 
UUhhh... Suddenly it worked... 
|
MrAMS
force-pushed
the
migrate-to-BCR-rules-scala
branch
from
079d867 to
590a331
Compare
|
I am trying to rebase main, could you tell which one is right? load("@rules_python//python:defs.bzl", "py_binary")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
# or
load("@bazel_orfs_rules_python//python:defs.bzl", "py_binary")
load("@bazel_orfs_rules_python//python:pip.bzl", "compile_pip_requirements") |
|
Oh I see pull/457, |
Replace custom Scala toolchain implementation with BCR rules_scala 7.1.5 while preserving Chisel wrapper API (chisel_library, chisel_binary, chisel_test). Changes: - Add rules_scala 7.1.5 dependency from Bazel Central Registry - Configure Scala 2.13.17 toolchain via BCR extensions - Rewrite Chisel wrappers to call BCR rules instead of custom implementation - Remove ~1500 lines of custom Scala toolchain code - Implement custom test wrapper with launcher script for environment variable expansion - Temporarily disable scala_bloop (to be migrated later) Benefits: - Simpler codebase (1900 lines removed, 400 lines added) - Standard community-maintained rules - Easier upgrades and better long-term maintainability
MrAMS
force-pushed
the
migrate-to-BCR-rules-scala
branch
from
590a331 to
8b4b904
Compare
Yes. |
Replace deprecated Bloop integration with modern Bazel BSP for Metals IDE support. Changes: - Support SemanticDB - Remove scala_bloop.bzl and tools/deploy.py (deprecated Bloop implementation) - Remove blooplib target (used only for Bloop, caused CI failures) - Remove CI Bloop check (replaced by Smoketests job) - Add .bazelproject to configure BSP target selection - Fix MODULE.bazel: use standard @rules_python for BSP compatibility - Fix generate.bzl: remove allow_single_file from fir_library generator - Fix chisel_test: use @bazel-orfs label for external dependency compatibility - Update README with complete BSP setup instructions - Add .bsp/ and .bazelbsp/ to .gitignore Bazel BSP provides direct integration with Bazel, better accuracy, and active community support. Refs: http://scalameta.org/metals/docs/build-tools/bazel/
MrAMS
force-pushed
the
migrate-to-BCR-rules-scala
branch
from
8b4b904 to
02c1976
Compare
|
CI should be passing, network issue. |
|
I'm testing with a large downstream project. It is not perfect and sometimes it doesn't work for no apparent reason, but it does work. I'm running some more tests and then I will take delivery and start learning and fixing and refining. We're in considerable pain with bloop with no hope of improvement. Switching to bsp will mean some immediate relief, some new pain but a good diagnosis 😌 Thank you so much for all the help! |
|
Great to hear! Our downstream projects will both benefit from this PR merge. Adopting |
Can I ask where it is being used? I'm sure @tspyrou and @maliberty would be interested to hear! |
|
😿 A few more nits... What's going on here? |
|
@MrAMS Could you test on a downstream repository? I think tests inside bazel-orfs are working but when chisel_test() is used from outside bazel-orfs, there are problems. More mysterious verilator problems when running on downstream repostories: m |
MrAMS
changed the title
|
I do not use |
We are a research group within the State Key Laboratory of Processors, Institute of Computing Technology, Chinese Academy of Sciences. |
Ouch... Is there any hope of getting this to work? |
|
I'll look into |
|
Before we can merge this, we need to know that we can upgrade OpenROAD.... Could you create a pull equest alongside this one that works? The-OpenROAD-Project/OpenROAD#9081 You can use the hash from a pull request in this repository when testing the pull request above with this pull request, no need to have it merged first. |
|
Hi @oharboe check The-OpenROAD-Project/OpenROAD/pull/9081. This workflow requires approval from a maintainer. |
Allright, CI is passing. I’m heading to bed 💤. Handle the remain issues another day. |
|
merged here #463 |
2 participants
Summary
This PR migrates the Scala toolchain from a custom implementation (~1500 lines) to the community-maintained BCR rules_scala 7.1.5, and replaces the deprecated Bloop IDE integration with modern Bazel BSP.
Changes
Scala Toolchain Migration:
IDE Support Modernization:
Compatibility Fixes:
Benefits