bazel-orfs and OpenROAD dependency

[oharboe]

bazel promises to be able to support exact dependencies, so eventually, the goal for bazel-orfs is that when yosys doesn't change, then synthesis isn't re-run and when OpenROAD doesn't change the flow after synthesis doesn't re-run.

However, OpenROAD is a very large collection of tools that turns into a single binary, so even if global routing changes, floorplan has to re-run.

If OpenROAD had a top executable that pulled in .so files for each of the major stages, then I'd be able to express more exact dependencies for floorplan, place, cts, grt, detailed routing, final and generate abstract.

On a practical side, I rarely change OpenROAD when working on the openroad repository, I contribute mostly to the test cases, so I only re-run the test cases that change.

In projects where I use bazel-orfs, I upgrade bazel-orfs, ORFS docker image rarely enough that this doesn't materially affect build times.

It would be neat though, if dependencies could be more accurately articulated and satisfying to change e.g. cts and have est cases run from cts and on only.

[maliberty]

I've thought about this too and it would be nice. It would be a large project and I expect it would quickly bottleneck at gpl which has a lot of dependencies.

[QuantamHD]

OpenROAD/src/odb/BUILD

Line 111 in 965bb20

py_library(

[oharboe]

We're going to to kill the -python version in favor of python_library for each tool, which is already at head. Just depend on the python lib of each tool in each of your steps, and build a py_binary for you tool calls in your bazel rules.

You'll keep the OPENROAD_EXE in ORFS though?

Note that the problem isn't using a different version, the problem is knowing which version to use.

[QuantamHD]

I mean the flow is pretty static and simple. Why can't you just manually go through and figure out what you call once. There's like 20 total steps

[oharboe]

Not exactly sure what you suggest.

bazel-orfs divides the flow into the major steps yosys canonicalization, synthesis, floorplan, place, cts, grt, detailed, final, generate abstract. The reason it doesn't divide it into finer steps is that that would create a LOT more artifacts, each of which is gigabytes for e.g. megaboom, so the current division is a compromise. There is a procedure to do debugging/hacking/investigation for each of these major steps starting from the beginning of the major step.

Setting up versions for each of these major steps would be painful to maintain manually.

Today I bump ORFS + bazel-orfs by running:

bazelisk run @bazel-orfs//:bump

This searches MODULE.bazel and updates bazel-orfs and ORFS.

[maliberty]

I think most people will use the Design/Tech APIs which have full program dependence anyways. To use individual libraries will be to recreate a lot of initialization bookkeeping which will be tough in python.

[oharboe]

Idea:

Sometimes I know that I want to test a cached flow with a new version of OpenROAD for a specific stage. I would hack my local bazel configuration to override the OpenROAD binary to use for CTS.

Here is a normal configuration:

https://github.com/The-OpenROAD-Project/megaboom/blob/616dca33782c2042f12e833e7b13a603d41bf3be/MODULE.bazel#L52-L56

Below, I use the OpenROAD in the docker image for all but the CTS stage:

orfs.default(
    image = "docker.io/openroad/orfs:v3.0-3872-g66e441c6c",
    # A label pointing to an OpenROAD binary, could be locally built or in a docker using further Bazel wizardry
    openroad_cts = "//:openroad",
    sha256 = "f75321f10023a8bf99b1ed5dfe1dab6351f588eda24f186a66e9145fd9b76fd1",
)

Developing this idea a bit further: if the Docker image contained copies of OpenROAD, one for each stage, each openroad binary the most recent version for each stage, then it would be trivial to modify bazel-orfs to use the binary for each stage. The ORFS docker image is large and compressed openroad binary is ca. 35mByte, so if there was a real upside to this, then cost isn't too great.

To automate this, some way to know which version of OpenROAD to use for each stage would be needed. I don't think a manually groomed list is a good idea :-)

One way to create this list of most recent version would be to scan the results of the previous ORFS docker image. If the result changed, then that stage gets a new openroad. This means that if cts changes, then we'll always get a new grt + route + final + generate_abstract binary, even if it isn't needed. Since the flow needs to re-run for those stages anyway, that seems tolerable.

If OpenROAD changes and it doesn't affect the output files, then it wouldn't be upgraded.

This idea works well for non-log files. They should be stable from build to build and no matter what machine you run on. Logs are a different matter, they differ from machine to machine from run to run by design.