chore(tr): add dependency graph to TR rpc interface by bkettle · Pull Request #348 · semgrep/semgrep-interfaces

bkettle · 2025-02-11T01:16:38Z

TR will require sending the dependency graph in addition to the transitive findings in order to do the filtering step. This adds a single dependency graph to the RPC interface.

I ran make setup && make to update the generated code after editing a .atd file (TODO: have a CI check)
I made sure we're still backward compatible with old versions of the CLI.
For example, the Semgrep backend need to still be able to consume data
generated by Semgrep 1.50.0.
See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades
Note that the types related to the semgrep-core JSON output or the
semgrep-core RPC do not need to be backward compatible!

bkettle · 2025-02-11T01:16:56Z

chore(tr): add dependency graph to TR rpc interface #348 👈 (View in Graphite)
chore(sca-rpc): add internal dependency type #345
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

github-actions · 2025-02-11T01:17:39Z

Backwards compatibility summary:

Checking backward compatibility of semgrep_output_v1.atd against past version v1.100.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.101.0
Skipping v1.102.0 because commit 1c82453e89e0b569630e48ddde015e201df0e5f9 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.103.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.104.0
Skipping v1.106.0 because commit 5e0c767ec323f3f2356d3bf8dbdf7c7836497d8a has already been checked
Skipping v1.107.0 because commit 5e0c767ec323f3f2356d3bf8dbdf7c7836497d8a has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.108.0
--- before.txt
+++ after.txt
@@ -0,0 +1,21 @@
+[105360ef] Incompatibility in both directions:
+Case 'TransitiveUndetermined' used to not have an argument.
+The following types are affected:
+  ci_scan_results
+  cli_match
+  cli_match_extra
+  cli_output
+  cli_output_extra
+  core_match
+  core_match_extra
+  core_output
+  finding
+  function_call
+  function_return
+  matching_explanation
+  matching_explanation_extra
+  partial_scan_result
+  sca_match
+  sca_match_kind
+  transitive_finding
+
ERROR: semgrep_output_v1.atd is not backward compatible with v1.108.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.109.0
--- before.txt
+++ after.txt
@@ -0,0 +1,21 @@
+[105360ef] Incompatibility in both directions:
+Case 'TransitiveUndetermined' used to not have an argument.
+The following types are affected:
+  ci_scan_results
+  cli_match
+  cli_match_extra
+  cli_output
+  cli_output_extra
+  core_match
+  core_match_extra
+  core_output
+  finding
+  function_call
+  function_return
+  matching_explanation
+  matching_explanation_extra
+  partial_scan_result
+  sca_match
+  sca_match_kind
+  transitive_finding
+
ERROR: semgrep_output_v1.atd is not backward compatible with v1.109.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.75.0
Skipping v1.76.0 because commit 9102031608aa4154e1c37f557550ec4eabc8780c has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.77.0
Skipping v1.78.0 because commit dcb5d77b420ddee61f58aadd3c2c7aef38778154 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.79.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.80.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.81.0
Skipping v1.82.0 because commit 9e0f3bec26b07b4fb6753a32cb75277f45f2572c has already been checked
Skipping v1.83.0 because commit 9e0f3bec26b07b4fb6753a32cb75277f45f2572c has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.84.0
Skipping v1.84.1 because commit 3daef49297ada205359cc1d2996354c94b628b0d has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.85.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.86.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.87.0
Skipping v1.88.0 because commit 512c0bd97db59c48a5705b2741662a338776e438 has already been checked
Skipping v1.89.0 because commit 512c0bd97db59c48a5705b2741662a338776e438 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.90.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.91.0
Skipping v1.92.0 because commit 2351c5e528cb7430422208dc66707894c066b508 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.93.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.94.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.95.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.96.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.97.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.98.0
Skipping v1.99.0 because commit 60809032a2e39742f42910d46b3e5dd305b8b8cf has already been checked

aryx · 2025-02-11T05:24:47Z

semgrep_output_v1.atd

-  | CallTransitiveReachabilityFilter of transitive_finding list
+  (* for now, the transitive reachability filter takes only a single dependency graph as input.
+   * it is up to the caller to call it several times, one for each subproject *)
+  | CallTransitiveReachabilityFilter of ((found_dependency * downloaded_dependency list) * transitive_finding list)


I think you want (found _dependency * downloaded_dependency) list no ?
Also why you talk about graph in the comment?

correct, fixed.

The (found _dependency * downloaded_dependency) list encodes the dependency graph, so the comment is referring to that

aryx · 2025-02-14T13:39:01Z

semgrep_output_v1.atd

   *)
 }

+type transitive_reachability_filter_params = {


bkettle · 2025-02-20T01:44:15Z

semgrep_output_v1.atd

   * (similar to (LockfileOnlyMatch Transitive))
  *)
-  | TransitiveUndetermined
+  | TransitiveUndetermined of transitive_undetermined


@aryx I think we will need to override the backwards compatibility check for this, does that seem reasonable? Should be okay since these types aren't consumed anywhere, right?

yes it's fine

can you override and merge? I don't think I have permission

bkettle mentioned this pull request Feb 11, 2025

chore(sca-rpc): add internal dependency type #345

Merged

2 tasks

bkettle marked this pull request as ready for review February 11, 2025 01:18

aryx approved these changes Feb 11, 2025

View reviewed changes

aryx approved these changes Feb 14, 2025

View reviewed changes

semgrep_output_v1.atd

*)

}

type transitive_reachability_filter_params = {

Copy link

Contributor

aryx Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense

bkettle force-pushed the bk/add-internal-dependency-type branch from b8aaedd to c70701f Compare February 18, 2025 19:07

bkettle force-pushed the bk/tr/add-dep-graph-to-tr-rpc-interface branch from 2c662e2 to 2fabafb Compare February 18, 2025 19:07

aryx approved these changes Feb 18, 2025

View reviewed changes

bkettle force-pushed the bk/add-internal-dependency-type branch from c70701f to 9056434 Compare February 20, 2025 01:39

bkettle force-pushed the bk/tr/add-dep-graph-to-tr-rpc-interface branch from 2fabafb to 031c5a8 Compare February 20, 2025 01:39

bkettle commented Feb 20, 2025

View reviewed changes

bkettle changed the base branch from bk/add-internal-dependency-type to graphite-base/348 February 20, 2025 01:48

chore(tr): add dependency graph to TR rpc interface

ed2508e

bkettle force-pushed the graphite-base/348 branch from 9056434 to 7efe189 Compare February 20, 2025 01:48

bkettle force-pushed the bk/tr/add-dep-graph-to-tr-rpc-interface branch from 031c5a8 to 2d8231e Compare February 20, 2025 01:48

bkettle changed the base branch from graphite-base/348 to main February 20, 2025 01:48

more stuff

8289a19

bkettle force-pushed the bk/tr/add-dep-graph-to-tr-rpc-interface branch from 2d8231e to 8289a19 Compare February 20, 2025 01:48

aryx merged commit 47b9407 into main Feb 21, 2025
3 of 4 checks passed

aryx deleted the bk/tr/add-dep-graph-to-tr-rpc-interface branch February 21, 2025 16:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(tr): add dependency graph to TR rpc interface#348

chore(tr): add dependency graph to TR rpc interface#348
aryx merged 2 commits intomainfrom
bk/tr/add-dep-graph-to-tr-rpc-interface

bkettle commented Feb 11, 2025 •

edited

Loading

Uh oh!

bkettle commented Feb 11, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Feb 11, 2025 •

edited

Loading

Uh oh!

aryx Feb 11, 2025

Uh oh!

bkettle Feb 18, 2025

Uh oh!

aryx Feb 14, 2025

Uh oh!

bkettle Feb 20, 2025

Uh oh!

aryx Feb 20, 2025

Uh oh!

bkettle Feb 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

bkettle commented Feb 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bkettle commented Feb 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aryx Feb 11, 2025

Choose a reason for hiding this comment

Uh oh!

bkettle Feb 18, 2025

Choose a reason for hiding this comment

Uh oh!

aryx Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

bkettle Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

aryx Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

bkettle Feb 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bkettle commented Feb 11, 2025 •

edited

Loading

bkettle commented Feb 11, 2025 •

edited

Loading

github-actions bot commented Feb 11, 2025 •

edited

Loading