Add generic xy endpoint in the server #203

siwei-zhang-work · 2025-07-01T12:43:58Z

What it does

This series of commits add the new /BarChart endpoint proposed in:

eclipse-cdt-cloud/trace-server-protocol#114

It requires the introduced bar chart data provider implemented in:

eclipse-tracecompass/org.eclipse.tracecompass#291

CallStackAnalysis is used as the initial example input to demonstrate this functionality through a function density view.

How to test

You can test the new bar chart data provider endpoints using the following sequence of HTTP requests.

Open trace

POST http://localhost:8080/tsp/api/traces
Content-Type: application/json

{
  "parameters": {
    "uri": "/path/to/lttng/ust/trace",
    "name": "ls_ust"
  }
}

Create an experiment

POST http://localhost:8080/tsp/api/experiments
Content-Type: application/json

{
  "parameters": {
    "name": "testing_ust",
    "traces": ["2b8477ca-25d9-3ed4-83db-6736a7a33d46"]
  }
}

(optional) Check bar chart provider availability

GET http://localhost:8080/tsp/api/experiments/613c18b7-d239-3377-9482-2136946d4749/outputs/org.eclipse.tracecompass.analysis.profiling.core.callstack.barchart.provider

Get tree and axis descriptions

POST http://localhost:8080/tsp/api/experiments/613c18b7-d239-3377-9482-2136946d4749/outputs/genericXY/org.eclipse.tracecompass.analysis.profiling.core.callstack.functiondensity.provider/tree
Content-Type: application/json

{
  "parameters": {
    "requested_timerange": {
      "start": 111111111,
      "end": 222222222
    }
  }
}

Get bar chart data

POST http://localhost:8080/tsp/api/experiments/613c18b7-d239-3377-9482-2136946d4749/outputs/genericXY/org.eclipse.tracecompass.analysis.profiling.core.callstack.functiondensity.provider/xy
Content-Type: application/json

{
  "parameters": {
    "requested_items": [1, 2, 3, 4],
    "requested_timerange": {
      "start": 1750773997065742471,
      "end": 1750774010897424378,
      "nbSamples": 5
    }
  }
}

Follow-ups

Additional changes will be submitted to tsp-client and vscode-trace-extension to support bar chart views end-to-end.

Review checklist

[Y] As an author, I have thoroughly tested my changes and carefully followed the instructions in this template

bhufmann · 2025-07-03T14:27:54Z

Thanks for the contribution. I'll review it and provide feedback. Due to the size and impact on APIs it will take some days.

bhufmann

Initial review comment for this.

bhufmann · 2025-07-08T19:29:57Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+     * @return an {@link GenericView} with the results
+     */
+    @POST
+    @Path("/BarChart/{outputId}/tree")


I think the new endpoint should be about new xy endpoint where the x-axis is not the trace time. The chart type is part of the DisplayType. We could have other xy-charts like line, scatter etc. with a time-less x-axis.

Renamed to genericXY.

bhufmann · 2025-07-08T19:30:07Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+     * @return an {@link GenericView} with the results
+     */
+    @POST
+    @Path("/BarChart/{outputId}/bars")


I think the new endpoint should be about new xy endpoint where the x-axis is not the trace time. The chart type is part of the DisplayType. We could have other xy-charts like line, scatter etc. with a time-less x-axis.

Renamed to genericXY.

bhufmann

Thanks for the contribution. It's clean and thoroughly implemented. I haven't done a full review though. I will do that once the mainline PR is approved (close to be approved). I added a comment what I noticed already.

bhufmann · 2025-07-14T20:09:24Z

...incubator/trace/server/jersey/rest/core/tests/services/GenericXYDataProviderServiceTest.java

+        for (int i = 0; i < EXPECTED_ENTRIES.size(); i++) {
+            XyEntryStub expected = EXPECTED_ENTRIES.get(i);
+            XyEntryStub actual = actualEntries.get(i);
+            assertEquals("Entry ID mismatch at index " + i, expected.getId(), actual.getId());


the ID is not always the same since it's a running counter for all data providers. So, you can check for a particular value. Same is for the parent ID.

bhufmann · 2025-07-15T14:05:27Z

...incubator/trace/server/jersey/rest/core/tests/services/GenericXYDataProviderServiceTest.java

+
+        // Validate fields
+        assertEquals("Name mismatch", "UNKNOWN_PID", seriesStub.getName());
+        assertEquals("ID mismatch", 1, seriesStub.getId());


the series ID is not always 1. It depends on how many other data providers were created in other tests before.

bhufmann

Ok, I did a deeper review. I ran the tests successfully and was able to query the new endpoints. I added some comments. Please have a look. Thanks!

bhufmann · 2025-07-23T13:54:48Z

...acecompass/incubator/internal/trace/server/jersey/rest/core/webapp/SamplingDeserializer.java

+/**
+ * Deserializer for Sampling that infers the type from array structure.
+ */
+public class SamplingDeserializer extends JsonDeserializer<ISampling> {


Is this still needed? This means that the client will provide the sampling as input parameters to http queries.

No, it's not. Removed.

bhufmann · 2025-07-23T14:13:26Z

...ecompass/incubator/internal/trace/server/jersey/rest/core/webapp/AxisDomainDeserializer.java

+ * @author Siwei Zhang
+ * @since 10.2
+ */
+public class AxisDomainDeserializer extends JsonDeserializer<IAxisDomain> {


Is this still needed? This means that the client will provide the axis domain as input parameters to http queries.

No, it's not. Removed.

bhufmann · 2025-07-23T14:17:47Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+    @Operation(summary = "API to get the tree for generic xy chart", description = TREE_ENTRIES, responses = {
+            @ApiResponse(responseCode = "200", description = "Returns a list of generic xy chart entries. " +
+                    CONSISTENT_PARENT, content = @Content(schema = @Schema(implementation = XYTreeResponse.class))),
+            @ApiResponse(responseCode = "400", description = INVALID_PARAMETERS, content = @Content(schema = @Schema(implementation = String.class))),


I just merged a PR #208 that fixes the error handling and when an error happens an ErrorResponse is returned instead of String. Please update the swagger doc accordingly for all errors. After rebasing check other implementations on how to do that.

I think it's all marked in other comments? I don't see other changes apart from the parts you already commented.

bhufmann · 2025-07-23T14:18:16Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+            @ApiResponse(responseCode = "200", description = "Returns a list of generic xy chart entries. " +
+                    CONSISTENT_PARENT, content = @Content(schema = @Schema(implementation = XYTreeResponse.class))),
+            @ApiResponse(responseCode = "400", description = INVALID_PARAMETERS, content = @Content(schema = @Schema(implementation = String.class))),
+            @ApiResponse(responseCode = "404", description = PROVIDER_NOT_FOUND, content = @Content(schema = @Schema(implementation = String.class))),


ErrorResponse.class instead of String.class

bhufmann · 2025-07-23T14:18:39Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+                    CONSISTENT_PARENT, content = @Content(schema = @Schema(implementation = XYTreeResponse.class))),
+            @ApiResponse(responseCode = "400", description = INVALID_PARAMETERS, content = @Content(schema = @Schema(implementation = String.class))),
+            @ApiResponse(responseCode = "404", description = PROVIDER_NOT_FOUND, content = @Content(schema = @Schema(implementation = String.class))),
+            @ApiResponse(responseCode = "405", description = NO_PROVIDER, content = @Content(schema = @Schema(implementation = String.class)))


ErrorResponse.class instead of String.class

bhufmann · 2025-07-23T14:22:05Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+                .setCategory(outputId).build()) {
+            TmfExperiment experiment = ExperimentManagerService.getExperimentByUUID(expUUID);
+            if (experiment == null) {
+                return Response.status(Status.NOT_FOUND).entity(NO_SUCH_TRACE).build();


Use ErrorResponseUtil

bhufmann · 2025-07-23T14:22:12Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+            Map<String, Object> params = queryParameters.getParameters();
+            String errorMessage = QueryParametersUtil.validateGenericXYQueryParameters(params);
+            if (errorMessage != null) {
+                return Response.status(Status.BAD_REQUEST).entity(errorMessage).build();


Use ErrorResponseUtil

bhufmann · 2025-07-23T14:22:23Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+
+            if (provider == null) {
+                // The analysis cannot be run on this trace
+                return Response.status(Status.METHOD_NOT_ALLOWED).entity(NO_PROVIDER).build();


Use ErrorResponseUtil

bhufmann · 2025-07-23T14:23:48Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+            @Parameter(description = OUTPUT_ID) @PathParam("outputId") String outputId,
+            @RequestBody(description = "Query parameters to fetch the xy model. " + TIMERANGE + " " + ITEMS_XY, content = {
+                    @Content(examples = @ExampleObject("{\"parameters\":{" + TIMERANGE_EX + "," + ITEMS_EX +
+                            "}}"), schema = @Schema(implementation = RequestedQueryParameters.class))


RequestedQueryParameters with swagger to generate the TSP API.yaml specification is not sufficient anymore and needs to be update or a new one needs to be introduced to reflect the new input parameters for this. Please update.

Created new class (GenericXYQueryParameters and GenericTimeRange) for it.

bhufmann · 2025-07-23T14:30:35Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+    @Produces(MediaType.APPLICATION_JSON)
+    @Operation(summary = "API to get the xy model", description = "Unique endpoint for all xy models, " +
+            "ensures that the same template is followed for all endpoints.", responses = {
+                    @ApiResponse(responseCode = "200", description = "Return the queried xy response", content = @Content(schema = @Schema(implementation = XYResponse.class))),


I have not fully checked but I think the XYResponse / XYModel / SeriesModel class that are used to create the TSP API.yaml specification need to be updated or (new ones created) to reflect the updated models returned (regarding sampling, axis description etc).

You can check the generated yaml file by typing http://localhost:8080/tsp/api/openapi.yaml in your browser with the updated server running.

Modified directly inside those classes considering those classes are shared with legacy xy in code.

bhufmann

I re-reviewed this PR. I was able to test and use the new server with existing front-end code. I found a couple of minor things mostly regarding swagger doc.

bhufmann · 2025-07-25T13:33:07Z

...eclipse/tracecompass/incubator/internal/trace/server/jersey/rest/core/model/IAxisDomain.java

+    @JsonSubTypes.Type(value = AxisDomainCategorical.class, name = "categorical"),
+    @JsonSubTypes.Type(value = AxisDomainTimeRange.class, name = "timeRange")
+})
+public interface IAxisDomain {


For swagger documentation, we decided to not use the usual Java syntax for interfaces, i.e. Interfaces start with an I. So, change it here to AxisDomain.

bhufmann · 2025-07-25T13:33:22Z

...g/eclipse/tracecompass/incubator/internal/trace/server/jersey/rest/core/model/ISampling.java

+        ISampling.CategorySampling.class,
+        ISampling.RangeSampling.class
+})
+public interface ISampling {


For swagger documentation, we decided to not use the usual Java syntax for interfaces, i.e. Interfaces start with an I. So, change it here to Sampling.

bhufmann · 2025-07-25T13:58:55Z

...racecompass/incubator/internal/trace/server/jersey/rest/core/services/EndpointConstants.java

    static final String MARKER_CATEGORIES_EX = "\"" + REQUESTED_MARKER_CATEGORIES_KEY + "\": [\"category1\", \"category2\"]"; //$NON-NLS-1$ //$NON-NLS-2$
    static final String MARKER_SET_EX = "\"" + REQUESTED_MARKER_SET_KEY + "\": \"markerSetId\","; //$NON-NLS-1$ //$NON-NLS-2$
-    static final String TIMERANGE_EX = "\"" + REQUESTED_TIMERANGE_KEY + "\": {\"start\": 111111111, \"end\": 222222222, \"nbTimes\": 1920}"; //$NON-NLS-1$ //$NON-NLS-2$
+    static final String TIMERANGE_EX = "\"" + REQUESTED_TIMERANGE_KEY + "\": {\"start\": 111111111, \"end\": 222222222, \"nbTimes/nbSamples\": 1920}"; //$NON-NLS-1$ //$NON-NLS-2$


This examples is used in different places. nbSamples is not always applicable. I thing the example should be per use case and show nbTimes or nbSamples where applicable.

bhufmann · 2025-07-25T14:04:18Z

...compass/incubator/internal/trace/server/jersey/rest/core/model/GenericXYQueryParameters.java

+    /**
+     * Property names below use underscores as per trace-server protocol.
+     */
+    interface RequestedParameters {


I think this one has to have a different name because there already another interface with that name: RequestedQueryParameters.RequesteParameters. When I checked the generated yaml file, I only see one in the file. For me, it was RequestedQueryParameters.RequesteParameters, but I think that's random and could be the other way round.

Renamed to GenericXYQueryParameters. Now there are both RequesteParameters and GenericXYQueryParameters generated in the yaml.

bhufmann · 2025-07-25T14:07:07Z

...racecompass/incubator/internal/trace/server/jersey/rest/core/webapp/ISamplingSerializer.java

+ * - Categories → array of strings: ["Read", "Write"]
+ * - TimeRanges → array of arrays: [[1, 2], [2, 3]]
+ */
+public class ISamplingSerializer extends StdSerializer<ISampling> {


Should be called SamplingSerializer because it's not an interface

bhufmann · 2025-07-25T14:07:29Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/webapp/IAxisDomainSerializer.java

+ * @author Siwei Zhang
+ * @since 10.2
+ */
+public class IAxisDomainSerializer extends JsonSerializer<IAxisDomain> {


Should be called AxisDomainSerializer because it's not an interface

PatrickTasse

The commit header should be updated since it no longer introduces a /bars endpoint.

PatrickTasse · 2025-07-31T20:40:48Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/QueryParametersUtil.java

+     *            the mutable map of query parameters
+     * @return an error message if validation fails, or null otherwise
+     */
+    private static String validateRequestedTimeRangeWithSamples(Map<String, Object> params) {


I think we should just have a method that checks validity of requested_timerange (presence and its 3 fields) but does not convert it to requested_times like the other method does.

PatrickTasse · 2025-07-31T20:41:04Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/QueryParametersUtil.java

+            return MISSING_PARAMETERS + SEP + REQUESTED_TIMERANGE_KEY;
+        }
+
+        if(!(requestedTimeRange instanceof Map)) {


space after if (global comment)

PatrickTasse · 2025-08-05T14:50:25Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+     * @return an {@link GenericView} with the results
+     */
+    @POST
+    @Path("/genericXY/{outputId}/tree")


Minor comment, I would suggest to move the method so that both /genericXY/ endpoint methods are together.

PatrickTasse · 2025-08-05T15:35:47Z

...compass/incubator/internal/trace/server/jersey/rest/core/model/GenericXYQueryParameters.java

+
+        @JsonProperty("requested_timerange")
+        @Schema(requiredMode = RequiredMode.REQUIRED)
+        GenericTimeRange getRequestedTimeRange();


For a generic time range that might not be timestamps, I would have suggested just a requested_range of type Range. But this would require corresponding rework in Trace Compass data provider parameter / fetch parameter utils.

Now we have two flavors of requested_timerange, (start, end, nbTimes) vs. (start, end, nbSamples), so it might not be a bad idea to use two different properties instead, e.g. requested_timerange vs. requested_range...

requested_timerange will always represent a time range. The only difference is that it uses nbSamples instead of nbTimes, indicating that the sampling points may not be time-based. Considering that they always represent time range, I think it would be better to keep calling it requested_timerange? It'll also be possible to let all usage of requested_timerange to use nbTimes later.

Those are the descriptions for it:

start would be the start time in the trace to get the data from, e.g. trace start time

end would be the end time in the trace to get the data from, e.g trace end time

nbSamples would be in this case the number of buckets to return. Then the back-end will be able to fill the buckets accordingly and the bucket range is determined by the corresponding duration in the range. The Front-end will decide the number of samples. For example, if the view width it 1000 pixels and the it would ask for 1000 samples, then the bucket width would be 1 pixel. If it wants to display a bigger bucket width (e.g. 10 pixels) it would ask for 100 buckets for the same 1000 view pixels.

This commit uses the newly introduced data provider type to support "bar chart" views. [Added] /genericXY/<id>/trees endpoint for generic xy view [Added] /genericXY/<id>/xy endpoint for generic xy view [Fixed] All request parameter "requested_timerange" with no number of times specified

I believe all comments were addressed

siwei-zhang-work force-pushed the bar-chart-2 branch from 1f8d158 to 071955b Compare July 1, 2025 13:33

siwei-zhang-work mentioned this pull request Jul 1, 2025

Add generic xy data provider eclipse-tracecompass/org.eclipse.tracecompass#291

Merged

siwei-zhang-work force-pushed the bar-chart-2 branch 2 times, most recently from 1f89a7b to 48e8d11 Compare July 2, 2025 08:23

siwei-zhang-work changed the title ~~[WIP] Add bar chart endpoint in the server~~ Add bar chart endpoint in the server Jul 2, 2025

bhufmann requested changes Jul 8, 2025

View reviewed changes

siwei-zhang-work force-pushed the bar-chart-2 branch 3 times, most recently from e650ab3 to e0eab17 Compare July 11, 2025 12:35

bhufmann reviewed Jul 15, 2025

View reviewed changes

bhufmann requested changes Jul 23, 2025

View reviewed changes

siwei-zhang-work force-pushed the bar-chart-2 branch from e0eab17 to bf42b95 Compare July 25, 2025 10:26

bhufmann previously requested changes Jul 25, 2025

View reviewed changes

siwei-zhang-work changed the title ~~Add bar chart endpoint in the server~~ Add generic xy endpoint in the server Jul 28, 2025

siwei-zhang-work force-pushed the bar-chart-2 branch from bf42b95 to 4beeedd Compare July 28, 2025 07:54

siwei-zhang-work force-pushed the bar-chart-2 branch from 4beeedd to 109d9f5 Compare August 5, 2025 08:24

PatrickTasse requested changes Aug 5, 2025

View reviewed changes

siwei-zhang-work force-pushed the bar-chart-2 branch from 109d9f5 to 3af3883 Compare August 6, 2025 08:24

PatrickTasse approved these changes Aug 6, 2025

View reviewed changes

PatrickTasse merged commit bbde152 into eclipse-tracecompass-incubator:master Aug 14, 2025
2 of 4 checks passed

siwei-zhang-work mentioned this pull request Aug 20, 2025

server: fix swagger documentation for generic xy endpoint #215

Merged

siwei-zhang-work mentioned this pull request Sep 3, 2025

Add generic XY view eclipse-cdt-cloud/vscode-trace-extension#348

Merged

Add generic xy endpoint in the server #203

Add generic xy endpoint in the server #203

Uh oh!

Conversation

siwei-zhang-work commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does

How to test

Open trace

Create an experiment

(optional) Check bar chart provider availability

Get tree and axis descriptions

Get bar chart data

Follow-ups

Review checklist

Uh oh!

bhufmann commented Jul 3, 2025

Uh oh!

bhufmann left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bhufmann left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bhufmann left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

siwei-zhang-work commented Jul 1, 2025 •

edited

Loading

bhufmann left a comment •

edited

Loading