unable to match the results from plot on x function and distance between shapelet and time series values, unable to verify the results for being correct

[Meghna-desai]

I am working on a problem to classify a multivariate time series, it is almost like identifying a specific moment in the time series and consider that moment as class1 and rest all as class 0. I have taken help of this superb library to train an rdst classifier to solve my problem partially. I have used version 0.11.1 which permitted use of different distance measures with the classifier. After training when i try to check dtw distance and euclidean distance values for the identified shapelets and a random time series window from validation set, there were some problems i am facing

1. when the shapelets sizes were smaller than the largest shapelet size, it use inf to fill the values in the small shapelets and distance always returned inf

2. then I shortened the considered length of time series window to consider only the length of actual shapelet size , ignore the indexes of shapelets with inf values, and then calculated the distance measures

3. this is my major problem right now,
   where I am getting minimum distance between shapelet and time series window ,
   comparatively the plot on x actually shows a better plot for different shapelet and the same example time series window in contrast to
   the shapelet and the same example having lowest distance.

4. the shapelets of one class visualization method gives the error below when trying to plot shapelets

---

ValueError                                Traceback (most recent call last)
/tmp/ipython-input-3510015363.py in <cell line: 0>()
      5 id_class = 1 # Class 1 for event data
      6 # Visualize the top 10 most important shapelets for class 1
----> 7 fig = stc_vis.visualize_shapelets_one_class(
      8     X_Val,
      9     y_val,

3 frames
/usr/local/lib/python3.11/dist-packages/aeon/visualisation/estimator/_shapelets.py in plot_distance_vector(self, X, ax, show_legend, show_threshold, line_options, threshold_options, figure_options, rc_Params_options, matplotlib_style)
    357             X_means, X_stds = sliding_mean_std_one_series(X, self.length, self.dilation)
    358             X_subs = normalize_subsequences(X_subs, X_means, X_stds)
--> 359             _values = (self.values - self.values.mean(axis=-1)) / self.values.std(
    360                 axis=1
    361             )

ValueError: operands could not be broadcast together with shapes (4,9) (4,) 

Will be very grateful for your help

[TonyBagnall]

hi, thanks for that, I'll tag @baraline in this since RDST is his

[Meghna-desai]

Thank you for the help, if required I can share more details as per the requirement

[baraline]

Hey, could you by any chance compare with the version of RDST from latest (main) ? Some bugs were fixed recently. You won't get the different distances unfortunately, because it was bug-prone with numba.

For your issue, I'll get a look at it when i'm back home and come back to you.
Ideally if you have some small reproducible example that could be nice.

[Meghna-desai]

Ok I will try with the new version today, thanks for the help. But the only problem I had was that distance functions were removed in newer versions and I will have to check what accuracy and f1 score do I get, Thank you, Meghna Desai

…

On Wed, 13 Aug, 2025, 10:34 pm Antoine Guillaume, ***@***.***> wrote: Hey, could you by any chance compare with the version of RDST from latest (main) ? Some bugs were fixed recently. For your issue, I'll get a look at it when i'm back home and come back to you — Reply to this email directly, view it on GitHub <#3027 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APSRQ44NMSN7J4G4UPTG7D33NNV3FAVCNFSM6AAAAACDY6F5RCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMBZGU2DINQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[baraline]

Some stuff changed on the shapelet sampling too, you can play with the alpha similarity parameter to sample more (closer to 0, for ex 0.1) shapelets, as the bug was that it didn't account properly for it.

As a result of this parameter now working as intended, you can now end up with just a few shapelets if you have a very small datasets, thus the need to tune this parameter.

For your original issue at first look I'd say that some of the components of the visualisation method were not correctly adapted for multivariate series. I'll try to find the source.

If you really need to use other distances, you should be able monkey patch the latest aeon source on your machine to do it. We can discuss that later.

[Meghna-desai]

Thank you Antoine, actually I got really good results using the classifier but I am unable to verify it, so I am in a little difficult situation, and I am deeply grateful for all the help. I will definitely let you know about the results and progress Thanks and Regards, Meghna Desai

…

On Thu, 14 Aug, 2025, 10:56 am Antoine Guillaume, ***@***.***> wrote: Some stuff changed on the shapelet sampling too, you can play with the alpha similarity parameter to sample more (closer to 0, for ex 0.1) shapelets, as the bug was that it didn't account properly for it. For your original issue at first look I'd say that some of the components of the visualisation method were not correctly adapted for multivariate series. I'll try to find the source. — Reply to this email directly, view it on GitHub <#3027 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APSRQ46GLUPXA4WEEKVRTA33NQMXFAVCNFSM6AAAAACDY6F5RCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMBZHEZTONA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Meghna-desai]

One more help would be appreciated, is it possible to get some documentation about the output of the parameters _transformer__shapelets, I print get fitted parameters but not able to understand the output of _transformer__shapelets. Thanks and Regards, Meghna Desai On Thu, 14 Aug, 2025, 11:19 am Meghna Desai, < ***@***.***> wrote:

…

Thank you Antoine, actually I got really good results using the classifier but I am unable to verify it, so I am in a little difficult situation, and I am deeply grateful for all the help. I will definitely let you know about the results and progress Thanks and Regards, Meghna Desai On Thu, 14 Aug, 2025, 10:56 am Antoine Guillaume, < ***@***.***> wrote: > Some stuff changed on the shapelet sampling too, you can play with the > alpha similarity parameter to sample more (closer to 0, for ex 0.1) > shapelets, as the bug was that it didn't account properly for it. > > For your original issue at first look I'd say that some of the components > of the visualisation method were not correctly adapted for multivariate > series. I'll try to find the source. > > — > Reply to this email directly, view it on GitHub > <#3027 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/APSRQ46GLUPXA4WEEKVRTA33NQMXFAVCNFSM6AAAAACDY6F5RCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMBZHEZTONA> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >

[Meghna-desai]

sorry for the constant interruptions and trouble, the error received after trying to do visualization with 1.2.0, in case it helps ```python from aeon.visualisation import ShapeletClassifierVisualizer stc_vis1 = ShapeletClassifierVisualizer(rdstclf) id_class = 1 for i in range(rdstclf.max_shapelets): fig1=stc_vis1.plot_on_X(i, X_Val[230],figure_options={'dpi': 100, 'figsize': (25, 15)}) # #fig1=stc_vis1.plot_on_X(9, X_Val[237],figure_options={'dpi': 100, 'figsize': (25, 15)}) ``` ``` /usr/local/lib/python3.11/dist-packages/aeon/visualisation/estimator/_shapelets.py:306: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`. fig = plt.figure(**figure_options) /usr/local/lib/python3.11/dist-packages/aeon/visualisation/estimator/_shapelets.py:281: RuntimeWarning: invalid value encountered in divide _values = ( /usr/local/lib/python3.11/dist-packages/aeon/visualisation/estimator/_shapelets.py:281: RuntimeWarning: invalid value encountered in divide _values = ( ``` --------------------------------------------------------------------------- ``` IndexError Traceback (most recent call last) /tmp/ipython-input-2595736505.py <https://localhost:8080/#> in <cell line: 0>() 4 id_class = 1 5 for i in range(rdstclf.max_shapelets):----> 6 fig1=stc_vis1.plot_on_X(i, X_Val[230],figure_options={'dpi': 100, 'figsize': (25, 15)}) 7 # 8 #fig1=stc_vis1.plot_on_X(9, X_Val[237],figure_options={'dpi': 100, 'figsize': (25, 15)}) ``` ------------------------------ 2 frames ------------------------------ ``` /usr/local/lib/python3.11/dist-packages/aeon/visualisation/estimator/_shapelets.py <https://localhost:8080/#> in _get_shapelet(self, id_shapelet) 433 def _get_shapelet(self, id_shapelet): 434 if isinstance(self.estimator, RandomDilatedShapeletTransform):--> 435 values_ = self.estimator.shapelets_[0][id_shapelet] 436 # startpos_ = self.estimator.shapelets_[1][id_shapelet] 437 length_ = self.estimator.shapelets_[2][id_shapelet] IndexError: index 7960 is out of bounds for axis 0 with size 7960 ``` Thanks and Regards, Meghna Desai On Thu, Aug 14, 2025 at 11:24 AM Meghna Desai < ***@***.***> wrote:

…

One more help would be appreciated, is it possible to get some documentation about the output of the parameters _transformer__shapelets, I print get fitted parameters but not able to understand the output of _transformer__shapelets. Thanks and Regards, Meghna Desai On Thu, 14 Aug, 2025, 11:19 am Meghna Desai, < ***@***.***> wrote: > Thank you Antoine, actually I got really good results using the > classifier but I am unable to verify it, so I am in a little difficult > situation, and I am deeply grateful for all the help. > I will definitely let you know about the results and progress > > > Thanks and Regards, > Meghna Desai > > On Thu, 14 Aug, 2025, 10:56 am Antoine Guillaume, < > ***@***.***> wrote: > >> Some stuff changed on the shapelet sampling too, you can play with the >> alpha similarity parameter to sample more (closer to 0, for ex 0.1) >> shapelets, as the bug was that it didn't account properly for it. >> >> For your original issue at first look I'd say that some of the >> components of the visualisation method were not correctly adapted for >> multivariate series. I'll try to find the source. >> >> — >> Reply to this email directly, view it on GitHub >> <#3027 (reply in thread)>, >> or unsubscribe >> <https://github.com/notifications/unsubscribe-auth/APSRQ46GLUPXA4WEEKVRTA33NQMXFAVCNFSM6AAAAACDY6F5RCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMBZHEZTONA> >> . >> You are receiving this because you authored the thread.Message ID: >> ***@***.***> >> >

[baraline]

Thanks, I'll take a look at it when I got some time, it's also possible that the visualisation simply didn't support multivariate to begin with... Which we can try to fix later if that's true.

What did you meant by "got really good results using the classifier but I am unable to verify it" ? Verify it by visualisation ?

[Meghna-desai]

Sir I got accuracy of 97% and f1 score of 92% on validation set, There were four features used to train the classifier, Now I tried to find distance between the shapelets and some examples, the distance values for class 0 and class 1 random examples were almost same for a particular shapelet I want verify and show the shapletes closer to Class0 examples and class 1 examples via distance, visualization some method But in both cases it's almost diffcult to validate the results and claim the suitability of identified shapelets towards classification score. I want to validate the same and visualization and distances should match for that I believe.

…

On Thu, 14 Aug, 2025, 8:31 pm Antoine Guillaume, ***@***.***> wrote: Thanks, I'll take a look at it when I got some time, it's also possible that the visualisation simply didn't support multivariate to begin with... Which we can try to fix later if that's true. What did you meant by "got really good results using the classifier but I am unable to verify it" ? Verify it by visualisation ? — Reply to this email directly, view it on GitHub <#3027 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APSRQ42EWDW2HYBX26PNTC33NSQDBAVCNFSM6AAAAACDY6F5RCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMJQGUZDAMY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[baraline]

What is happening on your previous example is that you are looping through all the shapelets using the max_shapelets parameter. The documentation state the following:

max_shapelets : int, default=10000
        The maximum number of shapelets to keep for the final transformation.
        A lower number of shapelets can be kept if alpha similarity has discarded the
        whole dataset.

The error you get :

IndexError: index 7960 is out of bounds for axis 0 with size 7960

Indicates that you are trying to access a shapelet that does not exists, as the algorithm can sometime sample less than max_shapelets depending on the alpha_similarity parameter and the size of your dataset.

To fix your issue :

You can extract the true number of shapelets by accessing the arrays that store the shapelets parameter in the transformation of the classifier as follows :

from aeon.classification.shapelet_based import RDSTClassifier
clf = RDSTClassifier(n_jobs=-1, random_state=42).fit(X_train, y_train)
n_shapelets = len(clf._transformer.shapelets_[0]) # fetch the first shapelets parameter array and take its length, as it is of size n_shapelets

You can then loop on this value to avoid the indexing error.

About multivariate visualization

• The method was clearly made with univariate series in mind, there is some bugs in the legends, for example when calling :

from aeon.visualisation import ShapeletClassifierVisualizer
visualizer = ShapeletClassifierVisualizer(clf)
fig = visualizer.visualize_shapelets_one_class(X_train, y_train, class_id=0)

and the visualisation is just messy and ugly, not helping much to understand what's going on for multivariate series.

So it is possible to visualize multivariate series and shapelets, but the tool was clearly not made with this use case in mind, especially when the number of channels grows.

You could get some idea of what features are important by looking at the boxplots. In your message you said that the distances between a shapelets and the two classes was very similar, then it is maybe the number of occurences of the shapelets (relative to its threshold parameter) or the localisation (argmin of the distance vector) that might be discriminative.

Issue to raise on our side to improve aeon:

• Accessing the number of shapelets should be easy and not hidden behind such complexity, exposing a n_shapelets parameter for the transformer and the classifier, and documenting its usage
• Fix the legend bug in the visualization of multivariate
• Rethink visualisation for multivariate series.

[Meghna-desai]

So thankful for the prompt responses, the quick help and also the time invested in helping me. If any other issue faced I will create a specific query related to the same. Request to enhance the documentation to some more detail.

[MatthewMiddlehurst]

If you have any improvements could you comment them on #3028. More likely yo be acted on if the requests is an an issue and for specific items.

[Meghna-desai]

Thank you for your response, I will definitely put with detailed information any issues I face so that you get clear idea and it can also help me solve the issues I am facing, Thanks again! Meghna Desai

…

On Fri, 15 Aug, 2025, 8:33 pm Matthew Middlehurst, ***@***.***> wrote: If you have any improvements could you comment them on #3028 <#3028>. More likely yo be acted on if the requests is an an issue and for specific items. — Reply to this email directly, view it on GitHub <#3027 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APSRQ46TJJYYWCT4INZPT433NXZFFAVCNFSM6AAAAACDY6F5RCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMJRG44TONQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>