Interactive callbacks and mutiple figures

[franalbani]

Hi!

Sorry for using this just for asking questions and not for any issue.

I'm making an educational Fourier Analysis tool that must show a signal and its spectrum side by side and immediately react to parameters changed by a slider widget.

I made this poc using Julia:

interactive_fourier.mp4

Would this be possible to to with this library?

I would very much love to use Haskell for the Linear Algebra part of the transform pairs.

Thanks!

[leftaroundabout]

module Main where

import Graphics.Dynamic.Plot.R2.Gtk

import qualified Data.Vector.Unboxed as V
import Data.Vector.FFT (fft)

import Data.Complex (magnitude, Complex((:+)))

rectangWindow :: Double -> Double -> Double
rectangWindow w x
 | x > -w/2  &&  x < w/2   = 1
 | otherwise               = 0

posInDiscrInterval :: Int -> (Double, Double) -> Int -> Double
posInDiscrInterval n (l,r) i = l + (r-l) * fromIntegral i / fromIntegral n

sampleTimeSeries :: Int -> (Double, Double) -> (Double -> Double) -> V.Vector Double
sampleTimeSeries n range f = V.generate n $ \i -> f $ posInDiscrInterval n range i

vectArrayPlot :: (Double, Double) -> V.Vector Double -> DynamicPlottable
vectArrayPlot range ys = lineSegPlot [ (posInDiscrInterval (V.length ys) range i, y)
                                     | (i,y) <- zip [0..] $ V.toList ys ]

windowAndSpectrumPlot :: Double -> DynamicPlottable
windowAndSpectrumPlot w = plotMultiple
           [ vectArrayPlot timeRange win
           , vectArrayPlot freqRange spectrum ]
 where timeRange = (-3,3)
       freqRange = (-3,3)
       nSample = 1024
       win = sampleTimeSeries 1024 timeRange $ rectangWindow w
       spectrum = V.map ((^^2) . magnitude . (/sqrt (fromIntegral nSample)))
                   $ V.drop (nSample`quot`2) fted <> V.take (nSample`quot`2) fted
        where fted = fft $ V.map (:+0) win


main :: IO ()
main = do
   plotWindow [ plot $ \(MousePress (xp,yp))
                          -> windowAndSpectrumPlot $ 2*xp
              , dynamicAxes
              ]
   return ()

Multi-plot figures are something I, somewhat embarrassingly, never got around to implementing. It could certainly be done though! Also, there are no labelled sliders. I personally prefer using the mouse position inside the plot, but YMMV.

[leftaroundabout]

@franalbani if you have any more questions, just ask ahead! I know this repo looks kind of dead (because I can't use it in my current main work projects), but I'm still very much invested in my Haskell libraries. Christmas is actually one of the better opportunities to make some progress here.

(More generally, I still work on Haskell, but have not pushed any of that to Github - mostly because I disagree with the company direction in the last years. Meaning open source will likely appear on Codeberg or SourceHut, not here.)

[franalbani]

Nice!

The code looks very simple: minimal scaffolding for achieving a simple thing! 🎩

How much of a challenge do you expect the multi-plot feature to be? Do you have more features in mind?

I don't want to provoke pointless work for you (yet 😆 ). I envision making a window with 6 or 8 plots of temporal and spectral aspects of typical DSP procedures like windowing, filtering, sampling, etc., with the minimal GUI to let the teacher/student choose and parametrize signals from a library. I'm not aware of any haskell library that provides:

• multi plots
• GUI widgets
• native (not web)
• reactive data

That may be a lot of work to do. Are this kind of things in the direction you intended?

Thanks!

[leftaroundabout]

How much of a challenge do you expect the multi-plot feature to be? Do you have more features in mind?

It depends on how to do it. My original plan (2016?) was to first add support to physical quantities, so you could for example have time on one axis, voltage on the other, both as type parameters to the DynamicPlottable type. This would give rise to a nice principled interface for how to put multiple plots together into a figure with matching x- or y-axes. I did not implement that then, out of fear that it would lead down a rabbit whole of type inference problems, but by now I have a good grasp on that.

As for the GUI aspects of putting plots together, it would probably be easiest to delegate it to diagrams.

I reckon it would be doable quite well in 1-2 weeks (but note that my time estimates are usually way off...)

Another matter is if additional information needs to be communicated between the subplots. What would be really nice is if any subplot could affect the others in a functional reactive programming framework. Then both axis-synchronization and control-panels could just be special cases of the general subplot-composition interface. I never used proper FRP though, and this could really require advanced typeclass trickery and might give rise to plenty of laziness problems that would need to be debugged.

GUI widgets

Right now there are only two displaying backends: popping up a GTK window, or rendering to a static file. It would be straightforward to add a function for putting a plot object into an existing GTK widget, but I'm not sure that's what you mean here?

native (not web)

– well, again: GTK, so yes this is native.

My agenda in the last year has rather been to move away from GTK, in favour of browser-based alternatives - but who knows when I'll have time for that. I certainly don't mind also keeping the GTK backend, the only problem here is that to make it future proof would require switching to diagrams-gi-cairo, which did not render correctly the last time I checked. But this is definitely fixable too.

reactive data

Can you say a bit more about what kind of data?

This library can both display animations, whether computed on-the-fly or from files, and interactions between real-time input and computed visualisations. Both are implemented in a somewhat hacky way though, in particular trying to combine multiple interactivity features doesn't really work (and unfortunately there isn't even a clear error in that case, but rather you get just a blank plot).

Again, the proper technique for all this would be FRP, but I don't have enough experience with that.

[franalbani]

I should clarify that when I wrote that list I wanted to emphasize that your library is very close to having all that I want, only missing multi plots and GUI widgets.

By GUI widgets I mean sliders, buttons, dropdown menus, etc. from which the user can choose and tweak signals under observation.

I have no experience with GTK, so I don't know if it is the best native option (which I value over web-based). I almost always prefer terminal UI for simplicity and portability but this app needs a GUI (and Web is OK). I tried and liked Julia's PlotUI lib running on pluto notebook but it couldn't keep up with the performance needed for real time modifications of the signals using sliders. Julia also has GLMakie which I used to do the video and is really fast and easy, but I would need to learn advanced Julia. Also I believe Haskell is much more appropriate to describe, teach and learn Fourier Analysis. I don't want just to plot; I also want the code to be aesthetically pleasing in a way I only know how to write in Haskell.

By reactive data I meant redrawing all plots with new samples calculated after some GUI widget triggered modifications on the signals under study. That needs to be fast to be useful. Your library seems to already achieve it.

As a curiosity, monomer's author chose Elm over FRP.

Thanks!