Spark Streaming support in spline

[vishalag001]

Hi @wajda,

Is there any plan to support streaming applications in spline. I see there are discussions already which points that spline model needs to be reworked.
I have 2 approaches in mind which I needed your feedback:

1. Use the StreamingQueryListener and implement the required methods.
2. Use the QueryListener which will only support forEachBatch based strreaming jobs. Idea is to use a Hashmap in the agent which keeps track of appId+targetDataSource. On first batch, plans and event are inserted to Arango and an entry is made to hashmap. But on subsequent calls, the inserts are skipped as hashmap already holds the appid+targetDataSource. This approach is just a workaround on the batch based solution.

Let me know your thoughts and thank you in advance !

[vishalag001]

Hi @wajda Can you share your thoughts on the above approach ?