Multiple Data Providers

[dpaulat]

Here is where my mind is at for a multiple-provider design, at least for Level 3. I think a sequence diagram probably illustrates this better than trying to describe it in paragraph form.

sequenceDiagram
    participant TimelineManager
    participant MapWidget
    participant RadarProductView
    participant RadarProductManager
    participant ProviderManager
    participant RadarProductDatastore


    # Initial data request
    Note over TimelineManager,RadarProductDatastore: Initial data request

    TimelineManager-)MapWidget: SelectTime
    MapWidget-)RadarProductView: SelectTime

    RadarProductView->>+RadarProductManager: GetLevel3Data
        RadarProductManager->>+RadarProductDatastore: GetProduct
        RadarProductDatastore-->>-RadarProductManager: Date Not Loaded
        RadarProductManager->>+ProviderManager: PopulateDate
            ProviderManager->>RadarProductDatastore: Set Date Loading
        ProviderManager-->>-RadarProductManager: OK
    RadarProductManager-->>-RadarProductView: Date Not Loaded
    RadarProductView--)MapWidget: Sweep Not Updated<br/>Date Not Loaded
    MapWidget--)TimelineManager: Sweep Not Updated<br/>Date Not Loaded


    # Continue product listing
    Note over TimelineManager,RadarProductDatastore: Product listing continues loading

    ProviderManager->>ProviderManager: Get available products for date
    ProviderManager->>RadarProductDatastore: Store Product List
    ProviderManager-)RadarProductManager: Product Listing Updated
    RadarProductManager-)RadarProductView: Product Listing Updated
    RadarProductView-)MapWidget: Needs Rendering


    # Product listing is now available, update again
    Note over TimelineManager,RadarProductDatastore: Data store now has a listing of products

    MapWidget->>RadarProductView: Update

    alt There is a matching product in the data store

        RadarProductView->>+RadarProductManager: GetLevel3Data
            RadarProductManager->>+RadarProductDatastore: GetProduct
            RadarProductDatastore-->>-RadarProductManager: Product Not Loaded (or Loading)

            Note over ProviderManager: If Product is Loading, it will not be reloaded

            RadarProductManager->>+ProviderManager: Load Product
                ProviderManager->>RadarProductDatastore: Set Product Loading
            ProviderManager-->>-RadarProductManager: OK
        RadarProductManager-->>-RadarProductView: Product Not Loaded
        RadarProductView--)MapWidget: Sweep Not Updated<br/>Product Not Loaded
        MapWidget--)TimelineManager: Sweep Not Updated<br/>Product Not Loaded

    else There is no product available, but another ProviderManager is still loading

        RadarProductView->>+RadarProductManager: GetLevel3Data
            RadarProductManager->>+RadarProductDatastore: GetProduct
            RadarProductDatastore-->>-RadarProductManager: Date Loading
        RadarProductManager-->>-RadarProductView: Date Loading
        RadarProductView--)MapWidget: Sweep Not Updated<br/>Date Loading
        MapWidget--)TimelineManager: Sweep Not Updated<br/>Date Loading

    else There is no product available

        RadarProductView->>+RadarProductManager: GetLevel3Data
            RadarProductManager->>+RadarProductDatastore: GetProduct
            RadarProductDatastore-->>-RadarProductManager: Product Not Available
        RadarProductManager-->>-RadarProductView: Product Not Available
        RadarProductView--)MapWidget: Sweep Not Updated<br/>Product Not Available
        MapWidget--)TimelineManager: Sweep Not Updated<br/>Product Not Available

    end


    # Product continues loading
    Note over TimelineManager,RadarProductDatastore: Product continues loading

    ProviderManager->>ProviderManager: Load Product
    ProviderManager->>RadarProductDatastore: Store Product
    
    Note over ProviderManager,RadarProductDatastore: If product is bad, will be marked<br/>in datastore to be disregarded.<br/>Product Loaded signal still<br/>occurs, allowing another<br/>product to be selected

    ProviderManager-)RadarProductManager: Product Loaded
    RadarProductManager-)RadarProductView: Product Loaded
    RadarProductView-)MapWidget: Needs Rendering


    # Product is loaded
    Note over TimelineManager,RadarProductDatastore: Product is loaded
    MapWidget->>RadarProductView: Update

    RadarProductView->>+RadarProductManager: GetLevel3Data
        RadarProductManager->>+RadarProductDatastore: GetProduct
        RadarProductDatastore-->>-RadarProductManager: Product
    RadarProductManager-->>-RadarProductView: Product
    RadarProductView--)MapWidget: Sweep Updated<br/>(or Unchanged)
    MapWidget--)TimelineManager: Sweep Updated<br/>(or Unchanged)

Loading

I think RadarProductManager has grown large, and would benefit from having ProviderManager broken out into its own file, completely separate.

If a file is loaded manually, it could be stored directly in the RadarProductDatastore.

I don't think this works out of the box for level 2 archive data, or for chunks. Maybe chunks is its own beast, and keeps its current handling mechanism inside RadarProductManager. I'm not sure if the datastore would be better off with an entire level 2 archive, or with individual elevation scans for each product. I'm leaning toward the latter, but then there's an indexing problem, not having the full index until the product is loaded (but maybe that's OK). The latter would work better in the event someone manually loads a file with a subset of an archive, or a single elevation scan.

Derived products and timeouts also need to be considered in this proposed design.