Skip to content

Change metadata file format to something WASM compatible #153

@andrewphilipsmith

Description

@andrewphilipsmith

To be decided

One possible workaround for Urban-Analytics-Technology-Platform/popgetter#73 is to change the file format used for publishing the metadata files.

  • The metadata files are always read in full.
  • (One of) the advantage of parquet is that partial files can be read over http.
  • Therefore, let us use a file format that can be read in WASM.
  • Gzipped CSVs might be a good enough fit.

(There is already a workaround for reading parquet metrics themselves using DuckDB)

@sgreenbury and @andrewphilipsmith have discussed this.

This is a big change therefore we would like @stuartlynn opinion before implementing this.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

In Progress:

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions