Floating Lidar File Format Standardization Development - 2024

[stephenholleran]

19-Sep-2024 First Meeting

Attendance:
Stephen Holleran (BrightWind), Jon Upton (Shell), Ryan Scott (RWE), Gibson Kersting (RWE), Jason Fields (RES), Amit Bohara (Altosphere), Anselme Troiville (Green Rebel), Hanna Torrens-Spence (BrightWind), Jose Miguel Garro (Eolos), Felix Kelberlau (Fugro), Kenneth John (RINA), Maximillian Schuberth (Recons), Richard Khaira-Creswell (Axys), David McCracken (BP), Philip Hargreaves (TGS) [15]

Minutes:

1. Quick intros.
2. Went through the presentation https://docs.google.com/presentation/d/1w-VNQEwlnxh1dC20qUz2uNqeTybNoFB18HVZ2esUNgg/edit?usp=sharing
3. A discussion followed.
4. Everyone on the call was committed to develop this standard.

Actions

• Prepare for next meeting in 2 weeks. Objective to pull from the data model the metadata we want to capture.

[stephenholleran]

17-Oct-2024 2nd Meeting

Attendance:
Stephen Holleran (BrightWind), Hanna Torrens-Spence (BrightWind), Jose Miguel Garro (Eolos), Ryan Scott (RWE), Jon Upton (Shell), Anselme Troiville (Green Rebel), Philip Hargreaves (TGS), Felix Kelberlau (Fugro) (temporarily), Amit Bohara (Altosphere) [9].

Minutes:

1. Quick intros.
2. We had to skip the call on the 3rd Oct due to the ACP conf.
3. I am thinking about this standard file format in 3 parts:
   1. File name and extension
   2. Header
   3. Column names
4. I think we should start with the header first. The below example is following the WRA Data Model from measurement_location to logger_main_config.

   name: Site X - FLS 01
   latitude_ddeg: 53.5017
   longitude_ddeg: -19.0966
   measurement_station_type: floating_lidar
   logger_main_config:
     - logger_oem: ZX Lidars
       logger_model_name: 300M
       logger_serial_number: 1234
       logger_firmware_version: v12345
       logger_id: 1234
       logger_name: FLS1
       offset_from_utc_hrs: 0.0
       averaging_period_minutes: 10
       timestamp_is_end_of_period: True
       lidar_config:
           flow_corrections_applied: false
           logger_stated_device_datum_plane_height_m: 2
           logger_stated_device_orientation_deg: 3
     - logger_oem: Nortek
       logger_model_name: Signature250
       logger_serial_number: 1234567
       offset_from_utc_hrs: 0.0
       averaging_period_minutes: 10

   1. Very minimal is actually required for the data model. The minimum would be name, latitude_ddeg, longitude_ddeg and measurement_station_type.
   2. If logger_main_config is provided, the minimal from that would be logger_oem and logger_serial_number.
   3. Leaving out the date_from and date_to as this config is to cover the config used to produce the timeseries data in the file. Yes, we all agree.
   4. Using YAML format instead of JSON to make it more readable and simple. We could just use the WRA Data Model JSON but I think it would be too long and complicated? JSON equivalent: Best to stick with JSON as it is one less thing to explain. And we are all used to JSON and the data model is already in JSON.

   {
       "name": "Site X - FLS 01",
       "latitude_ddeg": 53.5017,
       "longitude_ddeg": -19.0966,
       "measurement_station_type_id": "floating_lidar",
       "logger_main_config": 
       [{
   	    "logger_oem": "ZX Lidars",
   	    "logger_model_name": "300M",
   	    "logger_serial_number": "1234",
   	    "logger_firmware_version": "v12345",
   	    "logger_id": "1234",
   	    "logger_name": "FLS1",
   	    "offset_from_utc_hrs": 0.0,
   	    "averaging_period_minutes": 10,
   	    "timestamp_is_end_of_period": true,
   	    "lidar_config":
   	    {
   		    "flow_corrections_applied": false,
   		    "logger_stated_device_datum_plane_height_m": 2,
   		    "logger_stated_device_orientation_deg": 3
   	    }
       },{
   	    "logger_oem": "Nortek",
   	    "logger_model_name": "Signature250",
   	    "logger_serial_number": "1234567",
   	    "offset_from_utc_hrs": 0.0,
   	    "averaging_period_minutes": 10,
       }]
   }

   9. Do we need a section to capture 'file format version', 'software processing version', 'firmware version', ???? Yes. What are they:
      1. 'file format version' is the version of this file we are developing now.
      2. 'software processing version' could be the version of the floating lidar OEM pipeline to produce the file.
      3. 'buoy version' Do have buoy version numbers but probably not that important to capture. But could be used to verify the type verification. No harm in including as all these may be optional.
      4. Maybe these should be compulsory to set a standard that consumers of the data will know that this info is in there and any new entrants to the market would be encourage to have good processes in place for example, that they have good processes to track their pipeline process and can trace which version of the ZPH2CSV converter was used.
      5. We'll come back to this again in the next call.
   10. I think the measurement_point section of the data model can be covered by a structured column name of the timeseries data? Yes we think this would be a good thing. It is not deviating from the data model as such, we are just not including that part of the data model in the header.
5. Column names: these could contain the minimal information so we can avoid having a very large header with measurement point information.
   1. Format: <sensor_type>_<serial_number>_<measurement_type>_<height>_<measurement_units>_<statistic_type>
      Example: lidar_1234_wind_speed_110_m/s_avg
   2. Continue this discussion on the next call.

Actions

• Think about the extra section to capture versions.
• As well as prepare for next call to discuss column name format and file name.

[stephenholleran]

31-Oct-2024, 3rd Meeting

Attendance:
Stephen Holleran (BrightWind), Jon Upton (Shell), Anselme Troiville (Green Rebel), Hanna Torrens-Spence (BrightWind), Jose Miguel Garro (Eolos), Ryan Scott (RWE), Felix Kelberlau (Fugro), Gibson Kersting (RWE) [8]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. This isn't really a "daily" file format but a "data" file format. Be conscious of this when finalising the name for this.
4. Header section to capture OEM buoy specific details:
   1. Options:
      1. 'format_version' is the version of this file we are developing now. Alternatives: 'file_format_version', 'file_system_version'? Yes, we agree that this would be good to include. 'format_version' is good to use.
      2. 'software_version' could be the version of the floating lidar OEM pipeline to produce the file. Software version is too broad to know what it refers to. Also, narrowing down the usefulness of this format. Very difficult to version control the pipeline that generates this file. Not easy to maintain and so we decided not to include something like this.
      3. 'buoy_version' Type verification is a certain type of buoy with a certain type of lidar that gets type certification. 'fls_type' would be more appropriate. The serial number can allow the user to work out what fls type this is though. Therefore, not that useful so no need to include it.
      4. 'buoy_serial'? Yes, do include. Alt options: 'fls_serial_number', 'station_serial_number'. 'station_serial_number' is good to go with so there could be a buoy with no lidar (a basic metocean station). This 'station_serial_number' could also be put into the WRA Data Model under the 'measurement_location'. EDIT: Actually it can't as there are no date_from and date_to at this level to allow it to change over time.
      5. 'notes' could be useful to include. Notes are actually included at the measurement location level so any notes could be used there instead. Include it in the example.
   2. Should these be required? Yes. Good to include and is basic info so should be relatively easy to implement.
5. Column names:
   1. Suggested format: <sensor_type>_<serial_number>_<measurement_type>_<height_m>_<measurement_units>_<statistic_type>
      Example: lidar_1234_wind_speed_110_m/s_avg
   2. Where the values follow the data model type and enums.
   3. This can get very long especially with long serial numbers. When working with this data in Excel, Python or MATLAB it could be cumbersome. Also, when a lidar is swapped out the file would contain a whole load of different column names than the previous file that would all need to get mapped to the appropriate measurement point names. The only way to have shorter column names would be to include the 'measurement_point' section from the data model included in the header. EDIT: However, we also think a standard column name is worth pursuing so continued to develop it.
   4. Separator is an underscore however element names also use underscore. This can cause a problem with splitting out the data into individual components. Can we use a different separator or should we just handle the single underscores?
      1. Can't use a hyphen (-) as it can get mixed up with a negative height and dimensionless units.
      2. Can't use a period (.) as it can get mixed up with decimal heights and/or lat and long.
      3. Can't use a forward-slash (/) as it can get mixed up with units.
      4. A semicolon (;) can also be used as a delimiter so we should avoid that however, if the delimiter for the timeseries data is a comma (,) then we could use a semicolon.
      5. A colon (:) is an option.
      6. A double underscore is also an option. This is the preferred option instead of the single underscore.
      7. Examples:
         1. lidar:1234:wind_speed:110:m/s:avg
         2. lidar;1234;wind_speed;110;m/s;avg
         3. lidar__1234__wind_speed__110__m/s__avg
   5. Are all parts of the column name required?
      1. serial_number is probably the only one that may not be known. In the data model 'null' is allowed.
      2. They should all be required.

Actions

• [ ]

[stephenholleran]

14-Nov-2024, 4th Meeting

Attendance:
Stephen Holleran (BrightWind), Jon Upton (Shell), Gibson Kersting (RWE), Hanna Torrens-Spence (BrightWind), Jose Miguel Garro (Eolos), Michael Wiegele (RPS), Niko Simisiroglou (RWE), Philip Hargreaves (TGS) [8]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.

3. {
    "format_version": "1.0.0",
    "station_serial_number": "FLS 01",
    "name": "Site X",
    "latitude_ddeg": 53.5017,
    "longitude_ddeg": -19.0966,
    "measurement_station_type_id": "floating_lidar",
    "notes": "Some additional notes specific for this deployment.",
    "logger_main_config":
    [{
    	"logger_oem": "ZX Lidars",
    	"logger_model_name": "300M",
    	"logger_serial_number": "1234",
    	"logger_firmware_version": "v12345",
    	"logger_id": "1234",
    	"logger_name": "FLS1",
    	"offset_from_utc_hrs": 0.0,
    	"averaging_period_minutes": 10,
    	"timestamp_is_end_of_period": true,
    	"lidar_config":
    	{
    		"flow_corrections_applied": false,
    		"logger_stated_device_datum_plane_height_m": 2,
    		"logger_stated_device_orientation_deg": 3
    	}
    }, {
    	"logger_oem": "Nortek",
    	"logger_model_name": "Signature250",
    	"logger_serial_number": "1234567",
    	"offset_from_utc_hrs": 0.0,
    	"averaging_period_minutes": 10
    }]
   }

4. Column names:
   1. Suggested format: <sensor_type>__<serial_number>__<measurement_type>__<height_m>__<measurement_units>__<statistic_type>
      Examples:
      1. lidar__1234__wind_speed__110__m/s__avg
      2. lidar__1234__wind_speed__110__m/s__max
      3. adcp__4321__water_speed__-23__m/s__avg
      4. thermometer__null__water_temperature__-1__deg_C__avg
   2. Quality flags should be considered as a lidar could provide temperature and pressure values and a quality data column could relate to either the temperature or pressure as they would both be at the same height. Similar if it is from Airmar weather station sensor where it can output several measurement types where each could have a quality flag. The column name structure we have come up with would not be unique for each of the pressure, temperature, wind speed, wind direction measurements. lidar__1234__quality__1__%__avg is this for the temperature or pressure from the lidar which are both at 1 m height?
      1. lidar__1234__quality__1__%__avg__temperature tag on the end the measurement type the flag/quality column relates too?
      2. Maybe we could think about it like how Windographer flags data in that they duplicate the column name and tag on flag at the end. So each column name in the timeseries can have it's own flag e.g. thermometer__null__air_temperature__1__deg_C__avg__flag.
      3. Instead of tagging flag on the end, a possible solution is to allow a notes on the end. This is effectively a free field to allow the OEM to include whatever they want: <sensor_type>__<serial_number>__<measurement_type>__<height_m>__<measurement_units>__<statistic_type>__<notes>. This gives the OEM the option to use lidar__1234__quality__110__%__avg for the wind speed, vertical wind speed and wind direction values or use lidar__1234__air_temperature__1__deg_C__avg__flag for other sensors.
      4. It would be good to dwell on this and make a decision when more OEM representation is on the call.
   3. What about calculated column names? For example air density can be calculated by two different sensors.
      1. We could have in the sensor enum a calc option to indicate that this column of data is calculated from multiple other sensors or use other or null which are already options in the data model? Using calc might also solve another issue we have for soiling loss calculation in solar. Examples:
         1. calc__null__air_density__2__kg/m^3_avg
         2. other__null__air_density__2__kg/m^3_avg
         3. null__null__air_density__2__kg/m^3_avg
      2. Agreed to go with adding calc to the sensor_type enum.
      3. If there are 2 air density calculations derived at the same height from two different temperature and pressure sensors, how do we make them unique? Concatenate the serial numbers or just use a slightly different height?
         1. calc__temp1234_press4321__air_density__2__kg/m^3__avg
         2. calc__null__air_density__2.0__kg/m^3__avg
         3. calc__null__air_density__2.1__kg/m^3__avg
         4. Another option, if we allow 'notes' on the end as discussed above, could be calc__null__air_density__2.0__kg/m^3__avg__A and calc__null__air_density__2.0__kg/m^3__avg__B
         5. This is more of a choice for the FLS OEM as to which option they want to go with. They could use the <serial_number> to provide a derived identifier either instead of using the area.
   4. What about a battery voltage column names where we may have difficulty to distinguish which battery is associated with which logger/device/system?
      1. Could we use the serial number of the device that is linked to that battery e.g. use the lidar serial number for the lidar battery?
         1. voltmeter__1234__voltage__0__V__avg
      2. Again, maybe the field on the end can help distinguish which device/system this battery voltage relates too. Or again the <serial_number> could be used to distinguish.
      3. Only briefly touched on this in the call. Continue on next call.
      4. Also consider how we measure the fuel by possibly adding a fuel_level option in the measurement_type enum. This would also require a fuel_gauge in the sensor_type enum.
5. Meeting change date; 28th Nov is our Christmas party and Thanksgiving. Push it to the 5th Dec. and then break for Christmas?
   1. As we didn't get through all the issues with the column name we will pull forward the 28th to the 21st Nov and also have one on the 5th Dec to try and get 2 more in before we break for Christmas.

Actions

• [ ]

[stephenholleran]

21-Nov-2024, 5th Meeting

Attendance:
Stephen Holleran (BrightWind), Salma Yahiaoui (Akrocean), Anselme Troiville (Green Rebel), Jose Miguel Garro (Eolos), Ryan Scott (RWE), Matthew Linham (BP), Gibson Kersting (RWE) [7]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. Column names:
   1. How to include quality flags as discussed in the previous meeting?
      1. The suggestion is to include 'notes' as an optional item at the end of the filename. Therefore a flag label could be added for each measurement column name. E.g. lidar__1234__air_temperature__1__deg_C__avg and lidar__1234__air_temperature__1__deg_C__avg__flag.
      2. <sensor_type>__<serial_number>__<measurement_type>__<height_m>__<measurement_units>__<statistic_type>__<notes>
      3. This 'notes' can also be used as an option for when there are duplication of measurements at the same height but for different systems, e.g. distinguish between battery for lidar or battery for navigation lights.
      4. 'Notes' are all over the data model already as optional fields so probably reasonable to give this option here too. It could also cover any unforeseen scenarios.
      5. We think adding the 'notes' is a good option to include. Another example for its use came up in the discussions was to use it to distinguish between different 'wave_period' that are calculated from the same sensor but using a different algorithm. The column name without the 'notes' optionality would be a duplicate so the free field text of the notes could be used to distinguish between the methods of calculation.
   2. Add fuel_level option to the measurement_type enum and fuel_gauge in the sensor_type enum? Yes, do this.
   3. Should we standardise the timestamp column name?
      1. Options could be:
      2. Timestamp
      3. timestamp
      4. DateTime
      5. date_time
      6. Yes. We will go with 'timestamp', all lower case.
      7. The timestamp format should be ISO 8601 with the 'T' required, similar to the WRA Data Model, but with no time zone. E.g. 2024-12-12T13:40:00. Agreed.
4. File Name and Extension:
   1. For file extension I would keep this simple and use a file extension of .csv even though it is technically not a csv? It could be .txt, .dat or ??? Yes, we should standardise the file extension. We decided on '.csv', lower case.

Actions

• [ ]

[felixkelberlau]

Thanks to all for working on this yesterday. The meeting time makes it difficult for me to always join the calls.

I think for the column header names the most important parameter should come first because in many situations the long names will be displayed shortened and in alphabetical order. Thus, I am wondering if the order with <sensor_type> first is the best. Is <measurement_type> first maybe advantageous?

Thinking of how measurement parameters are typically sorted in reporting, measurement plans and tender documents, it appears to me that wind data, current data, meteorological parameters and so on are grouped together. So, I think that <measurement_type> in the first position will lead to more intuitive sorting. It will also keep identical/redundant measurements closer together.

When working with vertical profile data, I find it very convenient when the same type of measurements from each height are grouped together instead of being spread out over the file. Thus, I would keep <measurement_type> and <statistic_type> together. In this way, we will get one block of all mean wind speeds and so on. I know that this is different from ZPH and STA output but would you agree that it is more practical when working with data?

I also think that it is customary to have the <meaurement_units> at the end of a parameter name.

So, to summarize I am suggesting the following order:
<measurement_type>__<statistic_type>__<height_m>__<sensor_type>__<serial_number>__<measurement_units>__<notes>

[stephenholleran]

@felixkelberlau thanks very much for this and sorry for not replying sooner.

I think these are great suggestions. I will include them for discussion on today's call.

[stephenholleran]

@felixkelberlau
Thanks again for your suggestion. I have been thinking about it in prep for todays call and I am wondering what your thoughts are on swapping the height and statistic type around?

So instead of:
wind_speed__avg__110__lidar__1234__m/s
wind_speed__avg__120__lidar__1234__m/s
wind_speed__sd__110__lidar__1234__m/s
wind_speed__sd__120__lidar__1234__m/s

It would be:
wind_speed__110__avg__lidar__1234__m/s
wind_speed__120__avg__lidar__1234__m/s
wind_speed__110__sd__lidar__1234__m/s
wind_speed__120__sd__lidar__1234__m/s

You still get your requirement to group all the mean wind speeds together but I am thinking that the height is very useful info to have up the pecking order.

[stephenholleran]

@felixkelberlau sorry, I was completely wrong in what I wrote above. The 'avg' wouldn't be grouped together.

[stephenholleran]

5-Dec-2024, 6th Meeting

Attendance:
Stephen Holleran (BrightWind), Anselme Troiville (Green Rebel), Jose Miguel Garro (Eolos), Salma Yahiaoui (Akrocean) [4]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. Column names:
   1. Instead of this structure <sensor_type>__<serial_number>__<measurement_type>__<height_m>__<measurement_units>__<statistic_type>__<notes> Felix is suggesting <measurement_type>__<statistic_type>__<height_m>__<sensor_type>__<serial_number>__<measurement_units>__<notes>? Examples:
      1. wind_speed__avg__110__lidar__1234__m/s
      2. wind_speed__avg__120__lidar__1234__m/s
      3. wind_speed__sd__110__lidar__1234__m/s
      4. wind_speed__sd__120__lidar__1234__m/s
      5. air_density__avg__2__calc__null__kg/m^3
      6. air_temperature__avg__1__thermometer__null__deg_C
   2. Or we could swap the height and statistic_type around to bring up the height.
      1. wind_speed__110__avg__lidar__1234__m/s
      2. wind_speed__110__sd__lidar__1234__m/s
      3. wind_speed__120__avg__lidar__1234__m/s
      4. wind_speed__120__sd__lidar__1234__m/s
   3. Changing the ordering would split out the sensors from what measured them. For example, measurements coming from a lidar are all grouped together and ADCP are grouped together. However, we do understand where Felix is coming from and these will mostly be read by machines so it doesn't really matter what the order is. But it does help when doing some ad-hoc analysis in Excel or something so we are good to go with measurement_type first. We are ok with the statistic_type before the height so this wind_speed__avg__120__lidar__1234__m/s.
4. File Name and Extension:
   1. We decided that the file extension should be '.csv', lower case.
   2. Do we standardise the file name? It would be good to propose a best practice at least.
   3. Options could be:
      1. <station_serial_number>__<date_from>__<date_to>.csv
      2. <oem_name>__<station_serial_number>__<date_from>__<date_to>.csv
      3. <oem_name>__<station_serial_number>__<date_from>__<date_to>__<averaging_period_minutes>.csv
      4. <oem_name>__<station_serial_number>__<date_from>__<date_to>__<notes>.csv
      5. <oem_name>__<station_name>__<station_serial_number>__<date_from>__<date_to>__<notes>.csv
      6. <oem_name>__<station_name>__<station_serial_number>__<client_name>__<date_from>__<date_to>__<notes>.csv
      7. Fugro__Site X__FLS 01__2024-01-01_00_00_00__2024-01-31_23_50_00__LidarData.csv
      8. Eolos__Site X__FLS 01__2024-01-01_00_00_00__2024-01-01_23_50_00__10min.csv
      9. Do we include the client name? Client names can change and it really doesn't give the consumer any useful info to identify what measurement station this relates too. Don't want a very long name as, along with the folder path, we could exceed the Windows max number of characters. In saying that, it is mostly the folder path that causes the issues and not the filename and so we then think that option 'e' above is the one to go with <oem_name>__<station_name>__<station_serial_number>__<date_from>__<date_to>__<notes>.csv. We don't need the client as there may be more than one client and it has no impact on the data.
   4. Where date_from and date_to are the first and last timestamps found in the file? Yes
      1. Include the time or just the date? There may be multiple files that contain chunks of the same day and therefore would be a duplicate filename. Note; can't use ':' in a file name => 2024-01-01_235000 or 2024-01-01_23_50_00 or 2024-01-01_2350. We should include the time component to avoid duplicates. Other options are '20240101_235000', `2024-01-01_23-50-00'. We chose to go with '2024-01-01_23_50_00'.
      2. Should the date_to be optional? Maybe not as a daily file for the first of the month would have the same filename as a monthly file for that same month. No, we need a date_to.
      3. Could include 'to' between dates, e.g. 2024-01-01__to__2024-01-31, to make it more obvious what the dates mean? No, this is not necessary.
5. We will need to trial this in anger. How shall we go about doing that?
   2. SH to create a folder in the repo to capture all of the above decisions.
   3. Within this create a readme and a guidelines of use which would capture the best practice use of this.
   4. As well as a sample file, maybe a JSON Schema for the header too.
   5. All OEMs to send me (Stephen) a sample file as it exists at the moment. I will then pull out as many different column names and put them into this new format. This can then be used as a guide for the OEMs to try it out for themselves.
6. How is this to be packaged?
   1. Be part of the WRA Data Model repo where it lives in a dedicated folder or it's own separate repo? Can answer this at a later date.
7. Next meeting, 16th or 23rd January? Go with the 23rd to give me more time. Agenda will just be reviewing the sample file I created.

Actions

• [ ]

[stephenholleran]

23-Jan-2025, 7th Meeting

Attendance:
Stephen Holleran (BrightWind), Anselme Troiville (Green Rebel), Gibson Kersting (RWE), Jon Upton (Shell), Niko Simisiroglou (RWE), Léa Rolland (Akrocean), Scott Wylie (GreenRebel) [7]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. SH putting together a sample file and creating the documentation. We will review this PR: Floating lidar file format #271
   1. Folder in the repo to capture all of the above discussions. OK to have a subfolder in the WRA Data Model repo and its name is ok.
   2. In the Intro, list the companies involved.
   3. Dates in file name, could we use 'T' instead of the underscore? Forgot that the colon (:) can't be used in the file name as Windows doesn't allow : and so therefore it can't follow ISO 8601 anyway. However it was felt that the T does offer a distinction between the date and time parts and is not confused with the underscore separator of the individual time parts therefore it would be good to use. E.g. 2024-01-01T00_00_00.
   4. Put explanation of the double underscore for the file naming.
   5. Outline that users should use Window compatible characters in the file name.
   6. Explain in the header part more where this JSON Schema came from i.e. from the Data Model with unnecessary parts deleted.
   7. In the header section we would probably get away with not explaining all terms as the users will be familiar with JSON Schema and we can link to the WRA Data Model docs.
   8. In the notes variable, specify not to use double underscores.
   9. Got through most of the example column names. Need to revisit 'packets in avg' counter or quality columns.

Actions

• Continue working on a first draft of the ReadMe.
• Link up with floating lidar OEMs to come up with unusual column names.

[stephenholleran]

06-Feb-2025, 8th Meeting

Attendance:
Stephen Holleran (BrightWind), Anselme Troiville (Green Rebel), Gibson Kersting (RWE), Salma Yahiaoui (Akrocean), Ryan Scott (RWE), Mona Senelier (Vaisala) [6]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. SH putting together a sample file and creating the documentation. We will review this PR: Floating lidar file format #271
   1. Felix via email: "height_m: If not discussed before, I would like to encourage a discussion on whether the entries should have a leading zero, e.g., “80” or rather “080”. Without a leading zero alphabetical ordering ruins the order in ascending heights."
      1. Other examples e.g. -5 to -005, -5.5 to -005.5, 32 to 032.
      2. What would happen if we measure over 1,000 m? Could be possible, therefore there would be 4 digits required.
      3. We don't think this can be a requirement however what we can do is write a recommendation that using leading zeros will organise the column names better alphabetically. Give example of 4 digits.
   2. Felix via email: "measurement_units: Taking Matlab as an example, only letters, digits, and underscores are allowed as variable names. There are workarounds but it might be useful to accept this limitation and avoid “/” and “%” in our definition. So, is “ms” or “mps” maybe better than “m/s”?"
      1. SH; these units are an enum in the main data model so would be more difficult to change. We also try to keep to SI units as much as possible where "m/s" is the SI unit.
      2. Decided to keep them as is.
   3. Sensor can be a null error in the read me.
   4. voltage__avg__2__voltmeter__null__V__fog horn is ok to use to distinguish between voltage measured from a fog horn verses an aviation light or communications system.
   5. counter__count__120__lidar__1234__null__packets in avg and quality__quality__null__lidar__1234__%__proportion of packets with rain are ok examples of using the notes section to capture the 'packets' measured by ZX lidars.
   6. Anselme will come up with an example for an Aquadopp (a horizontal ADCP).
   7. Didn't get to discuss pitch, roll, yaw or heading.

Actions

• Continue working on a first draft of the ReadMe.
• Anselme will come up with an example for an Aquadopp (a horizontal ADCP).

[stephenholleran]

20-Feb-2025, 9th Meeting

Attendance:
Stephen Holleran (BrightWind), Jose Miguel Garro (Eolos), Anselme Troiville (Green Rebel), Scott Wylie (GreenRebel), Ryan Scott (RWE), Gibson Kersting (RWE), Jon Upton (Shell) [7]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. SH putting together a sample file and creating the documentation. We will review this PR: Floating lidar file format #271. Mostly working on the Column Names.
   1. Anselme has come up with an example for an Aquadopp (a horizontal ADCP). An Aquadopp is similar to a scanning or nacelle mounted lidar in wind. The data model can't yet handle scanning lidars and so the following is an example of how this floating lidar file format may be able to handle this Aquadopp data.
      1. water_speed__avg__-001__adcp__1234__m/s__meas_distance_3_m
      2. water_speed__avg__-001__adcp__1234__m/s__meas_distance_4_m
      3. water_speed__avg__-001__adcp__1234__m/s__meas_distance_5_m
      4. water_speed__avg__-001__adcp__1234__m/s__meas_distance_6_m
      5. water_speed__avg__-001__adcp__1234__m/s__meas_distance_7_m
      6. This is a good temporary solution until the data model gets better and can handle scanning lidars or side mounted ADCPs. Strongly recommend OEMs to use this format.
   2. What about pitch, roll, yaw, heading or bearing?
      1. Sensor type? The sensor could be one of many e.g. ADCP, IMU, so the pitch, roll is of the device. The CF Conventions has "platform_roll" and it says it covers the instruments. However, in floating lidar terms the "platform" would generally mean the buoy itself. So a bottom or side mounted ADCP's roll would be different to the buoy's roll. It would be best to just use "roll".
      2. Measurement type? Need to add 'roll', 'pitch', 'yaw', 'heading'.
      3. Units? Have deg already.
   3. From Anselme; what about 'biofouling sensor'? Metal plates mounted on the buoy. Sent off to a lab to measure. Therefore it is a once off dataset. Not a timeseries dataset so therefore no need to capture.
   4. Micro-wave profiler? Essentially a temperature profiler in the atmosphere.
      1. Sensor type? Add ``microwave_temperature_profiler` (MTP) https://airbornescience.nasa.gov/instrument/Microwave_Temperature_Profiler
   5. Bird/bat monitoring (data would be animal count)? Jon to try and find out about the data.
   6. Hydro-phonic sensor (data would be sound files, e.g. from marine animals)? These are sound files and have their own format so not really appropriate for this file format.
   7. Turbidity sensor / water quality (dissolved oxygen)?
      1. Sensor type? Add nephelometer https://en.wikipedia.org/wiki/Nephelometer
      2. Measurement type? Add turbidity, refer to CF Conventions but drop the 'sea_water' as floating lidar can be placed in lakes and rivers.
      3. Units? Dimensionless
   8. Visibility
      1. Sensor type? visibility_sensor, do a bit more research
      2. Measurement type? CF Conventions has visibility_in_air. Stick with the CF Conventions in this case as turbidity covers visibility in water.
      3. Units? m, so we have this already.
   9. "Wave Spectra" talk about on next call.

Actions

• Continue working on a first draft of the ReadMe.

[felixkelberlau]

I have a comment regarding ii.)
pitch, roll, yaw (, heading, bearing) require careful definitions.

The coordinate system (E,N,D, relative to true north or platform axes) and counting order need to be defined (https://en.wikipedia.org/wiki/Euler_angles).

In addition, it must be clear how ten minute statistics are calculated. The arithmetic mean of the pitch or roll angle for an FLS will usually be close to zero and is an indicator for static tilt of the platform. The mean of their absolute value might be already more interesting as it describes the average inclination in each direction. Maybe good to also include the variance of the roll/pitch angles to show what proportion is static and dynamic? Especially for axis-symmetric systems the mean tilt angle (as the quadratic sum of roll and pitch before averaging) and its direction can be the most significant parameters of all.

I am not aware of any widely-accepted guidance on this and I believe that without definitions, each OEM will provide different orientation data.

[stephenholleran]

20-Mar-2025, 10th Meeting

Attendance:
Stephen Holleran (BrightWind), Hanna Torrens-Spence (BrightWind), Ryan Scott (RWE) [3]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. SH putting together a sample file and creating the documentation. We will review this PR: Floating lidar file format #271. Mostly working on the Column Names.
   1. Review horizontally mounted ADCP example with wording.
   2. Review pitch, roll, yaw, and heading additions. Taking into account Felix's comments above. PR Disc #268 additions from floating lidar #282.
      1. The definition suggested is basically from the CF Conventions which is "pitch (from CF Conventions for 'platform_pitch') is a rotation about an axis that is perpendicular to both the local vertical axis and the nominal forward motion direction of the sensor. Pitch is relative to the 'at rest' rotation of the sensor with respect to the axis of rotation. The 'at rest' rotation of the sensor may change over time." The reference frame is the sensor itself. @felixkelberlau does this satisfy your concern regarding the coordinate system? The analyst would need to know how the sensor was actually mounted on the buoy. We are not sure if or how we could capture all of that information here in the data model.
   3. Are pitch, roll and yaw enough? Do FLS OEMs also provide Up-Down (Heave), Left-Right (Sway), Forward-Back (Surge) from the 6 degrees of freedom (https://en.wikipedia.org/wiki/Six_degrees_of_freedom)
   4. Review microwave_temperature_profiler addition.
   5. Bird/bat monitoring (data would be animal count), update from Jon? Jon not on call.
   6. Review nephelometer addition.
   7. Visibility "Sensor type" needed more research. I didn't get a chance to do this.
   8. "Wave Spectra"?
      1. Sensor type?
      2. Measurement type? Distribution of wave energy across different frequencies, directional or non-directional. Not sure if this is 'wave spectral density' also known as 'wave energy spectrum'. If it is in the frequency domain therefore it is derived from a timeseries signal after a period of time. What would that period of time be? How would it be represented? Come back to this.
      3. Units?
   9. Should wave_buoy be a sensor? An IMU may be attached to a buoy however there may be post-processing to convert the IMU data into the reference frame of the wave_buoy. Therefore, should wave_buoy be a sensor too? Should it be more generic such as wave_sensor? Come back to this.
   10. Should we define the methods of calculation for wave periods or other measurements? Could the statistic_type be used to distinguish between them? Similar to ti30sec that already exists. E.g. 'sea_surface_wave_mean_period_from_variance_spectral_density_first_frequency_moment' from the CF Conventions which is 'Tm01' (I think?). Is the statistic_type tm01 for the existing wave_period? Or is this a different measurement_type of spectral_mean_wave_period? Again, come back to this.

Actions

• Continue working on a first draft of the ReadMe.
• Research visibility sensor.

[felixkelberlau]

Hi Stephen and all, Thanks for sharing the minutes of yesterday’s meeting. To me the CF Conventions don’t really fit because for FLS because they are referring to a moving vessel that has a “forward motion”. I am supportive of defining the reference frame to be the motion sensor itself, then maybe define roll, pitch, and yaw/heading accordingly and suggest that the mounting orientation is defined in the comments. Location on the FLS does not matter for the measurements, only its orientation. Having defined roll, pitch and yaw/heading makes tilt_x, tilt_y, and tilt_z obsolete. (roll=tilt_x…). Alternatively or in addition, roll and pitch could be combined into tilt, i.e., tilt is the inclination from the vertical (tilt is quadratic sum of roll and pitch). You added yaw and heading, which in my understanding should be identical and it would be redundant to use both parameters (yaw=heading=tilt_z). Still, a definition is needed for how e.g., 10-minute averages of roll are calculated from the unaveraged samples. Imaging the unaveraged time series of roll data to be a sinusoidal curve with a certain amplitude. Averaging them would lead to zero, unless the buoy is tilted in the mean, using the maximum value would give the strongest tilt in roll direction amplitude, taking the standard deviation would be a reasonable alternative. This is my main point, that everyone understands something else when getting a ten-minute averaged roll value. 3.3: Yes, roll, pitch, and yaw are enough. For ten minute data surge,sway, and heave are typically not provided. Best, Felix Felix Kelberlau Floating Lidar Product Lead Fugro M +47 40 59 59 22 | E ***@***.******@***.***> | W fugro.com A Pirsenteret, Havnegata 9, P.O.Box 1224 Torgard, 7462 Trondheim, Norway Fugro Norway AS | Registration number: 969 059 789 | VAT: NO 969 059 789 MVA Together we create a safe and liveable world. From: stephenholleran ***@***.***> Sent: Thursday, March 20, 2025 7:32 PM To: IEA-Task-43/digital_wra_data_standard ***@***.***> Cc: Kelberlau, Felix ***@***.***>; Mention ***@***.***> Subject: Re: [IEA-Task-43/digital_wra_data_standard] Floating Lidar File Format Standardization Development - 2024 (Discussion #268) 20-Mar-2025, 10th Meeting Attendance: Stephen Holleran (BrightWind), Hanna Torrens-Spence (BrightWind), Ryan Scott (RWE) [3] Minutes: 1. Quick intros (if needed). 2. Recap on previous call. 3. SH putting together a sample file and creating the documentation. We will review this PR: #271<#271>. Mostly working on the Column Names. * Review horizontally mounted ADCP example with wording. * Review pitch, roll, yaw, and heading additions. Taking into account Felix's comments above. PR #282<#282>. * The definition suggested is basically from the CF Conventions which is "pitch (from CF Conventions for 'platform_pitch') is a rotation about an axis that is perpendicular to both the local vertical axis and the nominal forward motion direction of the sensor. Pitch is relative to the 'at rest' rotation of the sensor with respect to the axis of rotation. The 'at rest' rotation of the sensor may change over time." The reference frame is the sensor itself. @felixkelberlau<https://github.com/felixkelberlau> does this satisfy your concern regarding the coordinate system? The analyst would need to know how the sensor was actually mounted on the buoy. We are not sure if or how we could capture all of that information here in the data model. * Are pitch, roll and yaw enough? Do FLS OEMs also provide Up-Down (Heave), Left-Right (Sway), Forward-Back (Surge) from the 6 degrees of freedom (https://en.wikipedia.org/wiki/Six_degrees_of_freedom) * Review microwave_temperature_profiler addition. * Bird/bat monitoring (data would be animal count), update from Jon? Jon not on call. * Review nephelometer addition. * Visibility "Sensor type" needed more research. I didn't get a chance to do this. * "Wave Spectra"? * Sensor type? * Measurement type? Distribution of wave energy across different frequencies, directional or non-directional. Not sure if this is 'wave spectral density' also known as 'wave energy spectrum'. If it is in the frequency domain therefore it is derived from a timeseries signal after a period of time. What would that period of time be? How would it be represented? Come back to this. * Units? * Should wave_buoy be a sensor? An IMU may be attached to a buoy however there may be post-processing to convert the IMU data into the reference frame of the wave_buoy. Therefore, should wave_buoy be a sensor too? Should it be more generic such as wave_sensor? Come back to this. * Should we define the methods of calculation for wave periods or other measurements? Could the statistic_type be used to distinguish between them? Similar to ti30sec that already exists. E.g. 'sea_surface_wave_mean_period_from_variance_spectral_density_first_frequency_moment' from the CF Conventions which is 'Tm01' (I think?). Is the statistic_type tm01 for the existing wave_period? Or is this a different measurement_type of spectral_mean_wave_period? Again, come back to this. Actions * [ ] Continue working on a first draft of the ReadMe. * [ ] Research visibility sensor. — Reply to this email directly, view it on GitHub<#268 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BAWO6FV6BZJ44EY7RPULDGD2VMCTXAVCNFSM6AAAAABOQAWIJCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENJWG44TEOA>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[stephenholleran]

Thanks @felixkelberlau for your reply.

1. Instead of “forward motion” that the CF Convention uses, is there another term that describes the front of a buoy? There must be a point defined as the front so wind direction adjustments could be made. Maybe it is like the dead band in a wind vane.
2. Roll vs tilt_x: are you suggesting that we can remove tilt if we have roll? For onshore lidars there can be a tilt_x and tilt_y in the timeseries. This is where these came from. There is also tilt used for solar. So we would need to keep them but I think all offshore folks would use roll.
3. Yaw vs heading: you are correct. We can remove heading if it would cause confusion? I thought heading could be useful in maybe some other navigation context offshore.
4. Definition of how 10-min roll is calculated: do you provide a 10-min roll or pitch? Sounds like you might not but maybe provide it in another guise. I understand it would average to zero which isn't very useful. What these measurement_type's do is just say what is been measured. It doesn't actually specify if it is an average, max or standard deviation. That can be covered by the statistic_type. E.g. an anemometer with a typical onshore logger (e.g. Ammonit, NRG) will provide the average 10-min wind speed but also the max, min and standard deviation. The measurement_type is wind_speed where each data column will also have a specific statistic_type to say if it is the max or min. So you could provide a roll with only a column of data with standard deviation roll__sd__-1__adcp__1234__deg.

Thanks,
Stephen

[felixkelberlau]

Thanks, Stephen. Here my response:

1. I guess "bow" would be the term that describes the front of a vessel. We call it "buoy's north" although it will hardly ever align with true north. To generalize, I suggest to call it the first principal axis of the FLS. With this, the definition of pitch becomes "pitch (from CF Conventions for 'platform_pitch') is a rotation about an axis that is perpendicular to both the local vertical axis and the first principal axis of the sensor."
2. Yes, roll is the same as tilt_x and only one should be included.
3. Agreed to keeping only one of yaw or heading.
4. I am fine with using roll and pitch as measurement_type plus statistic_type as you are suggesting.

[stephenholleran]

03-Apr-2025, 11th Meeting

Attendance:
Stephen Holleran (BrightWind), Anselme Troiville (Green Rebel), Scott Wylie (Green Rebel), Salma Yahiaoui (Akrocean), Ryan Scott (RWE), Hanna Torrens-Spence (BrightWind) [6]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. SH putting together a sample file and creating the documentation. We will review this PR: Floating lidar file format #271. Mostly working on the Column Names.
   1. Updated the "special cases" examples of the column name to give more explanations to the examples. https://github.com/IEA-Task-43/digital_wra_data_standard/tree/conv268_floating_lidar_file_format/floating_lidar_file_format
   2. Review pitch, roll, yaw, and heading additions. Taking into account discussion with Felix above.
      1. Definition suggested "pitch (from CF Conventions for 'platform_pitch') is a rotation about an axis that is perpendicular to both the local vertical axis and the nominal forward motion direction of the sensor. If the sensor is the buoy itself, then the 'forward motion direction' can also be known as the 'first principal axis' or the 'buoy's north'. Pitch is relative to the 'at rest' rotation of the sensor with respect to the axis of rotation. The 'at rest' rotation of the sensor may change over time." The reference frame is the sensor itself.
      2. @felixkelberlau how does this sound?
      3. Those on the call agreed this is acceptable.
   3. Are pitch, roll and yaw enough? Do FLS OEMs also provide Up-Down (Heave), Left-Right (Sway), Forward-Back (Surge) from the 6 degrees of freedom (https://en.wikipedia.org/wiki/Six_degrees_of_freedom)?
      1. Felix says "Yes, roll, pitch, and yaw are enough. For ten minute data surge,sway, and heave are typically not provided."
      2. Agreed.
   4. Yaw vs heading; keep or drop heading?
      1. Felix "Agreed to keeping only one of yaw or heading.". Is heading used in other situations?
      2. Heading seems to be mostly used in the industry so we can drop yaw instead of heading. @felixkelberlau is this ok with you?
   5. roll vs tilt_x; need to keep tilt_x as it is used in solar and onshore lidars. roll would more commonly be used in offshore so we need to keep both. Agreed.
   6. Bird/bat monitoring (data would be animal count), update from Jon? Jon not on the call.
   7. visibility_in_air "Sensor type" needed more research.
      1. It is a "Transmissometer" http://en.wikipedia.org/wiki/Transmissometer . In FLS applications it would be a forward scatter visibility sensor that would be used. They are also commonly referred to as "visibility sensors". Agreed to just add transmissometer. We can add visibility_sensor later if needs be.
      2. visibility_in_air seems to also be know as "meteorological observable range" but I think we should stick with the CF Conventions. Agreed.
   8. Before getting into discussions on the below suggestions to add to the WRA Data Model it is worth noting that these do not impact the purpose of this particular discussion group in developing a floating lidar file format. Therefore, I think we should conclude the discussion on the file format and separate out discussions on these extra variables to add to the data model. These additional measurement types, sensors, units, etc. won't have any impact on the file format.
      1. Yes, agreed to close off the file format. The PR is https://github.com/IEA-Task-43/digital_wra_data_standard/pull/271/files . Would be good to review individually. Send an email saying that the call in 2 weeks time, 17th Apr, will be the day to do a final review of the file format and therefore would be good to have reviewed the PR in your own time leading up to that.
   9. "Wave Spectra"? Continue discussion from the 20th March. Didn't do.
      1. Sensor type?
      2. Measurement type?
      3. Units?
   10. Should wave_buoy be a sensor? Continue discussion from the 20th March.
       1. 
       2. A wave buoy like the one above just has an IMU inside it so the sensor should be an IMU. It could also have a GPS, a thermometer, etc. so all these sensors would just be wave_buoy. However, the documentation from the wave buoy doesn't really list the sensors or give serial numbers for each of the sensors on the wave buoy resulting in null serial numbers in the column name. It is a bit like a temperature sensor built into a lidar, one can just say that the lidar is the sensor. Therefore, we agreed that there is no harm adding wave_buoy to the sensor enum similar to lidar and sodar.
   11. Should we define the methods of calculation for wave periods or other measurements? Continue discussion from the 20th March. Didn't do.

Actions

• Finalise file format PR.
• Extend calendar invite and send email explaining final review of the file format will be the 17th.
• Add wave_buoy as a sensor_type.
• Add transmissometer as a sensor_type.
• Add visibility_in_air as a measurement_type.
• Update definitions for pitch, roll and heading.
• Drop yaw.

[stephenholleran]

Hi All,

We have made some great progress on the floating lidar file format. During the discussions on testing the suggested column name format lots of new measurement types, sensors, units have been suggested and added. These conversations are ongoing, however any new additions there won't have an impact on closing out the new floating lidar file format.

Therefore, we are now in a position to review the final Pull Request on GitHub that contains all the information for the floating lidar file format. This Pull Request is available here: #271

For those not familiar with GitHub you can view the files under the tab "Files Changed". The most important file to review is the "README.md" as this contains all the information required to create or read a file following this standard. To make it easier a rendered view of this file is at this link https://github.com/IEA-Task-43/digital_wra_data_standard/blob/conv268_floating_lidar_file_format/floating_lidar_file_format/README.md

Our next catch up call is Thur 17th April. It would be great to have comments back before this call so we can discuss and resolve them as a group.

Thanks very much for all your contributions so far. Looking forward to getting comments back.

Thanks,
Stephen

[stephenholleran]

17-Apr-2025, 12th Meeting

Attendance:
Stephen Holleran (BrightWind), Anselme Troiville (Green Rebel), Ryan Scott (RWE), Salma Yahiaoui (Akrocean) [4]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. File format PR:
   1. SH completed putting together a sample file and creating the documentation. We will review this PR: Floating lidar file format #271.
   2. The whole call was taken up with reading through the Read Me in the PR and made some minor edits.
   3. We will give some more time for people to review as WindEurope was on last week. We aim to get at least two people from different companies to comment on the PR with their approval. The intention is to merge the PR on the call in 2 weeks.
4. v v v v v We didn't get to look at any of these. We will continue with these discussions after the PR is merged. v v v v v
5. Additional measurement types, sensors, units, etc. to the core data model:
   1. Bird/bat monitoring (data would be animal count), update from Jon? Jon not on the call.
   2. "Wave Spectra"? Continue discussion from the 20th March. Didn't do.
      1. From the 20th "Distribution of wave energy across different frequencies, directional or non-directional. Not sure if this is 'wave spectral density' also known as 'wave energy spectrum'. If it is in the frequency domain therefore it is derived from a timeseries signal after a period of time. What would that period of time be? How would it be represented? Come back to this."
      2. Sensor type?
      3. Measurement type?
      4. Units?
   3. Should we define the methods of calculation for wave periods or other measurements? Continue discussion from the 20th March. Didn't do.
      1. From the 20th "Could the statistic_type be used to distinguish between them? Similar to ti30sec that already exists. E.g. 'sea_surface_wave_mean_period_from_variance_spectral_density_first_frequency_moment' from the CF Conventions which is 'Tm01' (I think?). Is the statistic_type tm01 for the existing wave_period? Or is this a different measurement_type of spectral_mean_wave_period? Again, come back to this."

Actions

• Clean up the edits we made during the call.

[stephenholleran]

01-May-2025, 13th Meeting

Attendance:
Stephen Holleran (BrightWind), Anselme Troiville (Green Rebel), Ryan Scott (RWE), Amit Bohara (Altosphere), Hanna Torrens-Spence (BrightWind) [5]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. File format PR:
   1. SH completed putting together a sample file and creating the documentation. We will review this PR: Floating lidar file format #271.
   2. Merged!!! 👏 🎉
4. Additional measurement types, sensors, units, etc. to the core data model:
   1. Bird/bat monitoring (data would be animal count), update from Jon? Jon not on the call.
   2. "Wave Spectra"? Continue discussion from the 20th March.

      1. From the 20th "Distribution of wave energy across different frequencies, directional or non-directional. Not sure if this is 'wave spectral density' also known as 'wave energy spectrum'. If it is in the frequency domain therefore it is derived from a timeseries signal after a period of time. What would that period of time be? How would it be represented? Come back to this."

      2. This can be provided on a rolling basis and so there can be a value, at each frequency, every 10/30/60-mins. Therefore, we should account for it. See below example.

      3. 

      4. The units of the values are m^2/Hz. This is because the wave height squared is proportional to the energy of the waves.

      5. Plot of the first row from the above table. Is my terminology correct (for my own understanding)?

      6. 

      7. The units seem ok to add. One issue is what is the measurement_type that represents these values? Another issue is how the Hz bins can be represented in the data model. For this we don't think the data model can handle it very well as the size of the bins are user specific however it might not need to handle them as the column name for these will be different. All these column names would be grouped under the one measurement_type. So the focus of the conversation was on what is the measurement_type.

      8. Is this more commonly referred to as the 'Power Spectra Density' (PSD) in the industry? In my limited research/knowledge this is the same as 'Wave Energy Spectrum' where PSD is a more general signal processing term. Which one is preferred?

      9. From the CF Conventions this seems to be the equivalent sea_surface_wave_variance_spectral_density. Is this correct?

      10. 

      11. There seems to be lots of wave spectral densities. Some with 'energy' and others with 'maximum'. Which is the correct one we should use to capture what is been captured by the values in the table above?

      12. Are there other spectra we should be capturing? This might be related to the other question below about 'Tm01'.

      13. Units? m^2/Hz

   3. Should we define the methods of calculation for wave periods or other measurements? Continue discussion from the 20th March. Didn't get to.
      1. From the 20th "Could the statistic_type be used to distinguish between them? Similar to ti30sec that already exists. E.g. 'sea_surface_wave_mean_period_from_variance_spectral_density_first_frequency_moment' from the CF Conventions which is 'Tm01' (I think?). Is the statistic_type tm01 for the existing wave_period? Or is this a different measurement_type of spectral_mean_wave_period? Again, come back to this."
5. Delay call on the 15th May by 1 week. Set up additional calls? Yes

Actions

• SH: Update invite and extend.

[stephenholleran]

22-May-2025, 14th Meeting

Attendance:
Stephen Holleran (BrightWind), Salma Yahiaoui (Akrocean), Matthew Linham (BP), Anselme Troiville (Green Rebel), Scott Wylie (Green Rebel), Hanna Torrens-Spence (BrightWind), Gibson Kersting (RWE) [7]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. File format PR:
   1. Any comments since merge? No.
4. Additional measurement types, sensors, units, etc. to the core data model:
   1. "Wave Energy Spectrum"? Continue discussion.
      1. Is this more commonly referred to as the 'Power Spectra Density' (PSD) in the industry? In my limited research/knowledge this is the same as 'Wave Energy Spectrum' where PSD is a more general signal processing term. Which one is preferred? Wave Energy Spectrum is preferred?
      2. From the CF Conventions this seems to be the equivalent sea_surface_wave_variance_spectral_density. Is this correct? From my investigations 'wave energy' and 'surface variance' can be interchangeable as energy is related to the variance (energy = ρg . variance). Therefore, the CF convention sea_surface_wave_variance_spectral_density is the same as wave_energy_spectrum. Agree? Yes, agree to use wave_energy_spectrum as the measurement_type. Relating back to the file format the 'notes' section of the column name can be used to identify the Hz bins.
      3. Do we need the 'maximum' ones that are listed in the CF conventions? Probably do. We probably need the maximum of the wave_energy_spectrum. This could possibly use the 'statistic_type' of max or it could be a measurement_type of it's own? We decided to use wave_energy_spectrum_maximum as this is the maximum value after averaging all the data for each Hz bin. Using the statistic_type would be confusing to explain. The CF Convention equivalent seems to be sea_surface_wave_energy_at_variance_spectral_density_maximum.
      4. By direction? Data can be presented as a table but it can also be presented as a time series. This is still the same measurement_type which is just split by direction. It was decided that we don't need a new measurement_type to cover direction. Again, the notes part of the column name would distinguish between direction bins however way the OEM specifies the direction bins.
   2. Should we define the methods of calculation for wave periods or other measurements? Continue discussion from the 20th March.
      1. From the 20th "Could the statistic_type be used to distinguish between them? Similar to ti30sec that already exists. E.g. 'sea_surface_wave_mean_period_from_variance_spectral_density_first_frequency_moment' from the CF Conventions which is 'Tm01' (I think?). Is the statistic_type tm01 for the existing wave_period? Or is this a different measurement_type of spectral_mean_wave_period? Again, come back to this."
      2. We could do either a new measurement_type or statistic_type, however there are so many different wave "periods" that we could end up going down a rabbit hole. We could though just pick the most common (Tm01, Tm02, Tz) and create measurement_types for these. For others, the user would have to use the notes with 'wave_period' to distinguish which method of calculation was used. Yes, let's do this. E.g. create new measurement_type of wave_period_first_frequency to represent Tm01 which has the equivalent CF Convention of sea_surface_swell_wave_mean_period_from_variance_spectral_density_first_frequency_moment.
      3. 

Actions

• SH: Add units of m^2/Hz.
• SH: Add measurement_types wave_energy_spectrum, wave_energy_spectrum_maximum.
• SH: Add measurement_types wave_period_first_frequency and similar for Tm02 and Tz.

[stephenholleran]

29-May-2025, 15th Meeting

Attendance:
Stephen Holleran (BrightWind), Gibson Kersting (RWE) [2]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. IEA Wind Task 43 AGM: 10th - 11th June, Colorado. https://iea-wind.org/task43/task-43-events/
4. File format PR:
   1. Any comments since merge? ****
5. Additional measurement types, sensors, units, etc. to the core data model:
   1. "Wave Energy Spectrum"
      1. Added what was decided last week.
      2. The CF Convention has units of "m2 s". So should we add these or is "m2/Hz" used all the time?
   2. Wave periods
      1. From last week we said wave_period_first_frequency was equivalent to sea_surface_swell_wave_mean_period_from_variance_spectral_density_first_frequency_moment however I think 'swell' shouldn't be in there are it is actually equivalent to sea_surface_wave_mean_period_from_variance_spectral_density_first_frequency_moment. This is the combination of 'swell' and 'wind' driven waves. Is this correct? ****
      2. What is Tz and what is it's CF Convention standard name? It is the mean wave period measured between the wave moving upward through the mean sea level instead of from crest to crest. There is no CF Convention specifically for this but there is an alias.
         
         Therefore, is there a need to define it for the data model?

Actions

• SH: Add measurement_type for Tz.

[stephenholleran]

05-Jun-2025, 16th Meeting

Attendance:
Stephen Holleran (BrightWind), Anselme Troiville (Green Rebel), Matthew Linham (BP), Gibson Kersting (RWE) [4]

Minutes:

1. Quick intros (if needed).

2. Recap on previous call.

3. IEA Wind Task 43 AGM: 10th - 11th June, Colorado. https://iea-wind.org/task43/task-43-events/

4. File format PR:

   1. Any comments since merge? No comments, yet.

5. Additional measurement types, sensors, units, etc. to the core data model:

   1. "Wave Energy Spectrum"
      1. Added what was decided 2 weeks ago. Briefly reviewed the definitions added.
      2. The CF Convention has units of "m2 s". So should we add these or is "m2/Hz" used all the time? The Channel Coastal Observatory reports their spectrum in "m^2s"
         
         (https://coastalmonitoring.org/realtimedata/?chart=104&tab=Wave%20Spectra&disp_option=). We should add this so users can choose. In CF Conventions they use a space between each unit (m^2 and s). This space could cause problems with the column name format and coding. It may be better to use a dot between them. This, I think, is still compliant with SI units. Investigate more but we are leaning towards the dot.
   2. Wave periods
      1. From 2 weeks ago we said wave_period_first_frequency was equivalent to sea_surface_swell_wave_mean_period_from_variance_spectral_density_first_frequency_moment however I think 'swell' shouldn't be in there and it is actually equivalent to sea_surface_wave_mean_period_from_variance_spectral_density_first_frequency_moment. This is the combination of 'swell' and 'wind' driven waves. Is this correct? Yes.
      2. What is Tz and what is it's CF Convention standard name? It is the mean wave period measured between the wave moving "upward" through the mean sea level instead of from crest to crest. There is no CF Convention specifically for this but there is an alias.
         
         This is good and it is ok to define that it is "upwards" in the definition to match the CF Conventions.
         

6. Next steps?

   1. Review the PR Disc #268 additions from floating lidar #282
   2. Once approved, merge into the dev branch.
   3. Do a release!

Actions

• SH: Add 'm^2.s' to units. Investigate if a dot should be added between them.
• ALL: review the PR Disc #268 additions from floating lidar #282
• SH: Prepare the release for next weeks call.
• SH: Add Pa (from outstanding issues)

[stephenholleran]

After doing some research the recommend practice would be to use an underscore (as Anselme suggested) to separate the meter seconds. Definitely shouldn't use the SI standard middle dot. A normal full stop can also cause problems if using libraries like pandas or even Excel. Joining them together, though may not be a problem in this example, could cause problems for say 'm s' as 'ms' can mean milliseconds. The safest option, though not great, is to use an underscore 'm_s' or in this case 'm^2_s'

[stephenholleran]

12-Jun-2025, 17th Meeting

Attendance:
Stephen Holleran (BrightWind), Anselme Troiville (Green Rebel), Matthew Linham (BP), Gibson Kersting (RWE), Hanna Torrens-Spence (BrightWind) [5]

Minutes:

1. Quick intros (if needed).
2. Recap on previous call.
3. IEA Wind Task 43 AGM: 10th - 11th June, Colorado.
   1. Gibson gave an update in the meeting of what we are doing with the floating lidar format. RES and Pattern have adopted it.
   2. There is a subgroup involved in producing open-source tools, WeDoWind that puts out challenges, there is a group working on Ontologies. These are interesting and overlap with what we are doing. Ontologies would be a natural next step for a JSON Schema to go.
4. Additional measurement types, sensors, units, etc. to the core data model:
   1. "Fluorometer" (from Anselme)
      1. Add Fluorometer (measuring intensity of light - fluorescence levels of organic matter)
      2. Mostly used for concentration of Chlorophill-a (in micrograms per liter), but other application as well
      3. Marine institute provide some data : https://erddap.marine.ie/erddap/tabledap/galway_obs_fluorometer.html
      4. Yes, lets add it.
      5. Sensor type is fluorometer.
      6. Measurement type mass_concentration. According to CF Conventions "mass_concentration_of_chlorophyll_a_in_sea_water". We won't try and capture all the different type of chlorophyll or any other matter a fluorometer can measure so we will just keep it as mass concentration.
      7. Units, ppm, ug/L (u is accepted as micro for micrograms per liter).
   2. Turbidity
      1. Add units FTU? https://www.kemtrak.com/the-difference-between-turbidity-units/#:~:text=NTU%20stands%20for%20%E2%80%9CNephelometric%20Turbidity,the%20instrument%20measures%20the%20sample.
      2. Yes, add NTU, FTU and FNU
   3. GPS_motion_unit (from Matt)
      1. Add gps_motion_unit or similar due to the Datawell DWR-G sensor which measures waves based on GPS motion and not like an IMU which uses accelerometers? https://www.sofarocean.com/products/spotter and https://datawell.nl/products/dwr-g-unit/
      2. Yes. Struggled with to find a standard name for this type of sensor. Settled on gps_motion_unit
5. Include the Pydantic work by RES in this release?
   1. We don't need to wait as we can do a release with the Pydantic model itself.
6. Next steps?
   1. Review the PR Disc #268 additions from floating lidar #282
   2. Do a release tomorrow (Friday the 13th !!).

Actions

• SH: add the above new items and msg reviewers when done.
• ALL: review the PR Disc #268 additions from floating lidar #282
• SH: Do a release.