Parameters used META 3.2 AVISO

[SusanaMSilva]

Hi!

I have been trying to reproduce the META 3.2 eddy tracking results to make sure that everything is working properly since my objective is to use the py-eddy-tracking on my ROMS model, but I am getting less tracked eddies when compared to the AVISO product.

I used the parameters below, as they were the ones referred in the manual:

 g = RegularGridDataset(grid_name, lon_name, lat_name)
g.lanczos_high_filter('adt', 700, order=1)
 
a, c = g.eddy_identification('adt','ugos',
    'vgos',  # Variables used for identification
    date,  # Date of identification
    0.002,  # step between two isolines of detection (m)
    pixel_limit=(5, 1000),  # Min and max pixel count for valid contour 
    shape_error=70,  # Error max (%) between ratio of circle fit and contour
)

Also, I'm using maps of ADT with a 0.25 resolution, which I think were the same used.
Do I need to add other parameters in the eddy identification? And also, can I change the minimum amplitude?

I then used the eddy traking.yaml:

PATHS:
 # Files produced with EddyIdentification
 FILES_PATTERN: /py-eddy-tracker/share/Anticyclonic_*.nc
 # Path to save outputs
 SAVE_DIR: 'py-eddy-tracker/share/'

# Number of consecutive timesteps with missing detection allowed
VIRTUAL_LENGTH_MAX: 3
# Minimal number of timesteps to considered as a long trajectory
TRACK_DURATION_MIN: 10

CLASS:
  MODULE: py_eddy_tracker.featured_tracking.area_tracker
  CLASS: AreaTracker

Can you help me with this?

Thank you,
Susana Silva

[AntSimi]

Before to compare tracking, did you have same identification?

[SusanaMSilva]

Hi,

In the manual it only says shape error ≤ 70%, amplitude ≤ 0.4cm, only one extremum, 5 ≤ Npixel ≤ 1000 (I guess that this Npixel limits the existence of one extreme?), but I don't know which shape error was used exactly in the META 3.2 identification.
Also, I don't know how can I modify the amplitude, if 0.4cm is not the default.

The ADT maps were filtered like it says in the META manual, using a lanczos_high_filter 1st order with a 700km cut off. If I use this function do I still have to subtract the result from the original maps? From what I understood, the result is already the filtered map.

So I guess there are some parameters that are different from those that I used and that's why I was wondering if you can tell me exactly the parameters used to obtain the results of META 3.2

Thank you again.

[AntSimi]

Sorry for my late answer
How did you compare your identification result with AVISO?
EddyQuickCompare could help you to do that, it will produce this type of output:

         nomatch      multi_match        low       intermediate       high          parent          twin          complex  
[ 0]   64.9% (2176)    4.1% (137)     14.8% (495)    10.4% (347)    5.8% (196)      1.8% (59)      1.9% (64)      0.4% (14)

if you produce same output high must be at 100%

identification parameter must produce close result with this arguments: if you take same input map

shape_error=70
step=0.002
pixel_limit=(5, 2000)
nb_step_to_be_mle=0
sampling=20
sampling_method="visvalingam"

[SusanaMSilva]

Hi!

I simply compared the tracks for both for different life times.
I also used maps of ADT for a delimited region not for the globe, maybe in the filtering step it could have a different result.

Can you tell me what is the EddyQuickCompare and how to use it?

Thanks again

[AntSimi]

Here an example of several run of identification with different parameter specification
In bash:

mkdir -p 01_default_detection
EddyId -v INFO src/py_eddy_tracker/data/dt_med_allsat_phy_l4_20160515_20190101.nc 20160515 \
    adt None None longitude latitude 01_default_detection
mkdir -p 02_detection_without_filtering
EddyId -v INFO src/py_eddy_tracker/data/dt_med_allsat_phy_l4_20160515_20190101.nc 20160515 \
    adt None None longitude latitude 02_detection_without_filtering --cut_wavelength 0
mkdir -p 03_detection_without_filtering_err_90
EddyId -v INFO src/py_eddy_tracker/data/dt_med_allsat_phy_l4_20160515_20190101.nc 20160515 \
    adt None None longitude latitude 03_detection_without_filtering_err_90 --cut_wavelength 0 --fit_errmax 90
EddyQuickCompare 0*/Anticyclonic*
EddyQuickCompare 0*/Cyclonic*

Produce this output, to compare eddies it use same method than describe here in figure 3. You could set threshold to change default one to qualify high/intermediate/low. A high score mean similar eddies. For each category we indicate percentage and absolute quantity in brackets.

• nomatch => score < 5%
• multi_match => several eddies (more than 2) in file could be associate with one in the other file
• parent => ones eddies in file associate with several in the other file
• twin => 2 eddies in file associate with the same in other file
• complex => there is no simple way to describe association

You could also look to options of EddyQuickCompare

[ref] 01_default_detection/Anticyclonic_20160515T000000.nc -> 60 obs
[0] 02_detection_without_filtering/Anticyclonic_20160515T000000.nc -> 48 obs
[1] 03_detection_without_filtering_err_90/Anticyclonic_20160515T000000.nc -> 51 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    30.0% (18)      1.7% (1)       13.3% (8)     55.0% (33)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 1]    28.3% (17)      1.7% (1)       13.3% (8)     56.7% (34)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]     12.5% (6)      2.1% (1)       16.7% (8)     68.8% (33)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 1]     15.7% (8)      2.0% (1)       15.7% (8)     66.7% (34)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ref] 01_default_detection/Cyclonic_20160515T000000.nc -> 63 obs
[0] 02_detection_without_filtering/Cyclonic_20160515T000000.nc -> 58 obs
[1] 03_detection_without_filtering_err_90/Cyclonic_20160515T000000.nc -> 63 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    25.4% (16)      3.2% (2)       11.1% (7)     60.3% (38)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 1]    22.2% (14)      4.8% (3)       11.1% (7)     61.9% (39)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    19.0% (11)      3.4% (2)       12.1% (7)     65.5% (38)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 1]    22.2% (14)      4.8% (3)       11.1% (7)     61.9% (39)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)

If use area options score comparison will be display in surface equivalent

EddyQuickCompare 0*/Anticyclonic* --area
EddyQuickCompare 0*/Cyclonic* --area

[ref] 01_default_detection/Anticyclonic_20160515T000000.nc -> 60 obs
[0] 02_detection_without_filtering/Anticyclonic_20160515T000000.nc -> 48 obs
[1] 03_detection_without_filtering_err_90/Anticyclonic_20160515T000000.nc -> 51 obs
          nomatch      5 <= low < 20    intermediate     40 <= high     multi_match        parent           twin          complex     
[ 0]  11.7% (0.0Mkm²)  3.2% (0.0Mkm²) 11.7% (0.0Mkm²) 73.4% (0.2Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²) 
[ 1]  10.5% (0.0Mkm²)  3.2% (0.0Mkm²) 11.7% (0.0Mkm²) 74.6% (0.2Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²) 
     Point of view of study dataset
          nomatch      5 <= low < 20    intermediate     40 <= high     multi_match        parent           twin          complex     
[ 0]   9.6% (0.0Mkm²)  0.6% (0.0Mkm²)  9.4% (0.0Mkm²) 80.3% (0.2Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²) 
[ 1]  10.5% (0.0Mkm²)  0.6% (0.0Mkm²)  9.2% (0.0Mkm²) 79.6% (0.2Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²) 
[ref] 01_default_detection/Cyclonic_20160515T000000.nc -> 63 obs
[0] 02_detection_without_filtering/Cyclonic_20160515T000000.nc -> 58 obs
[1] 03_detection_without_filtering_err_90/Cyclonic_20160515T000000.nc -> 63 obs
          nomatch      5 <= low < 20    intermediate     40 <= high     multi_match        parent           twin          complex     
[ 0]  20.0% (0.1Mkm²)  2.9% (0.0Mkm²)  6.5% (0.0Mkm²) 70.5% (0.2Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²) 
[ 1]  19.2% (0.1Mkm²)  3.4% (0.0Mkm²)  6.0% (0.0Mkm²) 71.5% (0.2Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²) 
     Point of view of study dataset
          nomatch      5 <= low < 20    intermediate     40 <= high     multi_match        parent           twin          complex     
[ 0]  11.6% (0.0Mkm²)  0.9% (0.0Mkm²)  6.3% (0.0Mkm²) 81.3% (0.2Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²) 
[ 1]  13.5% (0.0Mkm²)  5.4% (0.0Mkm²)  5.9% (0.0Mkm²) 75.3% (0.2Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)  0.0% (0.0Mkm²)

[benlombardi]

Hi,

I am running into similar issues as Susana did. I am trying to replicate the 1993 January eddy tracking and detections for META 3.2 DT twosat. I got the ADT maps from C3S Delayed-Time version DT 2021 which were linked on the AVISO ( https://cds.climate.copernicus.eu/cdsapp#!/dataset/satellite-sea-level-global?tab=overview ).

I then ran the following code for each of the 31 days in January 1993.

g = RegularGridDataset(daily_grid_path,"longitude","latitude")
g.lanczos_high_filter("adt",700,order=1)

a, c = g.eddy_identification("adt", 
"ugos", 
"vgos",
 date,
 0.002, 
shape_error=70, 
pixel_limit=(5, 1000),
sampling=20,
sampling_method="visvalingam",
nb_step_min=2)

I then ran EddyTracking with the following yaml file.

PATHS:
 # Files produced with EddyIdentification
 FILES_PATTERN: '/projects/verif_dets/cyc/cyc_*.nc'
 # Path to save outputs
 SAVE_DIR: '/projects/out_data/'

# Number of consecutive timesteps with missing detection allowed
VIRTUAL_LENGTH_MAX: 4
# Minimal number of timesteps to considered as a long trajectory
TRACK_DURATION_MIN: 10

CLASS:
  MODULE: py_eddy_tracker.featured_tracking.area_tracker
  CLASS: AreaTracker

However, the results of tracking are substantially off. I made two new files for META3.2, META_cyc_short_jan.nc and META_cyc_jan_long.nc. The long file contains trajectories in january that live at least 10 days and the short file contain eddies that live strictly less than 10 days. The results for EddyQuick Compare are

[ref] META_cyc_short_jan.nc -> 17546 obs
[0] out_data/Cyclonic_track_too_short.nc -> 17468 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]   33.6% (5901)     0.0% (8)       0.0% (0)       0.0% (0)     66.3% (11637)    0.1% (14)      0.1% (10)    66.2% (11613) 
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]   34.4% (6012)     0.0% (8)       0.0% (0)       0.0% (0)     65.5% (11448)    0.0% (5)       0.2% (28)    65.3% (11415)

and

[ref] META_cyc_long_jan.nc -> 95373 obs
[0] out_data/Cyclonic.nc -> 91297 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]   11.8% (11232)    0.0% (1)       0.0% (0)       0.0% (0)     88.2% (84140)    0.0% (1)       0.0% (0)     88.2% (84139) 
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    9.7% (8858)     0.0% (1)       0.0% (0)       0.0% (0)     90.3% (82438)    0.0% (0)       0.0% (2)     90.3% (82436) 

The detection results are substantially different and I am not quite sure what is wrong with the code as I thought that I was using the same parameters as listed in the META 3.2 handbook. I am not quite sure if there is an issue with how I installed pyeddytracker or some other problem. My current installation passes all of the unit tests, albeit with a warnings.

Any help would be appreciated!

Thanks!

[AntSimi]

Hi,
To do a fair comparison, first extract from meta atlas (long,short,untracked) a same day by example 1/1/1993.
And compare with your own identification those file.
And share this results

[benlombardi]

Looking at 1/1/1993, the comparison for the long file is

[ref] META_long_daily.nc -> 2611 obs
[0] ver_long_daily.nc -> 2262 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    23.9% (625)     0.5% (12)      1.8% (47)    73.8% (1927)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    12.2% (276)     0.5% (12)      2.1% (47)    85.2% (1927)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0) 

the comparison for the short file is

[ref] META_short_daily.nc -> 1086 obs
[0] ver_short_daily.nc -> 873 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    49.9% (542)     1.0% (11)      1.3% (14)     47.6% (517)     0.2% (2)       0.0% (0)       0.2% (2)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    37.8% (330)     1.3% (11)      1.6% (14)     59.2% (517)     0.1% (1)       0.1% (1)       0.0% (0)       0.0% (0)  

and the comparison for the untracked file is

[ref] META_untracked_daily.nc -> 67 obs
[0] ver_untracked_daily.nc -> 253 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    80.6% (54)      1.5% (1)       1.5% (1)      16.4% (11)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    94.9% (240)     0.4% (1)       0.4% (1)       4.3% (11)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)

I am not sure if this is relevant, but when plotting all of the trajectories for the month I have noticed that there seems to be fewer trajectories near the equator than there is in the META dataset.

Once again, thanks for all of the help.

[benlombardi]

Apologies, that was for 31/1/1993, the results for 1/1/1993 for the long file are

[ref] META_long_daily.nc -> 2326 obs
[0] ver_long_daily.nc -> 2235 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    16.0% (372)     0.5% (11)      2.3% (54)    81.1% (1887)     0.1% (2)       0.0% (0)       0.1% (2)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    12.6% (282)     0.5% (11)      2.4% (54)    84.4% (1887)     0.0% (1)       0.0% (1)       0.0% (0)       0.0% (0) 

the comparison for the short file is

[ref] META_short_daily.nc -> 725 obs
[0] ver_short_daily.nc -> 782 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    37.4% (271)     0.4% (3)       1.7% (12)     60.6% (439)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    41.9% (328)     0.4% (3)       1.5% (12)     56.1% (439)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)

and lastly the comparison for the untracked file is

[ref] META_untracked_daily.nc -> 398 obs
[0] ver_untracked_daily.nc -> 379 obs
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    69.6% (277)     0.5% (2)       1.8% (7)      28.1% (112)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    40 <= high     multi_match      parent          twin          complex    
[ 0]    68.1% (258)     0.5% (2)       1.8% (7)      29.6% (112)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)  

Sorry for the inconvenience.

[AntSimi]

Hi,
I didn't notice but you use lanczos_high_filter but you must us bessel_high_filter which is a lanczos weighting with bessel window.
also you need to add this keyword argument `nb_step_to_be_mle=0``in identification parameter.
It mus look like this:

g = RegularGridDataset(daily_grid_path,"longitude","latitude")
g.bessel_high_filter("adt",700,order=1)

a, c = g.eddy_identification("adt", 
"ugos", 
"vgos",
 date,
 0.002, 
shape_error=70, 
pixel_limit=(5, 2000),
sampling=20,
sampling_method="visvalingam",
nb_step_min=2,
nb_step_to_be_mle=0
)
a.write_file(filename='19930101_anticyclonic.nc')
c.write_file(filename='19930101_cyclonic.nc')

This configuration must be provide good score:
Atlas_anticyclonic_19930101.nc file is an equivalent of untracked+short+long for 19930101 days.
We set minimal score to 95% to be consider like a good match.

EddyQuickCompare 19930101_anticyclonic.nc Atlas_anticyclonic_19930101.nc --high 95
[ref] 19930101_anticyclonic.nc -> 3284 obs
[0] Atlas_anticyclonic_19930101.nc -> 3284 obs
          nomatch     5 <= low < 20  intermediate    95 <= high     multi_match      parent          twin          complex    
[ 0]     0.0% (0)       0.0% (0)       0.0% (0)     100.0% (3284)    0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch     5 <= low < 20  intermediate    95 <= high     multi_match      parent          twin          complex    
[ 0]     0.0% (0)       0.0% (0)       0.0% (0)     100.0% (3284)    0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)

I hope it solve your problem, confirm me if you get the same result.

[benlombardi]

Yes! Using the bessel_high_filter fixed the detection issue, and I am now getting the same daily detection results for each day.

However, my overall tracking is still a bit off. I am still getting a slightly fewer eddy observations than are in META 3.2, but more eddy trajectories.

My tracking yaml file has a VIRTUAL_LENGTH_MAX of 4 and uses the AreaTracker class. That should be correct for the META 3.2 tracking right?

Thanks!

[SusanaMSilva]

Hi! Can you share how you extracted the day from all files?
I did the tracking again using the new parameters but I still get different and less trajectories in my region.

Thank you, I'm still learning python (use mostly matlab)

[benlombardi]

To extract the observations for a day from all files from META3.2 I use

import xarray as xr 
import numpy as np

dataset_c_long = xr.open_dataset("META3.2_DT_twosat_Cyclonic_long_19930101_20220209.nc")
dataset_c_short = xr.open_dataset("META3.2_DT_twosat_Cyclonic_short_19930101_20220209.nc")
dataset_c_untracked = xr.open_dataset("META3.2_DT_twosat_Cyclonic_untracked_19930101_20220209.nc")

end_time = np.datetime64("1993-01-01")
subset_c_long = dataset_c_long.sel(obs=(dataset_c_long.time == end_time) & (dataset_c_long.observation_flag==0))
subset_c_short = dataset_c_short.sel(obs=(dataset_c_short.time == end_time)& (dataset_c_short.observation_flag==0))
subset_c_untracked = dataset_c_untracked.sel(obs=(dataset_c_untracked.time == end_time))


subset_c_untracked.to_netcdf("META_untracked_daily.nc")
new_daily = xr.concat([subset_c_long,subset_c_short],dim='obs') 
new_daily.to_netcdf("META_daily.nc")

Then you can use EddyQuickCompare on both the untracked and concatenated file

[AntSimi]

This method work.
But there is also built-in script to do that:

# Here we extract day 20000 since1950-01-01 => 2004-10-04
EddySubSetter \
       --no_raw_mode --period 20000 20000 -v INFO \
       META3.2_DT_twosat_Cyclonic_short_19930101_20220209.nc Cyclonic_20000.nc

This script allows several methods to select and extract eddies observations.

[SusanaMSilva]

Thank you!

[benlombardi]

Hi I believed, that I fixed my error in with the tracking and now have an extremely similar dataset to META3.2 for January 1993. I needed to change the cmin parameter in the AreaTracker from 0.2 to 0.05.

There are a few little things that I have a few questions about.

Looking at through the daily eddy detections, occasionally the META dataset will have 1-2 detections that have no match with my dataset and my dataset will occasionally have 1-2 detections that have no match with the META dataset. In the grand scheme of things, 1-2 detections on several days does not make a substantial difference, but I am curious if you had an idea on what could be the cause of this. Could this be caused by some version difference in one of the packages such as numpy?

Further, while I have almost identical detections and trajectories in my reconstructed datasets, I have noticed some small differences in the global distributions of eddy characteristics that rely on fitting a best fit circle to one of the contours. The most striking difference is with regard to the speed_contour_shape_error, but there are also small differences in the distributions for the speed_radius as well as effective_radius. These differences also cause slight discrepancies the latitude and longitude values. I was also curious if you had any idea for what could be causing this discrepancy. I haven't been able to find any option that changes how the best fit circles are calculated.

Here is our a few histograms illustrating what I have found. I created this by aggregating all eddy observations from January 1-20 1993 from all three eddy files: long, short, and untracked.

Once again, thank you for your time and assistance!

[AntSimi]

@benlombardi
When you compare detection from original atlas did you remove virtual observation(which are filled by interpolation)?

You could check with EddyInfos (here in this file we have 238153 virtual observations)

EddyInfos Anticyclonic_track_too_short.nc 
-- Anticyclonic_track_too_short.nc -- 
    | 2792321 observations from 15706.0 to 25932.0 (10227.0 days, ~273 obs/day)
    |   Speed area      : 1.56 Mkm²/day
    |   Effective area  : 1.85 Mkm²/day
    ----Distribution in Amplitude:
    |   Amplitude bounds (cm)        0.00      1.00      2.00      3.00      4.00      5.00     10.00    500.00
    |   Percent of eddies         :      47.39     33.77     10.46      4.05      1.85      2.14      0.35
    ----Distribution in Radius:
    |   Speed radius (km)            0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.22     30.96     42.91     12.95      6.00      4.64      2.29      0.02
    |   Effective radius (km)        0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.19     27.72     41.63     13.62      7.06      5.94      3.80      0.04
    ----Distribution in Latitude
        Latitude bounds            -90.00    -60.00    -15.00     15.00     60.00     90.00
        Percent of eddies         :       6.67     33.04     25.85     24.86      9.58
        Percent of speed area     :       3.77     20.57     55.59     16.31      3.76
        Percent of effective area :       3.79     18.73     58.67     15.06      3.74
        Mean speed radius (km)    :      31.68     34.25     61.85     35.32     26.92
        Mean effective radius (km):      34.13     35.39     69.01     36.73     28.97
        Mean amplitude (cm)       :       1.12      1.82      0.96      1.59      1.50
    | 583198 tracks (4.79 obs/tracks, shorter 2 obs, longer 9 obs)
    |   238153 filled observations (0.41 obs/tracks, 8.53 % of total)
    |   Intepolated speed area      : 0.14 Mkm²/day
    |   Intepolated effective area  : 0.17 Mkm²/day
    |   Distance by day             : Mean 9.02 , Median 6.20 km/day
    |   Distance by track           : Mean 34.15 , Median 22.92 km/track
    ----Distribution in lifetime:
    |   Lifetime (days  )            1.00     30.00     90.00    180.00    270.00    365.00   1000.00  10000.00
    |   Percent of tracks         :     100.00      0.00      0.00      0.00      0.00      0.00      0.00
    |   Percent of eddies         :     100.00      0.00      0.00      0.00      0.00      0.00      0.00

In order to separate check on identification and tracking algorithm, to be sure to build trajectories with same algorithm, run your tracking yaml on original meta detection that you may create from atlas:

from py_eddy_tracker.observations.tracking import TrackEddiesObservations
import numpy as np

a_long = TrackEddiesObservations.load_file('Anticyclonic.nc')
a_short = TrackEddiesObservations.load_file('Anticyclonic_track_too_short.nc')
a_untrack = TrackEddiesObservations.load_file('Anticyclonic_untracked.nc').add_fields(['virtual', 'n', 'track'])
bins = np.arange(a_long.time.min(), a_long.time.max() + 1.01)
iter_short = a_short.iter_on('time', bins=bins)
iter_long = a_long.iter_on('time', bins=bins)
iter_untracked = a_untrack.iter_on('time', bins=bins)

for (i_u, t0, _), (i_s, _, _), (i_l, _, _) in zip(iter_untracked, iter_short, iter_long):
    a_t0 = TrackEddiesObservations.concatenate([a_untrack.index(i_u), a_short.index(i_s), a_long.index(i_l)])
    a_t0 = a_t0.extract_with_mask(a_t0.virtual == 0)
    a_t0.write_file(filename='my_t0_file.nc')

In order to have a more accurate diagnostic on detection difference you could use match function to compare parameter for common detection, an example here

[benlombardi]

When comparing the detections with the original META3.2 dataset I did remove the virtual observations for comparison. I was also able to replicate the META dataset by running my tracking code on the META detections.

I think that I have identified a problem in that when I do the detections on the original adt data, the circles of best fit for the effective contour and speed contour are slightly too small. From what I can tell, the other attributes of the eddies (amplitude, area, contour height, location of extrema) almost exactly match exact for the attributes that are based on the best fit circles. Here are the results after running EddyInfos on both the META dataset and my detection and tracking on the upstream adt data. Please let me know if something seems out of the ordinary to you.

-- META_cyc_long_jan.nc -- 
    | 46681 observations from 15706.0 to 15720.0 (15.0 days, ~3112 obs/day)
    |   Speed area      : 29.88 Mkm²/day
    |   Effective area  : 42.12 Mkm²/day
    ----Distribution in Amplitude:
    |   Amplitude bounds (cm)        0.00      1.00      2.00      3.00      4.00      5.00     10.00    500.00
    |   Percent of eddies         :      12.80     23.30     15.78     11.55      7.51     17.62     11.43
    ----Distribution in Radius:
    |   Speed radius (km)            0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.02      7.15     37.52     25.16     13.35     10.20      6.53      0.08
    |   Effective radius (km)        0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.02      4.95     27.20     23.80     14.97     15.37     13.44      0.25
    ----Distribution in Latitude
        Latitude bounds            -90.00    -60.00    -15.00     15.00     60.00     90.00
        Percent of eddies         :       7.28     50.19     11.67     28.79      2.07
        Percent of speed area     :       3.80     45.10     25.29     25.01      0.81
        Percent of effective area :       3.87     46.23     23.68     25.45      0.76
        Mean speed radius (km)    :      41.02     52.53     83.19     52.14     35.41
        Mean effective radius (km):      48.98     62.76     96.09     61.54     40.42
        Mean amplitude (cm)       :       3.55      5.97      2.46      4.75      3.37
    | 3816 tracks (12.23 obs/tracks, shorter 1 obs, longer 15 obs)
    |   3197 filled observations (0.84 obs/tracks, 6.85 % of total)
    |   Intepolated speed area      : 1.96 Mkm²/day
    |   Intepolated effective area  : 2.33 Mkm²/day
    |   Distance by day             : Mean 7.04 , Median 4.50 km/day
    |   Distance by track           : Mean 79.08 , Median 62.93 km/track
    ----Distribution in lifetime:
    |   Lifetime (days  )            1.00     30.00     90.00    180.00    270.00    365.00   1000.00  10000.00
    |   Percent of tracks         :     100.00      0.00      0.00      0.00      0.00      0.00      0.00
    |   Percent of eddies         :     100.00      0.00      0.00      0.00      0.00      0.00      0.00

-- ver_cyc_long_jan.nc -- 
    | 46682 observations from 15706.0 to 15720.0 (15.0 days, ~3112 obs/day)
    |   Speed area      : 29.91 Mkm²/day
    |   Effective area  : 42.20 Mkm²/day
    ----Distribution in Amplitude:
    |   Amplitude bounds (cm)        0.00      1.00      2.00      3.00      4.00      5.00     10.00    500.00
    |   Percent of eddies         :      12.80     23.30     15.79     11.55      7.51     17.62     11.43
    ----Distribution in Radius:
    |   Speed radius (km)            0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.03     11.10     37.44     22.99     12.95      9.81      5.63      0.04
    |   Effective radius (km)        0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.03      7.81     27.97     22.18     14.80     15.03     12.03      0.15
    ----Distribution in Latitude
        Latitude bounds            -90.00    -60.00    -15.00     15.00     60.00     90.00
        Percent of eddies         :       7.28     50.19     11.67     28.78      2.07
        Percent of speed area     :       3.81     45.09     25.29     25.00      0.81
        Percent of effective area :       3.88     46.22     23.69     25.44      0.76
        Mean speed radius (km)    :      39.32     50.46     80.12     50.04     33.49
        Mean effective radius (km):      46.99     60.34     92.25     59.14     38.37
        Mean amplitude (cm)       :       3.55      5.97      2.46      4.75      3.37
    | 3817 tracks (12.23 obs/tracks, shorter 1 obs, longer 15 obs)
    |   3197 filled observations (0.84 obs/tracks, 6.85 % of total)
    |   Intepolated speed area      : 1.96 Mkm²/day
    |   Intepolated effective area  : 2.33 Mkm²/day
    |   Distance by day             : Mean 6.39 , Median 3.87 km/day
    |   Distance by track           : Mean 71.71 , Median 54.95 km/track
    ----Distribution in lifetime:
    |   Lifetime (days  )            1.00     30.00     90.00    180.00    270.00    365.00   1000.00  10000.00
    |   Percent of tracks         :     100.00      0.00      0.00      0.00      0.00      0.00      0.00
    |   Percent of eddies         :     100.00      0.00      0.00      0.00      0.00      0.00      0.00

I haven't tried using the match function yet, but will try that next.