Level2 magnetics data severely quantized (~6 effective bits despite float64 storage)

[RHH-CQER]

We are conducting disruption prediction research using our framework analysis on MAST Mirnov coil data, which relies on derivative-based metrics (instantaneous frequency, growth rates). This makes our work particularly sensitive to data resolution.

Summary

Level2 zarr data for MAST magnetics diagnostics (Mirnov coils, Saddle loops) appears to be quantized to approximately 6 effective bits of resolution, despite being stored as float64. This makes derivative-based analyses unreliable, as quantization steps generate artificial spikes. Level1 raw ADC data does not have this problem (~14.5 effective bits), so the issue appears to be in the Level2 processing pipeline.

Affected Data

• Archive path: s3://mast/level2/ via s3.echo.stfc.ac.uk
• Confirmed on: Shot 29980 (magnetics/b_field_pol_probe_cc_field, shape 39x269200, dtype float64)
• Likely scope: Unknown how many shots are affected; we tested one in detail

Reproduction

# Download shot 29980
s5cmd --no-sign-request --endpoint-url https://s3.echo.stfc.ac.uk \
  cp 's3://mast/level2/shots/29980.zarr/*' 29980.zarr/

import zarr, numpy as np

store = zarr.open('29980.zarr/magnetics', 'r')
signal = store['b_field_pol_probe_cc_field'][0, :]  # Mirnov channel 0

# Check unique values in a 2ms window (1000 samples at 500kHz)
window = signal[182000:183000]  # ~253-255ms
unique = np.unique(window)
print(f"Unique values: {len(unique)}")       # Expected: hundreds. Observed: 7-9
print(f"Min step: {np.min(np.diff(np.sort(unique)))}")  # Observed: 4.375e-08 T (uniform grid)

# Full-signal analysis
all_unique = np.unique(signal)
diffs = np.diff(np.sort(all_unique))
min_step = np.min(diffs[diffs > 0])
dynamic_range = np.max(signal) - np.min(signal)
effective_bits = np.log2(dynamic_range / min_step)
print(f"Effective bits: {effective_bits:.1f}")  # Observed: 6.1

Quantitative Evidence

Metric	Expected (float64)	Observed
Unique values per 2ms window	Hundreds+	Median 9, range 7-35
Effective bit depth	52+ (mantissa)	6.1
Shannon entropy per window	>20 bits	Median 2.39 bits
Quantization step	Variable	Uniform 43.7 nT grid
Windows failing 16-unique threshold	~0%	98.3% (170/173)

The uniform step size (4.375e-08 T ≈ 2⁻²⁴·⁵) across the entire signal confirms this is a digitization/processing artifact, not noise.

Impact

Any analysis computing derivatives on this data (dB/dt, growth rates, instantaneous frequency via Hilbert transform) will produce artificial spikes at quantization boundaries. We initially identified what appeared to be physically meaningful "bouncing ball" patterns in the pre-disruption window — these turned out to be pure quantization artifacts.

Workaround

Level1 raw ADC data for the same shots shows ~14.5 effective bits and does not exhibit this problem. We migrated our analysis pipeline to Level1 and the artifacts disappeared.

Additional Observation: Saddle Loop NaN Rates

On the same shot, Saddle loop data (b_field_tor_probe_saddle_field) has 56% NaN values (180,872 / 323,040), rising to 60% in the analysis window around the disruption. This may be a separate issue but worth flagging.

Environment

• Access method: s5cmd v2.x with --no-sign-request --endpoint-url https://s3.echo.stfc.ac.uk
• Analysis: Python 3.9, zarr, numpy, scipy
• Date of access: December 2025 – January 2026

Questions for the Team

1. Is the Level2 quantization a known limitation of the processing pipeline?
2. Is this specific to certain shot ranges or diagnostic channels?
3. Should downstream users default to Level1 data for quantitative analysis?
4. Is the Saddle loop NaN rate expected for shot 29980?

Thank you for maintaining this archive — it's a valuable resource and we want to make sure the community is aware of this limitation for derivative-sensitive analyses.

[samueljackson92]

Hi @RHH-CQER, thanks for the bug report. Just taking a quick look at this issue this morning

Reproduction

Just to check: I am not 100% sure the path you've provided in the example to download the shot is correct? There shouldn't be any data at s3://mast/data/shots/, the path I think you're refering to is s3://mast/level2/shots/. I am investigating with this data as I assume this is what you meant.

Quantitative Evidence

Reproducing the analysis on the raw original signal, (straight from the data archives) I get the following results:

orig_signal = xr.open_dataset("uda://xmc/CC/MV/201:29980", engine='uda')

orig_data = orig_signal.data.values
orig_time = orig_signal.time.values


tmin, tmax = time[182000], time[183000]
print(f"Time range: {tmin:.6f} to {tmax:.6f} s") 

orig_slice = orig_signal.sel(time=slice(tmin, tmax))
orig_time, orig_window = orig_slice.time, orig_slice.data

unique = np.unique(orig_window)
print(f"Unique values: {len(unique)}")       # Expected: hundreds. Observed: 7-9
print(f"Min step: {np.min(np.diff(np.sort(unique)))}")  # Observed: 4.375e-08 T (uniform grid)

# Full-signal analysis
all_unique = np.unique(orig_data)
diffs = np.diff(np.sort(all_unique))
min_step = np.min(diffs[diffs > 0])
dynamic_range = np.max(orig_data) - np.min(orig_data)
effective_bits = np.log2(dynamic_range / min_step)
print(f"Effective bits: {effective_bits:.1f}")  # Observed: 6.1

Time range: 0.264002 to 0.266002 s
Unique values: 11
Min step: 4.3744989852712024e-08
Effective bits: 6.1

However, here is a plot of the diff between the two signals, suggesting there could be some small quantisation artefacts which could be affecting your analysis.

Can you tell me which equivalent L1 signal you're looking at? I don't think raw L1 XMC data was ever pushed to the archive publicly? and there is no group called "ADC"? But I could be out of the loop these days...

[RHH-CQER]

Thank you for this. Let me check. I will get back to you ASAP.

[samueljackson92]

Re: saddle coils NaN percentage. For some shots, the not all of the signals are available due to historic data corruption/missing values. I believe this is the case here, for shot 29980 some of the coils are blank:

Checking with the data archive, this is true. Coil M10 for example returns an error saying not record is found. Therefore we just write the entry as nan (but keep the number of channels the same).

The data looks at least vaguely sensible to me otherwise:

[RHH-CQER]

Hi @samueljackson92, thank you for the thorough reproduction, the additional context on the Saddle coils, and for your email reply — all very helpful.

S3 Path

You're right — the download path should be s3://mast/level2/shots/. Apologies for the error in the original issue; I've edited it to correct this.

UDA Reproduction

Very useful that you've confirmed the same quantisation signature (11 unique values, 4.37e-08 step, 6.1 effective bits) directly from uda://xmc/CC/MV/201:29980. This confirms the quantisation is present in the original signal.

Could you clarify what the second signal in your diff plot is? I want to make sure I'm reading it correctly — is that Level2 zarr vs raw UDA, or something else?

Level1 / "ADC" Correction

I need to correct both our original issue and my email on this point. On re-checking our local data more carefully:

• We successfully downloaded Level1 Mirnov coil data from s3://mast/level1/shots/{SHOT}.zarr/amb/ccbvNN/ for the 11xxx and 12xxx shot series — those have 37–38 coils per shot and show effective bit depth well above Level2.
• For the 29xxx series, the Level1 path only contains amc/plasma_current — there is no Mirnov coil data under amb/. So we do not have Level1 XMC data for shot 29980, and our claim in the issue (and my email) about migrating to "Level1 raw ADC data" does not apply to this shot.
• Our use of "ADC" was imprecise shorthand for the Level1 amb group — there is no group literally called "ADC." Apologies for the confusion.

The 14.5+ effective bits figure was measured on 11xxx/12xxx series Level1 data where amb/ccbvNN coils are available. It does not apply to the 29xxx range.

Saddle NaN

Thank you for confirming the NaN values are due to historically missing coil records in the original archive. That makes sense.

Updated Questions

Given the above corrections:

1. Is there any way to access raw (pre-Level2-processing) XMC Mirnov data for the 29xxx shot range? Or is Level2 the only publicly available form for these shots?
2. Since the quantisation appears in the original UDA signal — does this reflect the native digitiser resolution for this diagnostic on these shots, or is it possible the update issue you mentioned also affected the UDA path?
3. Are there shot ranges or diagnostics you'd recommend for applications that need higher effective bit depth in magnetic field measurements?

I've also updated #212 to correct some inaccuracies there.

Thank you again — we really appreciate the fast engagement, and the dataset is genuinely valuable for our work.

[samueljackson92]

Hi @RHH-CQER

S3 Path

👍 I thought that was the case.

UDA Reproduction
Could you clarify what the second signal in your diff plot is? I want to make sure I'm reading it correctly — is that Level2 zarr vs raw UDA, or something else?

Top is L2 signal overlayed with original (unprocessed signal). Bottom is the difference between the two (a - b).

Level1 / "ADC" Correction
For the 29xxx series, the Level1 path only contains amc/plasma_current — there is no Mirnov coil data under amb/. So we do not have Level1 XMC data for shot 29980, and our claim in the issue (and my email) about migrating to "Level1 raw ADC data" does not apply to this shot.

Hmm, this sounds like an issue to me. There should be amb data for that range. We can investigate further.

Furthermore, I think we need to clarify: there were several sets of centre column coil arrays on MAST. amb contained a vertical center column array:

While xmc contain another set of vertical and toroidal arrays:

So they are different physical arrays, with potentially different measurement resolution, digitization, etc.

Saddle NaN

No problem. Sadly, in a machine last run over a decade ago things went wrong, got lost & corrupted, so the data record is not perfect...

Updated Questions
Is there any way to access raw (pre-Level2-processing) XMC Mirnov data for the 29xxx shot range? Or is Level2 the only publicly available form for these shots?

Currently no. For L1 we never pulled them all because they were really increasing the size of the dataset and we were hitting memory limitations on the machines we were using to do it. For a collaboration with L2 data we decided to just take a subset of the xmc coils.

Since the quantisation appears in the original UDA signal — does this reflect the native digitiser resolution for this diagnostic on these shots, or is it possible the update issue you mentioned also affected the UDA path?

I think this is likely the native digitizer resolution for this array.

Are there shot ranges or diagnostics you'd recommend for applications that need higher effective bit depth in magnetic field measurements?

You could look at shots that have OMAHA array data (b_field_tor_probe_omaha_voltage) in L2 (not available in L1 for the same reason as the above). I believe this was the highest resolution magnetics array they had on MAST. There should be shots with some OMAHA data in the M9 campaign (> 28390)

[samueljackson92]

After further email discussion with @RHH-CQER I believe everything in this issue has been addressed and this one can be closed.