Support Sen2Cor 2.10

[savmickael]

A new version of Sen2Cor will be available soon and will be used by S2 PDGS to produce new S2 L2A.

It could be interesting to:

• analyze and list the possibility offers by this new version
• compare results (SR and SCL mask) with the current version 2.9.

What I propose is to generate a set of data for VITO in order to analyze also the impact on classification results. We can use the tile 21HTC as site.

1. Run one date in 2019 with sen2cor2.9
2. Run the same date with 2.10 with same option
3. Run the same date with 2.10 and legacy mode

Pleas find the Sen2Cor_2_10_Toolbox_Delivery_Docs.zip

[savmickael]

the code which run sen2cor currently is here:

ewoc_sen2cor/utils.py

Line 105 in 54f7876

s2c_cmd = f"./Sen2Cor-02.09.00-Linux64/bin/L2A_Process {l1c_safe} --output_dir {l2a_out} --resolution 10"

[savmickael]

Following 2.10 options need to be analyzed:

• --sc_classic Performs scene classification in Sen2Cor 2.9 mode
• --sc_cog Export SCL image in COG format instead of JPEG_2000

Improvement of the COPDEM need to be analyzed

[mbattude-cs]

---

PART 1

Test on S2 image "S2B_MSIL2A_20190112T140049_N0211_R067_T21HTC_20190112T175414.SAFE"
The DEM used is 90m SRTM DEM.

Comparison of 4 products :

• L2A (reference)
• L2A from Sen2Cor v09
• L2A from Sen2Cor v10 legacy mode
• L2A from Sen2Cor v10

Analysis for bands in native resolution (10m and 20m only) :

• 10m : B02, B03, B04, B08
• 20m : B05, B06, B07, B11, B12, B8A + SCL

---

Extraction of some statistics (max, mean, min, std) using gdalinfo --mm -stats

		reference	sen2cor_v09	sen2cor_v10legacy	sen2cor_v10
B02	max	18112	18160	18160	17952
	min	0	0	0	0
	mean	515	510	515	512
	std	1050	1042	1048	1040
B03	max	17152	17200	17200	17104
	min	0	0	0	0
	mean	633	626	630	627
	std	1067	1057	1061	1054
B04	max	16496	16512	16512	16448
	min	0	0	0	0
	mean	568	563	565	563
	std	1042	1032	1035	1029
B08	max	15696	15696	15696	15664
	min	0	0	0	0
	mean	1752	1731	1732	1729
	std	2000	1977	1978	1974
B05	max	16172	16198	16198	16117
	min	0	0	0	0
	mean	834	829	829	826
	std	1207	1199	1199	1193
B06	max	15904	15909	15909	15867
	min	0	0	0	0
	mean	1496	1481	1481	1478
	std	1732	1715	1715	1710
B07	max	15767	15768	15768	15737
	min	0	0	0	0
	mean	1726	1708	1708	1705
	std	1969	1949	1949	1944
B11	max	14711	14773	14773	14883
	min	0	0	0	0
	mean	1303	1288	1288	1287
	std	1542	1524	1524	1522
B12	max	15104	15106	15106	15099
	min	0	0	0	0
	mean	877	868	868	868
	std	1196	1184	1184	1183
B8A	max	15623	15621	15621	15599
	min	0	0	0	0
	mean	1824	1804	1804	1801
	std	2053	2031	2031	2026
SCL	max	10	10	10	10
	min	0	0	0	0
	mean	3	3	3	3
	std	3	3	3	3

Comparison with otbcli_CompareImages :

		ref - v09	ref - v10legacy	ref - v10	v09 - v10legacy	v09 - v10	v10legacy - v10
B02	mse	320.4387207	485.068512	603.1402588	341.0184937	333.8317871	116.6472092
	mae	7.309188843	8.789256096	9.657939911	4.952044964	6.849100113	4.282206535
	psnr	60.10187912	58.30129623	57.3551445	59.85453796	59.94704056	64.51357269
	count	61004280	60665544	60431452	40161220	59356460	58444544
B03	mse	285.852417	319.4411011	463.2037048	166.3032684	172.9358368	81.64199829
	mae	7.725412369	7.857829571	9.248425484	3.455995083	4.984792709	3.529254198
	psnr	60.12487793	59.64238739	58.0285759	62.50156021	62.33171844	65.59143066
	count	61159596	60695536	61036808	34558328	56405120	55608880
B04	mse	225.8056946	221.3683777	361.4066772	77.86045837	101.0026627	67.09880829
	mae	6.540157795	6.307127476	7.733542442	2.379002333	3.910097599	3.06519556
	psnr	60.81022263	60.89641571	58.76761246	65.44282532	64.31266785	66.0888443
	count	60126880	59660328	59548664	29871888	56112448	55210536
B08	mse	1392.869751	1330.241699	1707.581543	29.87714767	121.7646866	111.4683762
	mae	21.71575546	20.8826561	24.11335373	1.136098862	5.102508068	4.981298923
	psnr	52.47667313	52.67647552	51.59196472	69.16239166	63.06056595	63.44426346
	count	62000432	61893556	62156344	14765252	55877316	56269264
B05	mse	276.6339111	276.5492249	455.5378113	0.0008451862959	68.87110901	68.87789154
	mae	7.954811096	7.953986168	9.944764137	0.0008451862959	3.293948889	3.294267416
	psnr	59.75622177	59.75754929	57.59003067	114.9197006	65.80886078	65.8084259
	count	15368979	15368977	15418900	25474	13995504	13995213
B06	mse	833.6872559	833.3521118	1171.915161	0.001846178318	114.8743362	114.8949509
	mae	16.14682961	16.1450119	19.41362381	0.001846178318	4.950418949	4.951440811
	psnr	54.82009506	54.82184219	53.34116745	111.3701248	63.43062592	63.42984772
	count	15627681	15627687	15597437	55644	15040288	15040705
B07	mse	1180.618164	1180.116577	1601.24707	0.002492261119	127.7617416	127.7913055
	mae	19.67552757	19.67307854	23.23455429	0.002492261119	5.350089073	5.351420879
	psnr	53.23388672	53.23573303	51.91039658	109.9895935	62.89152527	62.89051819
	count	15646529	15646541	15631545	75117	15105853	15106116
B11	mse	709.2764893	709.2640991	822.4084473	0.0002200059098	10.37019348	10.37104321
	mae	15.28015804	15.27994728	16.52532387	0.0002200059098	1.593846917	1.593981266
	psnr	54.84468842	54.84476471	54.20196915	119.9650345	73.23150635	73.23114777
	count	15505335	15505334	15606273	6631	14290321	14290959
B12	mse	369.3416443	369.3392639	413.9916687	9.482383757e-05	14.78582954	14.78587532
	mae	9.68846035	9.688375473	10.17606544	9.482383757e-05	0.752802074	0.7528271079
	psnr	57.90755844	57.90758514	57.41192245	123.8138123	71.88452911	71.88452148
	count	15509611	15509599	15478704	2858	9388116	9388426
B8A	mse	1386.315186	1385.834229	1761.86145	0.002630283358	105.4052811	105.43013
	mae	21.77262878	21.77004623	24.81944466	0.002630283358	4.706523895	4.707933903
	psnr	52.45666885	52.45817566	51.41556931	109.6741486	63.64555359	63.64452744
	count	15576441	15576427	15645155	79277	15066981	15067573
SCL	mse	0.4393551648	0.4393551648	0.7760353088	0	0.3964585662	0.3964585662
	mae	0.0895672217	0.0895672217	0.1705676466	0	0.09837475419	0.09837475419
	psnr	23.57184219	23.57184219	21.10118484	0	24.01802254	24.01802254
	count	641605	641605	1383056	0	902283	902283

For each band, comparison of the histograms of values ​​for the 4 products :

For each band, comparison of the histograms of differences between the 4 products :

Barplot of the maximum absolute difference for each band :

Barplot of the maximum absolute difference for each test :

Confusion matrix over SCL band, with a comparison of Sen2Cor_v09 (rows) and Sen2Cor_V10 (columns) labels :

	no_data	satured_or_defective	dark_area_pixels	cloud_shadows	vegetation	not_vegetated	water	unclassified	cloud_medium_probability	cloud_high_probability	thin_cirrus	snow
no_data	14417189	0	0	0	0	0	0	0	0	0	0	0
satured_or_defective	0	0	0	0	0	0	0	0	0	0	0	0
dark_area_pixels	0	0	2	58563	0	15609	0	373110	0	0	0	0
cloud_shadows	0	0	1	194901	3803	1041	297	11537	0	0	0	0
vegetation	0	0	2	63696	9795896	0	0	0	0	0	0	0
not_vegetated	0	0	0	1	0	2113623	0	99	0	0	0	0
water	0	0	0	787	0	0	28894	0	0	0	0	0
unclassified	0	0	1	956	0	258318	0	294645	0	0	0	0
cloud_medium_probability	0	0	0	0	0	103414	0	0	691620	0	0	0
cloud_high_probability	0	0	0	0	0	11048	0	0	0	1061767	0	0
thin_cirrus	0	0	0	0	0	0	0	0	0	0	639280	0
snow	0	0	0	0	0	0	0	0	0	0	0	0

Precision of the different classes: [1, NA, 0.333333, 0.611159, 0.999612, 0.844418, 0.989826, 0.43369, 1, 1, 1, NA]
Recall of the different classes: [1, NA, 4.47143e-06, 0.921169, 0.993539, 0.999953, 0.973485, 0.531927, 0.869925, 0.989702, 1, NA]
F-score of the different classes: [1, NA, 8.94274e-06, 0.734804, 0.996566, 0.915627, 0.981587, 0.477811, 0.930438, 0.994824, 1, NA]

Kappa index: 0.95438
Overall accuracy index: 0.970064

Confusion matrix in percent :
Note that the sum of each row is equal to 100%.

	no_data	satured_or_defective	dark_area_pixels	cloud_shadows	vegetation	not_vegetated	water	unclassified	cloud_medium_probability	cloud_high_probability	thin_cirrus	snow
no_data	100	NA	0	0	0	0	0	0	0	0	0	NA
satured_or_defective	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
dark_area_pixels	0	NA	0	13	0	3	0	83	0	0	0	NA
cloud_shadows	0	NA	0	92	2	0	0	5	0	0	0	NA
vegetation	0	NA	2	1	99	0	0	0	0	0	0	NA
not_vegetated	0	NA	0	0	0	100	0	0	0	0	0	NA
water	0	NA	0	3	0	0	97	0	0	0	0	NA
unclassified	0	NA	0	0	0	47	0	53	0	0	0	NA
cloud_medium_probability	0	NA	0	0	0	13	0	0	87	0	0	NA
cloud_high_probability	0	NA	0	0	0	1	0	0	0	99	0	NA
thin_cirrus	0	NA	0	0	0	0	0	0	0	0	100	NA
snow	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA

Main confusions :

• Class 'dark_area_pixels' is mixed with classes 'unclassified' (83%), 'cloud_shadows' (13%) and 'not_vegetated' (3%)
• Class 'unclassified' is mixed with class 'not_vegetated' (47%)
• Class 'cloud_medium_probability' is mixed with class 'not_vegetated' (13%)
• Class 'cloud_shadows' is mixed with classes 'unclassified' (5%) and 'vegetation' (2%)
• Class 'water' is mixed with class 'cloud_shadows' (3%)

Note that SCL obtained with Sen2Cor_v09 and Sen2Cor_v10legacy are identical.

---

PART 2

Sen2Cor v10 was run using 90m SRTM DEM ("srtm") and 90m Copernicus DEM ("copdem"). Below are some results of the comparison between these two versions.

---

Extraction of some statistics (max, mean, min, std) using gdalinfo --mm -stats

		sen2cor_v10_srtm	sen2cor_v10_copdem
B02	max	17952	17984
	min	0	0
	mean	512	512
	std	1040	1040
B03	max	17104	17104
	min	0	0
	mean	627	627
	std	1054	1054
B04	max	16448	16448
	min	0	0
	mean	563	563
	std	1029	1029
B08	max	15664	15664
	min	0	0
	mean	1729	1729
	std	1974	1974
B05	max	16117	16117
	min	0	0
	mean	826	826
	std	1193	1193
B06	max	15867	15866
	min	0	0
	mean	1478	1478
	std	1710	1710
B07	max	15737	15737
	min	0	0
	mean	1705	1705
	std	1944	1944
B11	max	14883	14622
	min	0	0
	mean	1287	1287
	std	1522	1522
B12	max	15099	15099
	min	0	0
	mean	868	868
	std	1183	1183
B8A	max	15599	15599
	min	0	0
	mean	1801	1801
	std	2026	2026
SCL	max	10	10
	min	0	0
	mean	3	3
	std	3	3

Comparison with otbcli_CompareImages :

		srtm - copdem
B02	mse	31.20344734
	mae	1.668593764
	psnr	70.14022827
	count	36992640
B03	mse	36.37800217
	mae	2.06709218
	psnr	69.05356598
	count	37124896
B04	mse	35.19551468
	mae	1.882104635
	psnr	68.85739136
	count	36702200
B08	mse	211.4936218
	mae	6.266269207
	psnr	60.64508057
	count	37142004
B05	mse	52.74589157
	mae	2.775371552
	psnr	66.92379761
	count	9210873
B06	mse	150.5742188
	mae	5.327167034
	psnr	62.23239136
	count	9331134
B07	mse	203.66185
	mae	6.20076704
	psnr	60.84934235
	count	9343658
B11	mse	103.0678482
	mae	4.360414982
	psnr	63.32257843
	count	9101953
B12	mse	51.40926361
	mae	2.818394184
	psnr	66.46855164
	count	9095261
B8A	mse	223.7093506
	mae	6.54197073
	psnr	60.36509323
	count	9308420
SCL	mse	1.360314036e-05
	mae	3.64962284e-06
	psnr	68.66360474
	count	34

For each band, comparison of the histograms of values ​​for the 2 products :

For each band, comparison of the histograms of differences between the 2 products :

Barplot of the maximum absolute difference for each band :

Confusion matrix over SCL band, with a comparison of Sen2Cor_v10_srtm (rows) and Sen2Cor_V10_copdem (columns) labels :

	no_data	satured_or_defective	dark_area_pixels	cloud_shadows	vegetation	not_vegetated	water	unclassified	cloud_medium_probability	cloud_high_probability	thin_cirrus	snow
no_data	14417189	0	0	0	0	0	0	0	0	0	0	0
satured_or_defective	0	0	0	0	0	0	0	0	0	0	0	0
dark_area_pixels	0	0	1	1	2	0	0	2	0	0	0	0
cloud_shadows	0	0	1	318903	0	0	0	0	0	0	0	0
vegetation	0	0	5	0	9799694	0	0	0	0	0	0	0
not_vegetated	0	0	12	0	0	2503039	0	0	2	0	0	0
water	0	0	0	0	0	0	29191	0	0	0	0	0
unclassified	0	0	8	0	0	1	0	679382	0	0	0	0
cloud_medium_probability	0	0	0	0	0	0	0	0	691620	0	0	0
cloud_high_probability	0	0	0	0	0	0	0	0	0	1061767	0	0
thin_cirrus	0	0	0	0	0	0	0	0	0	0	639280	0
snow	0	0	0	0	0	0	0	0	0	0	0	0

Precision of the different classes: [1, NA, 0.037037, 0.999997, 1, 1, 1, 0.999997, 0.999997, 1, 1, NA]
Recall of the different classes: [1, NA, 0.166667, 0.999997, 0.999999, 0.999994, 1, 0.999987, 1, 1, 1, NA]
F-score of the different classes: [1, NA, 0.0606061, 0.999997, 1, 0.999997, 1, 0.999992, 0.999999, 1, 1, NA]

Kappa index: 0.999998
Overall accuracy index: 0.999999

Confusion matrix in percent :
Note that the sum of each row is equal to 100%.

	no_data	satured_or_defective	dark_area_pixels	cloud_shadows	vegetation	not_vegetated	water	unclassified	cloud_medium_probability	cloud_high_probability	thin_cirrus	snow
no_data	100	NA	0	0	0	0	0	0	0	0	0	NA
satured_or_defective	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
dark_area_pixels	0	NA	17	17	33	0	0	33	0	0	0	NA
cloud_shadows	0	NA	0	100	0	0	0	0	0	0	0	NA
vegetation	0	NA	0	0	100	0	0	0	0	0	0	NA
not_vegetated	0	NA	0	0	0	100	0	0	0	0	0	NA
water	0	NA	0	0	0	0	100	0	0	0	0	NA
unclassified	0	NA	0	0	0	0	0	100	0	0	0	NA
cloud_medium_probability	0	NA	0	0	0	0	0	0	100	0	0	NA
cloud_high_probability	0	NA	0	0	0	0	0	0	0	100	0	NA
thin_cirrus	0	NA	0	0	0	0	0	0	0	0	100	NA
snow	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA

---

MAIN CONCLUSIONS

BANDS

• Sen2Cor_v09, Sen2Cor_v10 and Sen2Cor_v10legacy give similar results (the histogram of the values of the different bands are similar and the maximum absolute difference between the products is small).
• Sen2Cor_v09 and Sen2Cor_v10legacy are almost identical for 20m resolution bands (i.e. B05, B06, B07, B8A, B11, B12).
• Differences are observed for the 10m resolution bands (i.e. B02, B03, B04, B08), with lower reflectance values in products from Sen2Cor_v09 compared to Sen2Cor_v10legacy.

SCL

• SCL obtained with Sen2Cor_v09 and Sen2Cor_v10legacy are identical.
• SCL obtained with Sen2Cor_v09 and Sen2Cor_v10 are similar, however we see some confusions for classes 'dark_area_pixels' (recall=0%), 'unclassified' (recall=53%) and 'cloud_medium_probability' (recall=87%).

DEM

• 90m SRTM DEM and 90m Copernicus DEM tested in Sen2Cor_v10 give nearly identical results. Only few pixels do not belong to the same SCL class if we compare the two versions. Note that the class 'dark_area_pixels' has a recall=17% but this class contains only 6 pixels.

---

[savmickael]

Please find here a presentation about the sen2cor 2.10 version and the differences with previous version.

[savmickael]

Please find here a presentation about the impact of the type of DEM on Sen2Cor.