Commit 6f254f2
Add operator plotmap, intake catalogs, fixes for IDRIS and check(), run on Obelix, validate plotmap with C-ESM-EP, allow for data check by ds() (#260)
* PEP008 on environment.py and test_example_data_plot.py
* use generate_pdf.py also at IPSL
* Remove test condition SkipIf_CondaEnv
* Fix some project's root for IDRIS
* Fix for IDRIS (atIDRIS diag, path for igcm_out)
* Another fix for IDRIS (igcm_cmip6)
* Fix concurrency issue in makedirs(tmpdir); don't use pdfjam at TGCC and IDRIS
os.makedirs(tmpdir) more robust
* Create tests/reference_data/test_data_plot/idris_20230611_V3.0_IPSL2
Only one plot change vs. spirit_conda_20221224
* Fix typo in ea4c992 (makedirs(test_ok=True))
* Improve tools_for_tests (compare_picture_files)
compare_picture_files don't anymore stop on first instance of
different pictures
It also allows to create a html file showing the differences and the two
images that differ
* PEP008 on environment.py and test_example_data_plot.py
* use generate_pdf.py also at IPSL
* Remove test condition SkipIf_CondaEnv
* Fix concurrency issue in makedirs(tmpdir); don't use pdfjam at TGCC and IDRIS
os.makedirs(tmpdir) more robust
* Fix typo in ea4c992 (makedirs(test_ok=True))
* Restore utils/remove_keys_with_same_values
* Fix on : Restore utils/remove_keys_with_same_values
* dataset.glob() also includes facet 'project' in returned dict(s)
* dataset.glob() returns intersection of requested and avilable periods
* dataset.glob() has argument ensure_period (default is True)
* tests.tools_for_tests.build_diffs_html() now based on chtml
* dataset.check handles monthly data (and cleaner)
* PEP8
* Add facet 'frequency' in project CMIP6
* Add arg 'check' to ds(). Create period.build_date_rexep()
* PEP008
* Some flesh in plotmap, and its interface in plot_operators.py
* Change plotmap Climaf interface. Stage plotmap example and Aladin sfcWind file
* 1st version OK with xarray for LambertII
* Add script output format 'show' (no output, value forwarded to script)
* Script call parameters are dumped in json format
* Discard now extraneous " in script call parameters. Needs testing !
* Discard empty arguments in instanciated script calls
* plotmap.py interface looks almost like gplot.ncl
* Add data samples : Nemo, Aladin, uas + vas
* Fix vector plots, titles. Add gv methods. CCRS from metadata
Fix vector plots
Fix titles
Diagnose CCRS from metadata
Add gv methods
Default plot_engine is contourf
* Fix cscript() call for Ncl scripts, and fix those scripts re. decoding "true"
* Fix plotmap.py re. compute_xy and default projection
* Stage plotmap_demo.py, python version of the plotmap example notebook
* Fix tools_for_tests for comparing figure ensembles
* For ensemble members title, fix combining ensemble title and member_label
* version of plotmap.py mimicking gplot.ncl, with demo notebook
* api.cshow now calls display o Image if running in a notebook
* More fixes and improvements for plotmap and related notebooks
* GitHub actions : upgrade versions. Update environment for plotmap
* GitHub actions : fix setup-miniconda version
* Github Actions : fix cache version and update test_environment.yml
* Fix ipython in conda_test_env.yml
* Create doc for plotmap + some changes on plotmap
* Stage plotmapdemo and Getting_started_with_plotmap (html and ipynb)
* Polish doc/news
* Fix doc/conf.py
* Introduce env.robust_pid for alleviating issue with docker at TGCC
* Fix ensemble_time_series.py for case with only one time step
* Handle IPSL intake catalogs
* Fixes and doc (in news) for intake
* Add forgotten module intake_search
* Add project PMIP3
* Fixes re. intake related changes, and fix tests
* Add reference_data/test_index_html/spiritIMv7 for ImageMagick V7
* Fix calling convert (arguments order)
* Fix setting a robust PID when running in a container
* At TGCC (i.e. in a container), use SLURM_JOBID as PID
* Fix on using SLURM_JOBID
* Use SLURM_JOBID also at IDRIS
* gplot.ncl understands both 'true' and 'True'
* store_wildcard_facet_value admits facet filenameVar
* Fix intake_search and allow version='latest' for CMIP6 and CMIP5
* Fix intake re. cfreqs. Fix Cordex cfreqs. Fix intake_search. latest in Cordex
* Add projects's method for solving ambiguity in facet values
With instances for projects igcm_out and cmip5
* plotmap avoids decoding time when not necessary
* Introduce a wrapper for 'plot', which may call plotmap
Function plot_operators.plot calls plotmap if
env.environment.plot_use_plotmap=True (otherwise it calls plot)
It transforms some plot arguments to plotmap's equivalent
* driver.scripts_ouput_write_mode allows for mode append for last.out
* Fixes for plot wrapper and for classes.dataset.prefered_values()
* plot wrapper handles cnMissingValFillColor and cnLineLabelsOn
* Add driver.cxr(); fix bug in find_files when using fileNameVar
Also fix typo in standard_operators (for curves)
* chtml.cell allows to choose image filename
* Add args for ensemble_time_series_plot : 'year_delta' and 'draw_grid'
* Called scripts arguments are not json-coded when declared as {!arg}
* Ensure backward compatbility for tests regarding chtml.cell()
* Add operator cnkso, which calls ncks with a (possibly composite) operator
* Update doc and news re. ensemble_ts_plot args draw_grid and year_delta
* Changes to .gitignore
* Fix intake_search re. e.g. CORDEX_domain
* Fix gplot (NhlSetColor). Fix period.start_with re. fx.
* Fix gplot re NhlSetColor. Fix ensemble_ts_plot re legendHandles
* ds.resolve() stores files list. Fix operator rescale. Improve plot scripts
ensemble_time_series_plot show months names on seasonal cycle plots
gplot.ncl knows more latnames and lonnames
* gplot.ncl does not change all colormaps when defining a grey
* Fix errors with check for some observation datasets
* Fix typos and polish doc
* PEP8
* Test suite on GitHub uses Ubuntu 24.04
* Test suite on GitHub uses Miniforge3
* Doc build on GitHub uses Ubuntu 24.04 and Miniforge3
* Disable test_html in GitHub test workflow
* For doc build on GitHub, update upload-artifact to v4
* GitHub test suite does install Image magick and pdftk
* Fix some doc strings
* Insert changes for obelix. Simplify find_files
* chtml.cell() fixed re. file extension; ensemble_time_series_plot allows for single time
* Netcdfbasics.varsOfFile() is flexible about nav_lat*
* functions.tsplot() handle argument None
* ds() has new args data_check and check_type, with default values in env
* Fix cache.cdrop for index entries without corresponding file
* Tune plotmap and its interface with a C-ESM-EP run. Fix hurs plot params
For hurs bias : avoid defining 24 levels when using a 12 colors colormap
* Default behaviour for ds() doesn't include a data check
env.environment.data_check turned to False
* Fix at CNRM
---------
Co-authored-by: Stephane Senesi <ssenesi@obelix4.lsce.ipsl.fr>
Co-authored-by: rigoudyg <gaelle.rigoudy@meteo.fr>1 parent 6d118e3 commit 6f254f2
File tree
140 files changed
+27305
-2313
lines changed- .github/workflows
- climaf
- derived_variables
- plot
- projects
- doc
- scripts
- env
- examples
- data
- scripts
- tests
- reference_data
- test_data_plot
- idris_20230611_V3.0_IPSL2
- obelix_20250128
- test_index_html
- obelix_20250128
- spirit_IMv7
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
140 files changed
+27305
-2313
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
| 17 | + | |
18 | 18 | | |
19 | | - | |
20 | | - | |
| 19 | + | |
| 20 | + | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
| 47 | + | |
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
11 | | - | |
| 10 | + | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
23 | | - | |
24 | | - | |
| 23 | + | |
| 24 | + | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
31 | 43 | | |
32 | | - | |
| 44 | + | |
33 | 45 | | |
34 | 46 | | |
35 | 47 | | |
| |||
58 | 70 | | |
59 | 71 | | |
60 | 72 | | |
61 | | - | |
| 73 | + | |
62 | 74 | | |
63 | 75 | | |
64 | 76 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
98 | | - | |
| 98 | + | |
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
0 commit comments