Skip to content

Commit 7f02c90

Browse files
bruceravelbruceravel
committed
add three more tests to XDI_validate_item + provide test cases
also improve documentation in README.md in the c/ folder
1 parent 892a3bc commit 7f02c90

File tree

10 files changed

+267
-68
lines changed

10 files changed

+267
-68
lines changed

‎baddata/BadFiles.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,9 @@ bad_29.xdi file read(1) Y date-time has invalid range (month > 12)
3636
bad_30.xdi file read(1) Y Invalid element.edge and element.symbol, test accumulation in XDI_required_metadata
3737
bad_31.xdi file read(1) Y Invalid value for d-spacing, trigger error in XDI_required_metadata
3838
bad_32.xdi file read(7) Y Missing recommended fields
39+
bad_33.xdi file read(1) Y Invalid Sample.temperature
40+
bad_34.xdi file read(1) Y Invalid Facility.energy
41+
bad_35.xdi file read(1) Y Invalid Facility.current
3942

4043
Notes:
4144
1. files read, return value > 0, indicating Warning

‎baddata/bad_33.xdi‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# XDI/1.0 GSE/1.0
2+
# Column.1: energy eV
3+
# Column.2: i0
4+
# Column.3: itrans
5+
# Column.4: mutrans
6+
# Element.edge: K
7+
# Element.symbol: Cu
8+
# Scan.edge_energy: 8980.0
9+
# Mono.name: Si 111
10+
# Mono.d_spacing: 3.13553
11+
# Beamline.name: 13ID
12+
# Beamline.collimation: none
13+
# Beamline.focusing: yes
14+
# Beamline.harmonic_rejection: rhodium-coated mirror
15+
# Facility.name: APS
16+
# Facility.energy: 7.00 GeV
17+
# Facility.xray_source: APS Undulator A
18+
# Scan.start_time: 2001-06-26T22:27:31
19+
# Detector.I0: 10cm N2
20+
# Detector.I1: 10cm N2
21+
# Sample.name: Cu
22+
# Sample.prep: Cu metal foil
23+
# Sample.temperature: 300.0a K
24+
# GSE.EXTRA: config 1
25+
# ///
26+
# Cu foil Room Temperature
27+
# measured at beamline 13-ID
28+
#----
29+
# energy i0 itrans mutrans
30+
8779.0 149013.7 550643.089065 -1.3070486
31+
8789.0 144864.7 531876.119084 -1.3006104
32+
8799.0 132978.7 489591.10592 -1.3033816
33+
8809.0 125444.7 463051.104096 -1.3059724
34+
8819.0 121324.7 449969.103983 -1.3107085
35+
8829.0 119447.7 444386.117562 -1.3138152
36+
8839.0 119100.7 440176.091039 -1.3072055
37+
8849.0 117707.7 440448.106567 -1.3195882
38+
8859.0 117754.7 442302.10637 -1.3233895
39+
8869.0 117428.7 441944.116528 -1.3253521
40+
8879.0 117383.7 442810.120466 -1.327693
41+
8889.0 117185.7 443658.11566 -1.3312944

‎baddata/bad_34.xdi‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# XDI/1.0 GSE/1.0
2+
# Column.1: energy eV
3+
# Column.2: i0
4+
# Column.3: itrans
5+
# Column.4: mutrans
6+
# Element.edge: K
7+
# Element.symbol: Cu
8+
# Scan.edge_energy: 8980.0
9+
# Mono.name: Si 111
10+
# Mono.d_spacing: 3.13553
11+
# Beamline.name: 13ID
12+
# Beamline.collimation: none
13+
# Beamline.focusing: yes
14+
# Beamline.harmonic_rejection: rhodium-coated mirror
15+
# Facility.name: APS
16+
# Facility.energy: 7.00 missing units
17+
# Facility.xray_source: APS Undulator A
18+
# Scan.start_time: 2001-06-26T22:27:31
19+
# Detector.I0: 10cm N2
20+
# Detector.I1: 10cm N2
21+
# Sample.name: Cu
22+
# Sample.prep: Cu metal foil
23+
# GSE.EXTRA: config 1
24+
# ///
25+
# Cu foil Room Temperature
26+
# measured at beamline 13-ID
27+
#----
28+
# energy i0 itrans mutrans
29+
8779.0 149013.7 550643.089065 -1.3070486
30+
8789.0 144864.7 531876.119084 -1.3006104
31+
8799.0 132978.7 489591.10592 -1.3033816
32+
8809.0 125444.7 463051.104096 -1.3059724
33+
8819.0 121324.7 449969.103983 -1.3107085
34+
8829.0 119447.7 444386.117562 -1.3138152
35+
8839.0 119100.7 440176.091039 -1.3072055
36+
8849.0 117707.7 440448.106567 -1.3195882
37+
8859.0 117754.7 442302.10637 -1.3233895
38+
8869.0 117428.7 441944.116528 -1.3253521
39+
8879.0 117383.7 442810.120466 -1.327693
40+
8889.0 117185.7 443658.11566 -1.3312944

‎baddata/bad_35.xdi‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# XDI/1.0 GSE/1.0
2+
# Column.1: energy eV
3+
# Column.2: i0
4+
# Column.3: itrans
5+
# Column.4: mutrans
6+
# Element.edge: K
7+
# Element.symbol: Cu
8+
# Scan.edge_energy: 8980.0
9+
# Mono.name: Si 111
10+
# Mono.d_spacing: 3.13553
11+
# Beamline.name: 13ID
12+
# Beamline.collimation: none
13+
# Beamline.focusing: yes
14+
# Beamline.harmonic_rejection: rhodium-coated mirror
15+
# Facility.name: APS
16+
# Facility.energy: 7.00 GeV
17+
# Facility.current: 101x ma
18+
# Facility.xray_source: APS Undulator A
19+
# Scan.start_time: 2001-06-26T22:27:31
20+
# Detector.I0: 10cm N2
21+
# Detector.I1: 10cm N2
22+
# Sample.name: Cu
23+
# Sample.prep: Cu metal foil
24+
# GSE.EXTRA: config 1
25+
# ///
26+
# Cu foil Room Temperature
27+
# measured at beamline 13-ID
28+
#----
29+
# energy i0 itrans mutrans
30+
8779.0 149013.7 550643.089065 -1.3070486
31+
8789.0 144864.7 531876.119084 -1.3006104
32+
8799.0 132978.7 489591.10592 -1.3033816
33+
8809.0 125444.7 463051.104096 -1.3059724
34+
8819.0 121324.7 449969.103983 -1.3107085
35+
8829.0 119447.7 444386.117562 -1.3138152
36+
8839.0 119100.7 440176.091039 -1.3072055
37+
8849.0 117707.7 440448.106567 -1.3195882
38+
8859.0 117754.7 442302.10637 -1.3233895
39+
8869.0 117428.7 441944.116528 -1.3253521
40+
8879.0 117383.7 442810.120466 -1.327693
41+
8889.0 117185.7 443658.11566 -1.3312944

‎c/README.md‎

Lines changed: 76 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -9,45 +9,48 @@ least very similar).
99

1010
See `xdi_reader.c` for an example of a C program using `libxdifile` to
1111
import and interpret XDI-formatted data. See the python and perl
12-
wrappers for examples of language specific implementations which use
13-
`libxdifile`.
12+
wrappers for examples of language specific implementations which link
13+
directly to the `libxdifile` library.
1414

1515
`libxdifile` was written by Matt Newville and Bruce Ravel.
1616

1717
## XDIFile struct
1818

1919
This is the content of the XDIFile struct. It will contain the entire
20-
contents of the XDI file along with a few particularly important items
21-
(d-spacing, element, and edge).
22-
23-
| attribute | type | explanation |
24-
| --------------- | ------------------ | ----------- |
25-
| filename | char* | the name of the XDI file |
26-
| nmetadata | long | number of metadata fields |
27-
| meta\_families | array of char* | array of family names found among the metadata, indexed to nmetadata |
20+
contents of the XDI file along with a few particularly important
21+
metadata items (d-spacing, element, and edge).
22+
23+
| attribute | type | explanation |
24+
| --------------- | ------------------ | --------------------------------------------------------------------- |
25+
| filename | char* | the name of the XDI file |
26+
| nmetadata | long | number of metadata fields |
27+
| meta\_families | array of char* | array of family names found among the metadata, indexed to nmetadata |
2828
| meta\_keywords | array of char* | array of keyword names found among the metadata, indexed to nmetadata |
29-
| meta\_values | array of char* | array of values found among the metadata, indexed to nmetadata |
30-
| narrays | long | number of arrays in data table |
31-
| npts | long | number of rows in data table |
32-
| array | 2D array of double | the data table |
33-
| narray\_labels | long | number of array labels |
34-
| array\_labels | array of char* | array of labels for arrays in the data table |
35-
| array\_units | array of char* | array of units for arrays in the data table |
36-
| comments | char* | the user supplied comments from the XDI file |
37-
| xdi\_libversion | char* | the `libxdifile` version number |
38-
| xdi\_version | char* | the XDI file specification version number from the file |
39-
| extra\_version | char* | versioning information for extension fields |
40-
| dspacing | double | the Mono.d-spacing value |
29+
| meta\_values | array of char* | array of values found among the metadata, indexed to nmetadata |
30+
| narrays | long | number of arrays in data table |
31+
| npts | long | number of rows in data table |
32+
| array | 2D array of double | the data table |
33+
| narray\_labels | long | number of array labels |
34+
| array\_labels | array of char* | array of labels for arrays in the data table |
35+
| array\_units | array of char* | array of units for arrays in the data table |
36+
| comments | char* | the user supplied comments from the XDI file |
37+
| xdi\_libversion | char* | the `libxdifile` version number |
38+
| xdi\_version | char* | the XDI file specification version number from the file |
39+
| extra\_version | char* | versioning information for extension fields |
40+
| dspacing | double | the Mono.d-spacing value |
4141
| element | char* | the Element.symbol value, the 1, 2, or 3 letter symbol of the element |
42-
| edge | char* | the Element.edge value, the the 1 or 2 letter symbol of the edge |
43-
| error\_lineno | long | the line number of a line returning an error |
44-
| error\_line | char* | the line returning an error |
45-
| error\_message | char* | the error message |
46-
| nouter | long | |
47-
| outer\_label | array char* | |
48-
| outer\_array | array of double | |
49-
| outer\_breakpts | array of long | |
50-
42+
| edge | char* | the Element.edge value, the 1 or 2 letter symbol of the edge |
43+
| error\_lineno | long | the line number of a line returning an error |
44+
| error\_line | char* | the line returning an error |
45+
| error\_message | char* | the error message |
46+
| nouter | long | (\*) |
47+
| outer\_label | array char* | (\*) |
48+
| outer\_array | array of double | (\*) |
49+
| outer\_breakpts | array of long | (\*) |
50+
51+
(\*) Currently undocumented struct elements intended for support of
52+
two-dimensional data. This may be supported in a future version of
53+
`libxdifile`.
5154

5255
## API
5356

@@ -57,7 +60,7 @@ parsing the file and validating its content.
5760
The main point of entry is the function `XDI_readfile` which creates
5861
an `XDIFile` struct, then opens and parses a file for XDI content.
5962
This function will signal an error and issue a useful error message in
60-
the even that some content of the file precludes its being understood
63+
the event that some content of the file precludes its being understood
6164
as XDI data. These situations include such things as
6265

6366
* Missing XDI versioning information in the first line of the file
@@ -67,10 +70,22 @@ as XDI data. These situations include such things as
6770

6871
Certain conditions that are non-conforming with the XDI specification
6972
but which do not preclude partial interpretation of the XDI file
70-
result in warning messages at this time.
73+
result in warning messages issued by `XDI_readfile`.
74+
75+
Validating the content of the XDI metadata is treated in three
76+
separate steps. The philosophy of use of the XDI library is:
7177

72-
Validating the content of the XDI metadata is treated as a separate
73-
step. There are three main functions for this purpose:
78+
1. Read the XDI file, stopping only in the case of an unrecoverable error, then
79+
2. Check that the metadata identified in the specification as **required** are present, then
80+
3. Check that the metadata identified in the specification as **recommended** are present, then
81+
4. Validate each individual metadata item against its dictionary definition
82+
83+
This allows an application to take a fine-grained approach to how
84+
strictly it will follow the details of the XDI specification. Thus, a
85+
file can be read and used with any level of validation -- including
86+
_no validation_ -- being performed.
87+
88+
The three validation functions are:
7489

7590
* `XDI_required_metadata`: This function tests that the pieces of
7691
metadata which are **required** by the specification are present in
@@ -105,10 +120,6 @@ condition of the most recently performed action. The relation between
105120
the returned error/warning codes and the error messages are tabulated
106121
below.
107122

108-
The value of separating most validation chores from the parsing of the
109-
file is that it allows an application to choose to accept structurally
110-
compliant data then to perform the level of validation appropriate to
111-
the task at hand.
112123

113124

114125
### Read an XDI file
@@ -117,6 +128,8 @@ Read an XDI file, store it's content in an XDIFile struct, and return
117128
an integer return code.
118129

119130
```C
131+
#include "xdifile.h"
132+
120133
XDIFile *xdifile;
121134
int ret;
122135

@@ -126,11 +139,11 @@ an integer return code.
126139

127140
### Interpret the XDI_readfile error code
128141

129-
Interpret the return code by printing the corresponding error message
130-
to the screen:
142+
Interpret the `XDI_readfile` return code by printing the corresponding
143+
error or warning message to the screen:
131144

132145
```C
133-
/* react to a terminal error */
146+
/* react to an error reading the file */
134147
if (ret < 0) {
135148
printf("Error reading XDI file '%s':\n %s\t(error code = %ld)\n",
136149
argv[1], xdifile->error_message, ret);
@@ -149,7 +162,9 @@ to the screen:
149162
The `xdifile->error_message` attribute will always contain the error
150163
or warning message from the most recent action. If an error code
151164
returns as non-zero, the content of `xdifile->error_message` will
152-
explain the meaning of the error code in English.
165+
explain the meaning of the error code in English. Note that cleanup
166+
and deallocation must be performed in the event of an error precluding
167+
the reading of the file.
153168
154169
### Test for required metadata
155170
@@ -193,7 +208,8 @@ This `for` loop iterates through each set of metadata family, keyword,
193208
and value found in the XDI file. `XDI_validate_item` tests the value
194209
to see if it meets the recommendation of the specification.
195210
Validation tests do not exist for all items in the metadata dictionary
196-
-- `XDI_validate_item` will return 0 for
211+
-- `XDI_validate_item` will return 0 for any free-format dictionary
212+
item.
197213
198214
```C
199215
for (i=0; i < xdifile->nmetadata; i++) {
@@ -271,9 +287,9 @@ translation of a table of error messages into another language.
271287
| -16 | number of columns changes in data table |
272288
| -32 | non-numeric value in data table: `<word>` |
273289
274-
Here `<word>` will be the the text that triggered the error.
290+
Here `<word>` will be the text that triggered the error.
275291
276-
### XDI_readfile warning codes
292+
### XDI_readfile and XDI_validate_item warning codes
277293
278294
| code | message |
279295
| ----: | ------------------------------------------------------------ |
@@ -288,13 +304,16 @@ Here `<word>` will be the the text that triggered the error.
288304
| 256 | Column.1 is not "energy" or "angle" |
289305
| 512 | invalid timestamp: format should be ISO 8601 (YYYY-MM-DD HH:MM:SS) |
290306
| 1024 | invalid timestamp: date out of valid range |
307+
| 2048 | <not used> |
308+
| 4096 | bad value in Sample namespace |
309+
| 8192 | bad value in Facility namespace |
291310
292311
293312
### XDI_required_metadata return codes
294313
295-
The return code from `XDI_required_metadata` can be interpreted
296-
bitwise. That is, a return code of 7 means that all three required
297-
metadata fields were missing.
314+
The return code from `XDI_required_metadata` is interpreted bitwise.
315+
That is, a return code of 7 means that all three required metadata
316+
fields were missing or invalid.
298317
299318
| code | message |
300319
| ---: | -------------------------------------- |
@@ -305,15 +324,15 @@ metadata fields were missing.
305324
306325
### XDI_recommended_metadata return codes
307326
308-
The return code from `XDI_recommended_metadata` can be interpreted
327+
The return code from `XDI_recommended_metadata` is interpreted
309328
bitwise. That is, a return code of 7 means that the first three
310329
recommended metadata fields were missing.
311330
312-
| code | message |
313-
| ---: | --------------------------------------------------- |
314-
| 1 | Missing recommended metadata field: Facility.name |
315-
| 2 | Missing recommended metadata field: Facility.source |
316-
| 4 | Missing recommended metadata field: Beamline.name |
317-
| 8 | Missing recommended metadata field: Scan.start_time |
318-
| 16 | Missing recommended metadata field: Column.1 |
331+
| code | message |
332+
| ---: | --------------------------------------------------------- |
333+
| 1 | Missing recommended metadata field: Facility.name |
334+
| 2 | Missing recommended metadata field: Facility.xray\_source |
335+
| 4 | Missing recommended metadata field: Beamline.name |
336+
| 8 | Missing recommended metadata field: Scan.start_time |
337+
| 16 | Missing recommended metadata field: Column.1 |
319338

‎c/test_valgrind.pl‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,11 @@
22

33
use strict;
44
use warnings;
5-
use Test::More tests => 144;
5+
use Test::More tests => 154;
6+
use File::Which qw(which);
7+
8+
ok(which('valgrind'), "valgrind is in the execution path");
9+
610

711
## good data
812
foreach my $file (qw(co_metal_rt.xdi
@@ -35,10 +39,10 @@
3539
'12' => 0, '13' => 1, '14' => 1, '15' => 1, '16' => 1, '17' => 1,
3640
'18' => 1, '19' => 1, '20' => 1, '21' => 1, '22' => 1, '23' => 0,
3741
'24' => 1, '25' => 0, '26' => 0, '27' => 0, '28' => 0, '29' => 0,
38-
'30' => 0, '31' => 0, '32' => 0);
42+
'30' => 0, '31' => 0, '32' => 0, '33' => 0, '34' => 0, '35' => 0);
3943

4044
## bad data
41-
foreach my $i (0 .. 32) {
45+
foreach my $i (0 .. 35) {
4246
my $n = sprintf("%2.2d", $i);
4347
my $command = "valgrind --track-origins=yes --leak-check=full --show-leak-kinds=all ./xdi_reader ../baddata/bad_$n.xdi 2>&1";
4448
my $x = `$command`;

0 commit comments

Comments
 (0)