Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 22 additions & 11 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,28 @@

## Version 0.5.0 (in development)

- Introduced type aliases `ConfigLike` and `ConfigObjectLike`.
- Renamed multiple components for improved clarity and consistency:
- Renamed `Config` into `ConfigObject`
- Renamed `ConfigList.configs` into `config_objects`
- Renamed `ConfigList` into `Config`
- Renamed `ConfigList.compute_config()` into `compute_config_object()`
- Renamed `Result.config` into `config_object`
- Renamed `XRLint.load_config_list()` into `init_config()`
- Renamed `XRLint.verify_datasets()` into `verify_files()`
- Added class method `from_config()` to `ConfigList`.
- Removed function `xrlint.config.merge_configs` as it was no longer used.
### Incompatible API changes:

- Renamed nodes and node properties for consistency and clarity:
- renamed `DataArrayNode` into `VariableNode`
- renamed `DataArrayNode.data_array` into `VariableNode.array`

- Changed general use of term _verify_ into _validate_:
- prefixed `RuleOp` methods by `validate_` for clarity.
- renamed `XRLint.verify_datasets()` into `validate_files()`
- renamed `Lint.verify_dataset()` into `validate()`

- Various changes for improved clarity and consistency
regarding configuration management:
- introduced type aliases `ConfigLike` and `ConfigObjectLike`.
- renamed `Config` into `ConfigObject`
- renamed `ConfigList.configs` into `config_objects`
- renamed `ConfigList` into `Config`
- renamed `ConfigList.compute_config()` into `compute_config_object()`
- renamed `Result.config` into `config_object`
- renamed `XRLint.load_config_list()` into `init_config()`
- added class method `from_config()` to `ConfigList`.
- removed function `xrlint.config.merge_configs` as it was no longer used.

## Version 0.4.1 (from 2025-01-31)

Expand Down
2 changes: 1 addition & 1 deletion docs/start.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,5 +106,5 @@ import xrlint.all as xrl
test_ds = xr.Dataset(attrs={"title": "Test Dataset"})

linter = xrl.new_linter("recommended")
linter.verify_dataset(test_ds)
linter.validate(test_ds)
```
23 changes: 8 additions & 15 deletions docs/todo.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,6 @@
- use mkdocstrings ref syntax in docstrings
- provide configuration examples (use as tests?)
- add `docs_url` to all existing rules
- API changes for v0.5:
- clarify when users can pass configuration objects like values
and when configuration like values
- config class naming is confusing,
change `Config` -> `ConfigObject`, `ConfigList` -> `Config`
- Change `verify` -> `validate`,
prefix `RuleOp` methods by `validate_` for clarity.

## Desired

Expand Down Expand Up @@ -51,30 +44,30 @@

## Generalize data linting

Do not limit verification to `xr.Dataset`.
Do not limit validations to `xr.Dataset`.
However, this requires new rule sets.

To allow for other data models, we need to allow
for a specific verifier type for a given data type.
for a specific validator type for a given data type.

The verifier verifies specific node types
The validator validates specific node types
that are characteristic for a data type.

To do so a traverser must traverse the elements of the data
and pass each node to the verifier.
and pass each node to the validator.

Note, this is the [_Visitor Pattern_](https://en.wikipedia.org/wiki/Visitor_pattern),
where the verifier is the _Visitor_ and a node refers to _Element_.
where the validator is the _Visitor_ and a node refers to _Element_.

To support the CLI mode, we need different data opener
types that can read the data from a file path.

1. open data, if given data is a file path:
- find opener for file path
- open data
2. verify data
2. validate data
- find root element type and visitor type for data
- call the root element `accept(verifier)` that verifies the
root element `verify.root()` and starts traversal of
- call the root element `accept(validator)` that validates the
root element `validate.root()` and starts traversal of
child elements.

2 changes: 1 addition & 1 deletion examples/check_s3_bucket.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@

xrlint = xrl.XRLint(no_config_lookup=True)
xrlint.init_config("recommended")
results = xrlint.verify_files([URL])
results = xrlint.validate_files([URL])
print(xrlint.format_results(results))
2 changes: 1 addition & 1 deletion examples/plugin_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
class GoodTitle(RuleOp):
"""Dataset title should be 'Hello World!'."""

def dataset(self, ctx: RuleContext, node: DatasetNode):
def validate_dataset(self, ctx: RuleContext, node: DatasetNode):
good_title = "Hello World!"
if node.dataset.attrs.get("title") != good_title:
ctx.report(
Expand Down
2 changes: 1 addition & 1 deletion examples/rule_testing.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
class GoodTitle(RuleOp):
"""Dataset title should be 'Hello World!'."""

def dataset(self, ctx: RuleContext, node: DatasetNode):
def validate_dataset(self, ctx: RuleContext, node: DatasetNode):
good_title = "Hello World!"
if node.dataset.attrs.get("title") != good_title:
ctx.report(
Expand Down
2 changes: 1 addition & 1 deletion examples/virtual_plugin_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

@define_rule("good-title", description="Dataset title should be 'Hello World!'.")
class GoodTitle(RuleOp):
def dataset(self, ctx: RuleContext, node: DatasetNode):
def validate_dataset(self, ctx: RuleContext, node: DatasetNode):
good_title = "Hello World!"
if node.dataset.attrs.get("title") != good_title:
ctx.report(
Expand Down
8 changes: 4 additions & 4 deletions notebooks/xrlint-linter.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -532,7 +532,7 @@
}
],
"source": [
"linter.verify_dataset(ds)"
"linter.validate(ds)"
]
},
{
Expand Down Expand Up @@ -1019,7 +1019,7 @@
}
],
"source": [
"linter.verify_dataset(invalid_ds)"
"linter.validate(invalid_ds)"
]
},
{
Expand Down Expand Up @@ -1100,7 +1100,7 @@
}
],
"source": [
"linter.verify_dataset(invalid_ds)"
"linter.validate(invalid_ds)"
]
},
{
Expand Down Expand Up @@ -1159,7 +1159,7 @@
}
],
"source": [
"linter.verify_dataset(invalid_ds)"
"linter.validate(invalid_ds)"
]
},
{
Expand Down
40 changes: 19 additions & 21 deletions tests/test_linter.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from xrlint.config import Config, ConfigObject
from xrlint.constants import CORE_PLUGIN_NAME, NODE_ROOT_NAME
from xrlint.linter import Linter, new_linter
from xrlint.node import AttrNode, AttrsNode, DataArrayNode, DatasetNode
from xrlint.node import AttrNode, AttrsNode, VariableNode, DatasetNode
from xrlint.plugin import new_plugin
from xrlint.processor import ProcessorOp
from xrlint.result import Message, Result
Expand Down Expand Up @@ -50,34 +50,34 @@ def test_new_linter_all(self):
self.assertIn("coords-for-dims", config_obj_1.rules)


class LinterVerifyConfigTest(TestCase):
class LinterValidateWithConfigTest(TestCase):
def test_config_with_config_list(self):
linter = new_linter()
result = linter.verify_dataset(
result = linter.validate(
xr.Dataset(),
config=Config.from_value([{"rules": {"no-empty-attrs": 2}}]),
)
self.assert_result_ok(result, "Missing metadata, attributes are empty.")

def test_config_with_list_of_config(self):
linter = new_linter()
result = linter.verify_dataset(
result = linter.validate(
xr.Dataset(),
config=[{"rules": {"no-empty-attrs": 2}}],
)
self.assert_result_ok(result, "Missing metadata, attributes are empty.")

def test_config_with_config_obj(self):
linter = new_linter()
result = linter.verify_dataset(
result = linter.validate(
xr.Dataset(),
config={"rules": {"no-empty-attrs": 2}},
)
self.assert_result_ok(result, "Missing metadata, attributes are empty.")

def test_no_config(self):
linter = Linter()
result = linter.verify_dataset(
result = linter.validate(
xr.Dataset(),
)
self.assert_result_ok(result, "No configuration given or matches '<dataset>'.")
Expand All @@ -89,27 +89,27 @@ def assert_result_ok(self, result: Result, expected_message: str):
self.assertEqual(expected_message, result.messages[0].message)


class LinterVerifyTest(TestCase):
class LinterValidateTest(TestCase):
def setUp(self):
plugin = new_plugin(name="test")

@plugin.define_rule("no-space-in-attr-name")
class AttrVer(RuleOp):
def attr(self, ctx: RuleContext, node: AttrNode):
def validate_attr(self, ctx: RuleContext, node: AttrNode):
if " " in node.name:
ctx.report(f"Attribute name with space: {node.name!r}")

@plugin.define_rule("no-empty-attrs")
class AttrsVer(RuleOp):
def attrs(self, ctx: RuleContext, node: AttrsNode):
def validate_attrs(self, ctx: RuleContext, node: AttrsNode):
if not node.attrs:
ctx.report("Empty attributes")

@plugin.define_rule("data-var-dim-must-have-coord")
class DataArrayVer(RuleOp):
def data_array(self, ctx: RuleContext, node: DataArrayNode):
def validate_variable(self, ctx: RuleContext, node: VariableNode):
if node.in_data_vars():
for dim_name in node.data_array.dims:
for dim_name in node.array.dims:
if dim_name not in ctx.dataset.coords:
ctx.report(
f"Dimension {dim_name!r}"
Expand All @@ -119,7 +119,7 @@ def data_array(self, ctx: RuleContext, node: DataArrayNode):

@plugin.define_rule("dataset-without-data-vars")
class DatasetVer(RuleOp):
def dataset(self, ctx: RuleContext, node: DatasetNode):
def validate_dataset(self, ctx: RuleContext, node: DatasetNode):
if len(node.dataset.data_vars) == 0:
ctx.report("Dataset does not have data variables")
raise RuleExit # no need to traverse further
Expand Down Expand Up @@ -157,7 +157,7 @@ def test_rules_are_ok(self):
)

def test_linter_respects_rule_severity_error(self):
result = self.linter.verify_dataset(
result = self.linter.validate(
xr.Dataset(), rules={"test/dataset-without-data-vars": 2}
)
self.assertEqual(
Expand All @@ -182,7 +182,7 @@ def test_linter_respects_rule_severity_error(self):
)

def test_linter_respects_rule_severity_warn(self):
result = self.linter.verify_dataset(
result = self.linter.validate(
xr.Dataset(), rules={"test/dataset-without-data-vars": 1}
)
self.assertEqual(
Expand All @@ -207,7 +207,7 @@ def test_linter_respects_rule_severity_warn(self):
)

def test_linter_respects_rule_severity_off(self):
result = self.linter.verify_dataset(
result = self.linter.validate(
xr.Dataset(), rules={"test/dataset-without-data-vars": 0}
)
self.assertEqual(
Expand All @@ -225,9 +225,7 @@ def test_linter_respects_rule_severity_off(self):
)

def test_linter_recognized_unknown_rule(self):
result = self.linter.verify_dataset(
xr.Dataset(), rules={"test/dataset-is-fast": 2}
)
result = self.linter.validate(xr.Dataset(), rules={"test/dataset-is-fast": 2})
self.assertEqual(
[
Message(
Expand Down Expand Up @@ -266,7 +264,7 @@ def test_linter_real_life_scenario(self):
)
dataset.encoding["source"] = "chl-tsm.zarr"

result = self.linter.verify_dataset(
result = self.linter.validate(
dataset,
config={
"rules": {
Expand Down Expand Up @@ -325,7 +323,7 @@ def test_linter_real_life_scenario(self):
)

def test_processor_ok(self):
result = self.linter.verify_dataset(
result = self.linter.validate(
"test.levels",
config={
"processor": "test/multi-level-dataset",
Expand All @@ -352,7 +350,7 @@ def test_processor_ok(self):
)

def test_processor_fail(self):
result = self.linter.verify_dataset(
result = self.linter.validate(
"bad.levels",
config={
"processor": "test/multi-level-dataset",
Expand Down
2 changes: 1 addition & 1 deletion tests/test_testing.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@


class ForceTitle(RuleOp):
def dataset(self, ctx: RuleContext, node: DatasetNode):
def validate_dataset(self, ctx: RuleContext, node: DatasetNode):
title = node.dataset.attrs.get("title")
if not title:
ctx.report("Datasets must have a title")
Expand Down
Loading