diff --git a/README.md b/README.md index efc776899..af358b8a9 100644 --- a/README.md +++ b/README.md @@ -117,6 +117,37 @@ To disable colored output, use the `--no-color` flag: metaschema-cli --no-color ``` +#### Shell Completion + +The CLI supports tab completion for Bash and Zsh shells, providing intelligent suggestions for commands, subcommands, and options. + +**Bash:** + +```bash +# Generate and source completion (temporary, current session only) +source <(metaschema-cli shell-completion bash) + +# Or save to a file and source it in your ~/.bashrc for persistence +metaschema-cli shell-completion bash > ~/.metaschema-completion.bash +echo 'source ~/.metaschema-completion.bash' >> ~/.bashrc +``` + +**Zsh:** + +```zsh +# Ensure your completions directory exists +mkdir -p ~/.zsh/completions + +# Generate completion script +metaschema-cli shell-completion zsh > ~/.zsh/completions/_metaschema-cli + +# Add to your ~/.zshrc if not already configured +echo 'fpath=(~/.zsh/completions $fpath)' >> ~/.zshrc +echo 'autoload -Uz compinit && compinit' >> ~/.zshrc +``` + +After setting up completion, restart your shell or source the configuration file. + ## Relationship to prior work The contents of this repository is based on work from the [Metaschema Java repository](https://github.com/usnistgov/metaschema-java/) maintained by the National Institute of Standards and Technology (NIST), the [contents of which have been dedicated in the worldwide public domain](https://github.com/usnistgov/metaschema-java/blob/1a496e4bcf905add6b00a77a762ed3cc31bf77e6/LICENSE.md) using the [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/) public domain dedication. This repository builds on this prior work, maintaining the [CCO license](https://github.com/metaschema-framework/metaschema-java/blob/main/LICENSE.md) on any new works in this repository. diff --git a/src/site/markdown/building.md.vm b/src/site/markdown/building.md.vm new file mode 100644 index 000000000..4d2da4824 --- /dev/null +++ b/src/site/markdown/building.md.vm @@ -0,0 +1,166 @@ +# Building from Source + +This guide explains how to build the Metaschema Java project from source code. + +## Prerequisites + +| Requirement | Minimum Version | Notes | +|:------------|:----------------|:------| +| Java JDK | 17 | Required for building (output JARs are JDK 11 compatible) | +| Maven | 3.9.0 | Required for building | +| Git | 2.0 | Required for cloning | + +Ensure `JAVA_HOME` is set correctly: + +```bash +# Linux/macOS +export JAVA_HOME=$(/usr/libexec/java_home -v 17) + +# Windows +set JAVA_HOME=C:\Program Files\Java\jdk-17 +``` + +## Clone the Repository + +Clone the repository with submodules: + +```bash +git clone --recurse-submodules https://github.com/metaschema-framework/metaschema-java.git +cd metaschema-java +``` + +If you already cloned without submodules, initialize them: + +```bash +git submodule update --init --recursive +``` + +## Build Commands + +### Basic Build + +Build and install to your local Maven repository: + +```bash +mvn install +``` + +### Run Tests Only + +```bash +mvn test +``` + +### Run a Single Test + +```bash +# Run a single test class +mvn test -Dtest=MetaschemaModuleTest + +# Run a single test method +mvn test -Dtest=MetaschemaModuleTest#testLoadModule +``` + +### Skip Tests + +```bash +mvn install -DskipTests +``` + +### CI/CD Build + +Replicate the full CI/CD build (recommended before pushing): + +```bash +mvn install -PCI -Prelease +``` + +This enables additional checks including: + +- License header verification +- Checkstyle code style checks +- SpotBugs static analysis +- Full test suite + +## Code Quality Commands + +### Check License Headers + +```bash +mvn license:check +``` + +### Auto-format Source Code + +```bash +mvn formatter:format +``` + +### Check for Checkstyle Issues + +```bash +mvn checkstyle:check +``` + +## Common Build Issues + +### Submodule Not Initialized + +**Symptom:** Build fails with missing Metaschema schema files. + +**Solution:** Initialize submodules: + +```bash +git submodule update --init --recursive +``` + +### Java Version Mismatch + +**Symptom:** Compilation errors or "unsupported class file version" errors. + +**Solution:** Ensure you're using Java 17 or later for building: + +```bash +java -version +mvn -version +``` + +### Maven Version Too Old + +**Symptom:** Build fails with plugin compatibility errors or enforcer rule failures. + +**Solution:** Upgrade Maven to 3.9.0 or later: + +```bash +mvn -version +``` + +### Out of Memory + +**Symptom:** Build fails with `OutOfMemoryError`. + +**Solution:** Increase Maven's heap size: + +```bash +export MAVEN_OPTS="-Xmx2g" +mvn install +``` + +## Building the Site + +To build the project documentation site: + +```bash +mvn site +``` + +The generated site appears in `target/site/`. For multi-module builds, each module's site is in its respective `target/site/` directory. + +## Contributing + +For contribution guidelines, including code style requirements and the pull request process, see [CONTRIBUTING.md](${project.scm.url}/blob/develop/CONTRIBUTING.md). + +## Next Steps + +- [Installation](installation.html) - Add the library to your project +- [Architecture](guides/architecture.html) - Understand the module structure diff --git a/src/site/markdown/claude-integration.md.vm b/src/site/markdown/claude-integration.md.vm new file mode 100644 index 000000000..c8c6c21f4 --- /dev/null +++ b/src/site/markdown/claude-integration.md.vm @@ -0,0 +1,112 @@ +# Using with Claude Code + +[Claude Code](https://claude.ai/code) is an AI-powered coding assistant that can help you work with Metaschema and OSCAL tools more effectively. + +## Overview + +Claude Code plugins provide specialized skills for: + +- **Metaschema authoring** - Creating and validating Metaschema modules +- **OSCAL document creation** - Building catalogs, profiles, SSPs +- **CLI assistance** - Running commands and interpreting output +- **Metapath expressions** - Writing and debugging queries + +## Prerequisites + +1. [Install Claude Code](https://docs.anthropic.com/en/docs/claude-code) +2. Install the [Metaschema plugins](https://github.com/metaschema-framework/claude-plugins/tree/main) + +## Available Plugins + +### Metaschema Plugin + +Skills for working with Metaschema definitions: + +| Skill | Description | +|:------|:------------| +| `metaschema:metaschema-basics` | Introduction to Metaschema concepts | +| `metaschema:metaschema-module-authoring` | Creating and modifying modules | +| `metaschema:metaschema-constraints-authoring` | Defining validation constraints | +| `metaschema:metapath-expressions` | Writing Metapath queries | + +### OSCAL Plugin + +Skills for working with OSCAL documents: + +| Skill | Description | +|:------|:------------| +| `oscal:oscal-basics` | OSCAL document types and structure | + +### Tool Plugins + +Skills for using CLI and library tools: + +| Skill | Description | +|:------|:------------| +| `oscal-tools:using-oscal-cli` | Running oscal-cli commands | +| `metaschema-tools:using-metaschema-java` | Using the Metaschema Java library | + +### Development Plugin + +For contributors to the Metaschema framework: + +| Skill | Description | +|:------|:------------| +| `dev-metaschema:development-workflow` | TDD, debugging, testing patterns | +| `dev-metaschema:repo-organization` | Project structure and relationships | +| `dev-metaschema:unit-test-writing` | Writing comprehensive tests | + +## Example Workflows + +### Generate Java Classes from Metaschema + +Ask Claude: +> "Help me set up the Maven plugin to generate Java classes from my Metaschema module" + +Claude will guide you through the plugin configuration and code generation process. + +### Write a Metapath Expression + +Ask Claude: +> "Help me write a Metapath expression that finds all controls with a specific property" + +Claude will use the Metapath skill to construct and explain the expression. + +### Validate a Metaschema Module + +Ask Claude: +> "Validate my Metaschema module at `src/main/metaschema/my-model.xml`" + +Claude will run validation and explain any errors. + +### Debug Constraint Errors + +Ask Claude: +> "I'm getting constraint validation errors. Can you help me understand what's wrong?" + +Claude will analyze the errors, explain the constraint requirements, and suggest fixes. + +## Tips for Effective Use + +1. **Be specific** - Mention file paths and exact error messages +2. **Provide context** - Share relevant portions of your Metaschema definitions +3. **Ask for explanations** - Claude can explain why something is required by the specification +4. **Request examples** - Ask for sample code or Metaschema snippets + +## Integration with This Library + +When working with the Metaschema Java library, Claude can help you: + +- Configure the Maven plugin for code generation +- Write code using generated binding classes +- Execute Metapath queries programmatically +- Implement custom constraint validators +- Troubleshoot serialization and deserialization issues + +## Learn More + +For more information about Claude Code and the technologies used in this project: + +- [Claude Code Documentation](https://docs.anthropic.com/en/docs/claude-code) +- [Metaschema Specification](https://framework.metaschema.dev/specification/) +- [Metapath Reference](https://framework.metaschema.dev/specification/metapath/) diff --git a/src/site/markdown/guides/architecture.md.vm b/src/site/markdown/guides/architecture.md.vm new file mode 100644 index 000000000..ae932ab52 --- /dev/null +++ b/src/site/markdown/guides/architecture.md.vm @@ -0,0 +1,358 @@ +# Architecture + +This guide explains the architecture and module structure of the Metaschema Java Tools. + +## Overview + +The Metaschema Java Tools project provides a modular framework for working with Metaschema-based data models. Understanding the architecture helps you choose the right components for your use case and understand how the pieces fit together. + +The framework is organized around five core capabilities: + +- **Loading and processing** - Parse Metaschema module definitions and resolve imports +- **Code generation** - Generate Java classes and XML/JSON schemas from modules +- **Data binding** - Serialize and deserialize data in XML, JSON, and YAML formats +- **Query execution** - Evaluate Metapath expressions against loaded data +- **Constraint validation** - Validate data against Metaschema-defined rules + +## Module Structure + +This section describes how the project is organized into Maven modules and Java packages. + +### Module Hierarchy + +The project is divided into several Maven modules, each with a specific responsibility. This separation allows you to include only what you need: + +``` +metaschema-java +├── metaschema-core # Core Metaschema model and Metapath +├── metaschema-model # Module loading and processing +├── metaschema-databind # Data binding framework +├── metaschema-schemagen # Schema generation (XSD, JSON Schema) +├── metaschema-cli # Command-line interface +└── metaschema-maven-plugin # Maven integration +``` + +### Core Modules + +Each module builds on the ones above it. The following sections describe what each module provides and show typical usage patterns. + +#### metaschema-core + +The foundation layer that all other modules depend on. It provides: + +- Metaschema model interfaces (`IModule`, `IAssemblyDefinition`, etc.) +- Metapath expression engine +- Constraint definitions +- Common utilities + +```java +// Core interfaces +IModule module; +IAssemblyDefinition assembly; +IFieldDefinition field; +IFlagDefinition flag; +IMetapathExpression expression; +``` + +#### metaschema-model + +Builds on `metaschema-core` to provide module loading and processing capabilities: + +- Metaschema XML parser +- Module resolution and imports +- Definition lookup + +```java +MetaschemaLoader loader = new MetaschemaLoader(); +IModule module = loader.load(path); +``` + +#### metaschema-databind + +The data binding layer that most applications will use directly. This module handles: + +- Serialization/deserialization +- Format support (XML, JSON, YAML) +- Binding context management +- Constraint validation during binding + +```java +IBindingContext context = IBindingContext.instance(); +IDeserializer reader = context.newDeserializer(Format.JSON, clazz); +ISerializer writer = context.newSerializer(Format.JSON, clazz); +``` + +#### metaschema-schemagen + +Schema generation: + +- XML Schema (XSD) generation +- JSON Schema generation + +```java +XmlSchemaGenerator xsdGen = new XmlSchemaGenerator(); +JsonSchemaGenerator jsonGen = new JsonSchemaGenerator(); +``` + +#### metaschema-cli + +Command-line tools: + +- Module validation +- Schema generation +- Document validation +- Metapath evaluation + +```bash +metaschema-cli validate module.xml +metaschema-cli generate-schema --as=xsd module.xml +metaschema-cli metapath eval -e "//control" document.json +``` + +#### metaschema-maven-plugin + +Maven integration: + +- Java class generation +- Schema generation +- Build integration + +```xml + + dev.metaschema.java + metaschema-maven-plugin + +``` + +### Package Structure + +The Java packages follow a consistent naming convention that mirrors the module structure. This organization makes it easy to locate classes based on their functionality: + +``` +dev.metaschema +├── core +│ ├── model # Core model interfaces +│ ├── metapath # Metapath engine +│ └── constraint # Constraint definitions +├── databind +│ ├── io # Serialization/deserialization +│ └── model # Binding model +├── schemagen +│ ├── xml # XSD generation +│ └── json # JSON Schema generation +└── cli # Command-line tools +``` + +## Key Abstractions + +Understanding the core interfaces helps you navigate the codebase and work effectively with the API. This section describes the main abstractions and their relationships. + +### Module System + +A Metaschema module represents a data model definition. The `IModule` interface provides access to all definitions within that module: + +``` +IModule +├── getName() # get the module's name +├── getXmlNamespace() # get the module's namespace used in XML +├── getAssemblyDefinitions() # get referenceable assembly definitions +├── getFieldDefinitions() # get referenceable field definitions +├── getFlagDefinitions() # get referenceable flag definitions +└── getImportedModules() # get other module's imported by this module +``` + +### Definition Hierarchy + +Metaschema defines three types of model elements. Assemblies are complex structures that contain other elements. Fields hold simple values but can have attributes (flags). Flags are the simplest elements, representing single attributes or properties: + +``` +IDefinition +├── IAssemblyDefinition # Complex structures +│ └── getModelInstances() +├── IFieldDefinition # Simple values with flags +│ └── getJavaTypeAdapter() +└── IFlagDefinition # Attributes/properties + └── isRequired() +``` + +### Binding Context + +The binding context is your main entry point for serialization operations. It knows about all registered model classes and creates serializers and deserializers on demand: + +``` +IBindingContext +├── newDeserializer(Format, Class) +├── newSerializer(Format, Class) +├── validate(Object) +├── getStaticContext() # For Metapath +└── registerModule(IModule) +``` + +### Metapath Engine + +Metapath expressions are compiled once and can be evaluated multiple times against different documents. The `StaticContext` provides namespace bindings and other configuration: + +``` +MetapathExpression +├── compile(String, StaticContext) +└── evaluate(Object) → ISequence + +StaticContext +├── builder() +├── namespace(prefix, uri) +└── baseUri(URI) +``` + +## Data Flow + +Understanding how data flows through the system helps you debug issues and optimize performance. This section traces data through the major operations. + +### Module Loading + +When you load a Metaschema module, the system parses the XML/JSON/YAML resource, resolves any imports, and builds an in-memory model: + +``` +Metaschema Module XML/JSON/YAML + ↓ +MetaschemaLoader.load() + ↓ +XML/JSON/YAML Module Parsing + ↓ +IModule (in-memory model) + ↓ +Definition Resolution + ↓ +Constraint Processing +``` + +### Code Generation + +The Maven plugin generates Java source code from your module definitions. JavaPoet handles the actual code generation, producing clean, readable Java files: + +``` +IModule + ↓ +ClassGenerator + ↓ +JavaPoet + ↓ +.java Source Files + ↓ +Maven Compilation + ↓ +.class Files +``` + +### Data Binding + +When deserializing data, the system detects the format, parses it using the appropriate library (Jackson for JSON/YAML, StAX for XML), and maps the content to your generated model objects: + +``` +JSON/XML/YAML File + ↓ +Format Detection + ↓ +Parser (Jackson/StAX) + ↓ +Binding Layer + ↓ +Generated Model Objects + ↓ +Optional Constraint Validation +``` + +### Metapath Evaluation + +Metapath expressions are first compiled into an abstract syntax tree, then evaluated against a document to produce a sequence of results: + +``` +Expression String + ↓ +MetapathExpression.compile() + ↓ +Abstract Syntax Tree + ↓ +evaluate(document) + ↓ +ISequence +``` + +## Development Considerations + +This section covers topics important when building applications with the framework. + +### Extension Points + +The framework provides several extension points for customization. These allow you to add domain-specific functionality without modifying the core libraries. + +#### Custom Functions + +You can register custom Metapath functions to extend the expression language with domain-specific operations: + +```java +StaticContext context = StaticContext.builder() + .function(myFunction) + .build(); +``` + +#### Custom Constraint Handlers + +To customize how constraint violations are reported or handled, implement a custom validation handler: + +```java +IConstraintValidationHandler handler = new MyHandler(); +deserializer.setConstraintValidationHandler(handler); +``` + +#### Custom Type Adapters + +For data types not natively supported by Metaschema, you can create custom type adapters that handle serialization and deserialization: + +```java +// For custom data types +IJavaTypeAdapter adapter = new MyTypeAdapter(); +``` + +### Threading Model + +When using the library in multi-threaded applications, it's important to understand which components are thread-safe. In general, immutable objects and read-only operations are safe to share across threads, while serializers and deserializers should be created per-use: + +| Component | Thread Safety | +|:----------|:--------------| +| `IBindingContext` | Thread-safe (read) | +| `IDeserializer` | Not thread-safe (create per use) | +| `ISerializer` | Not thread-safe (create per use) | +| `IModule` | Immutable (thread-safe) | +| `MetapathExpression` | Immutable (thread-safe) | + +### Dependencies + +The library has minimal external dependencies. At runtime, it relies on standard parsing libraries and logging infrastructure. Build-time dependencies are only needed when compiling from source or generating code. + +#### Runtime Dependencies + +- Jackson (JSON/YAML processing) +- StAX (XML processing) +- SLF4J (logging) +- SpotBugs annotations (null safety) + +#### Build Dependencies + +- Maven +- JavaPoet (code generation) + +## Related Projects + +| Project | Relationship | +|:--------|:-------------| +| [liboscal-java](https://github.com/metaschema-framework/liboscal-java) | OSCAL bindings built on this | +| [oscal-cli](https://github.com/metaschema-framework/oscal-cli) | CLI using OSCAL bindings | +| [Metaschema](https://github.com/metaschema-framework/metaschema) | Metaschema specification | + +## Next Steps + +Continue learning about the Metaschema Java Tools with these related guides: + +- [Loading Modules](loading-metaschema-modules.html) - Work with modules +- [Generating Java Classes](generating-java-classes.html) - Code generation +- [Installation](../installation.html) - Add to your project diff --git a/src/site/markdown/guides/executing-metapath.md.vm b/src/site/markdown/guides/executing-metapath.md.vm new file mode 100644 index 000000000..c6c090358 --- /dev/null +++ b/src/site/markdown/guides/executing-metapath.md.vm @@ -0,0 +1,327 @@ +# Executing Metapath + +This guide explains how to use Metapath expressions to query Metaschema-based data. + +## Overview + +Metapath is an expression language for querying Metaschema-based documents. It's similar to XPath but works across all serialization formats (XML, JSON, YAML). + +## Basic Metapath Evaluation + +```java +import dev.metaschema.core.metapath.MetapathExpression; +import dev.metaschema.core.metapath.IMetapathExpression; +import dev.metaschema.core.metapath.StaticContext; +import dev.metaschema.core.metapath.item.ISequence; +import dev.metaschema.core.metapath.item.IItem; + +// Create static context +StaticContext staticContext = StaticContext.builder().build(); + +// Compile expression +IMetapathExpression expression = MetapathExpression.compile( + "//control", staticContext); + +// Evaluate against a document +ISequence results = expression.evaluate(document); + +// Process results +for (IItem item : results) { + System.out.println("Found: " + item); +} +``` + +## Path Expressions + +### Basic Navigation + +```java +// Root element +"/catalog" + +// All descendants named 'control' +"//control" + +// Direct children +"catalog/group" + +// Parent +".." + +// Current node +"." +``` + +### Predicates + +```java +// By position +"//control[1]" // First control +"//control[last()]" // Last control +"//control[position() < 5]" // First four + +// By attribute/flag +"//control[@id='ac-1']" // Control with id='ac-1' + +// By child existence +"//control[title]" // Controls with title child +``` + +### Combining Paths + +```java +// Union +"//assembly | //field" + +// Multiple predicates +"//control[@id='ac-1'][title]" +``` + +## Working with Results + +### Getting Atomic Values + +```java +IMetapathExpression expr = MetapathExpression.compile( + "//control/@id", staticContext); +ISequence results = expr.evaluate(document); + +results.forEach(item -> { + String id = item.toAtomicItem().asString(); + System.out.println("ID: " + id); +}); +``` + +### Getting Bound Objects + +```java +import dev.metaschema.databind.model.IBoundObject; + +ISequence results = expression.evaluate(document); + +for (IItem item : results) { + if (item instanceof IBoundObject) { + Object value = ((IBoundObject) item).getValue(); + // Work with the Java object + } +} +``` + +### Counting Results + +```java +IMetapathExpression expr = MetapathExpression.compile( + "count(//control)", staticContext); +ISequence result = expr.evaluate(document); + +int count = result.getFirstItem(true) + .toAtomicItem() + .asInteger() + .intValue(); +``` + +## Built-in Functions + +### String Functions + +```java +// String length +"string-length(//title)" + +// Substring +"substring(//id, 1, 3)" + +// String matching +"starts-with(@id, 'ac-')" +"ends-with(@id, '-1')" +"contains(title, 'Access')" + +// Case conversion +"upper-case(title)" +"lower-case(title)" + +// Concatenation +"concat(title, ' - ', @id)" +``` + +### Numeric Functions + +```java +// Count +"count(//control)" + +// Sum +"sum(//value)" + +// Math +"ceiling(3.14)" +"floor(3.14)" +"round(3.14)" +"abs(-5)" +``` + +### Boolean Functions + +```java +// Existence +"exists(//control)" +"empty(//withdrawn)" + +// Logical +"not(//control)" +"true()" +"false()" +``` + +### Sequence Functions + +```java +// First/last +"head(//control)" +"tail(//control)" + +// Reverse +"reverse(//control)" + +// Distinct values +"distinct-values(//prop/@name)" +``` + +## Static Context Configuration + +### Default Configuration + +```java +StaticContext staticContext = StaticContext.builder().build(); +``` + +### With Base URI + +```java +StaticContext staticContext = StaticContext.builder() + .baseUri(URI.create("https://example.com/")) + .build(); +``` + +### With Namespace Bindings + +```java +StaticContext staticContext = StaticContext.builder() + .namespace("oscal", "http://csrc.nist.gov/ns/oscal/1.0") + .build(); +``` + +## Caching Compiled Expressions + +```java +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + +public class MetapathCache { + private final Map cache = + new ConcurrentHashMap<>(); + private final StaticContext staticContext; + + public MetapathCache(StaticContext staticContext) { + this.staticContext = staticContext; + } + + public IMetapathExpression get(String path) { + return cache.computeIfAbsent(path, p -> + MetapathExpression.compile(p, staticContext)); + } + + public ISequence evaluate(Object document, String path) { + return get(path).evaluate(document); + } +} +``` + +## Error Handling + +```java +import dev.metaschema.core.metapath.MetapathException; + +try { + IMetapathExpression expr = MetapathExpression.compile( + "//control[invalid", staticContext); +} catch (MetapathException e) { + System.err.println("Invalid expression: " + e.getMessage()); +} +``` + +## Common Patterns + +### Find by Attribute + +```java +// Find element with specific ID +"//control[@id='ac-1']" + +// Find elements with any value for attribute +"//control[@id]" + +// Find elements without attribute +"//control[not(@id)]" +``` + +### Navigate Relationships + +```java +// Children +"control/*" + +// Specific child +"control/title" + +// All descendants +"catalog//control" + +// Parent's other children (siblings) +"../control" +``` + +### Filter by Content + +```java +// Elements containing text +"//control[contains(., 'security')]" + +// Elements with specific child value +"//control[title = 'Access Control']" +``` + +### Aggregate Data + +```java +// Count by type +"count(//control[prop[@name='type'][@value='technical']])" + +// Get unique values +"distinct-values(//prop/@name)" +``` + +## Metapath vs XPath + +| Feature | Metapath | XPath | +|---------|----------|-------| +| Format support | XML, JSON, YAML | XML only | +| Axis support | Simplified | Full | +| Node types | Metaschema types | XML nodes | +| Functions | Metaschema-specific | XPath functions | + +## Best Practices + +1. **Compile once, use many** - Cache compiled expressions +2. **Be specific** - Narrow paths are more efficient +3. **Use predicates** - Filter in expression, not in Java +4. **Handle empty results** - Check sequence length +5. **Test expressions** - Validate before embedding + +## Next Steps + +Continue learning about the Metaschema Java Tools with these related guides: + +- [Validating with Constraints](validating-with-constraints.html) - Validate data +- [Reading & Writing Data](reading-writing-data.html) - Load documents +- [Architecture](architecture.html) - Understand Metapath internals diff --git a/src/site/markdown/guides/generating-java-classes.md.vm b/src/site/markdown/guides/generating-java-classes.md.vm new file mode 100644 index 000000000..31895e1e2 --- /dev/null +++ b/src/site/markdown/guides/generating-java-classes.md.vm @@ -0,0 +1,324 @@ +# Generating Java Classes + +This guide explains how to generate Java binding classes from Metaschema modules. + +## Why Generate Classes? + +When working with structured data, you have several options: parse XML/JSON manually, use generic tree structures like DOM or JsonNode, or use strongly-typed classes. Strongly-typed classes offer significant advantages: + +- **Compile-time safety** - The compiler catches type errors before runtime +- **IDE support** - Code completion, refactoring, and navigation work automatically +- **Self-documenting code** - Method names like `getCatalog().getMetadata().getTitle()` clearly express intent +- **No boilerplate** - Serialization and deserialization are handled automatically + +The challenge is keeping your Java classes synchronized with your data model. If the model changes, you need to update the classes—a tedious and error-prone process. The Metaschema Maven plugin solves this by generating Java classes directly from Metaschema module definitions. When your model changes, regenerate the classes and the compiler tells you what code needs updating. + +## Overview + +The Metaschema Maven plugin integrates into your Maven build to generate Java classes from Metaschema module definitions. These generated classes provide: + +- **Type-safe data binding** - Fields have proper Java types matching Metaschema definitions +- **Automatic serialization** - Read and write XML, JSON, and YAML without manual parsing +- **IDE integration** - Full code completion and type checking in your IDE +- **Compile-time validation** - Catch errors at build time rather than runtime + +## Maven Plugin Configuration + +### Basic Configuration + +Add to your `pom.xml`: + +```xml + + + + dev.metaschema.java + metaschema-maven-plugin + ${project.version} + + + generate-sources + + generate-sources + + + src/main/metaschema + + my-model_metaschema.xml + + + + + + + +``` + +### Full Configuration Options + +```xml + + + src/main/metaschema + + + + **/*_metaschema.xml + + + + + **/draft-*.xml + + + + ${project.build.directory}/generated-sources/metaschema + +``` + +## Running Generation + +```bash +# Generate sources +mvn generate-sources + +# Or as part of full build +mvn compile +``` + +Generated classes appear in `target/generated-sources/metaschema/`. + +## Understanding Generated Classes + +### From Metaschema Definition + +```xml + + Catalog + A collection of controls. + catalog + + Catalog UUID + + + + + + +``` + +### Generated Java Class + +```java +@MetaschemaAssembly( + name = "catalog", + rootName = "catalog" +) +public class Catalog { + @BoundFlag(name = "uuid", required = true) + private UUID uuid; + + @BoundAssembly(minOccurs = 1) + private Metadata metadata; + + @BoundAssembly(maxOccurs = -1) + private List groups; + + // Getters and setters + public UUID getUuid() { return uuid; } + public void setUuid(UUID uuid) { this.uuid = uuid; } + + public Metadata getMetadata() { return metadata; } + public void setMetadata(Metadata metadata) { this.metadata = metadata; } + + public List getGroups() { return groups; } + public void setGroups(List groups) { this.groups = groups; } +} +``` + +## Generated Class Features + +### Type Mapping + +| Metaschema Type | Java Type | +|-----------------|-----------| +| `uuid` | `java.util.UUID` | +| `date-time` | `java.time.ZonedDateTime` | +| `string` | `String` | +| `integer` | `java.math.BigInteger` | +| `boolean` | `Boolean` | +| `uri` | `java.net.URI` | +| `markup-line` | `MarkupLine` | +| `markup-multiline` | `MarkupMultiline` | + +### Cardinality Handling + +| Metaschema | Java | +|------------|------| +| `min-occurs="0"` | Nullable field | +| `min-occurs="1"` | `@NonNull` on getter/setter methods | +| `max-occurs="1"` | Single instance | +| `max-occurs="unbounded"` | `List` | + +### Annotations + +Generated classes include annotations for: + +```java +@MetaschemaAssembly // Assembly definitions +@MetaschemaField // Field definitions +@BoundFlag // Flag instances +@BoundField // Field instances +@BoundAssembly // Assembly instances +``` + +## Using Generated Classes + +### Creating Instances + +```java +Catalog catalog = new Catalog(); +catalog.setUuid(UUID.randomUUID()); + +Metadata metadata = new Metadata(); +metadata.setTitle(MarkupLine.fromMarkdown("My Catalog")); +metadata.setLastModified(ZonedDateTime.now()); +catalog.setMetadata(metadata); +``` + +### Serialization + +```java +import dev.metaschema.databind.IBindingContext; +import dev.metaschema.databind.io.Format; +import dev.metaschema.databind.io.ISerializer; + +IBindingContext context = IBindingContext.instance(); +ISerializer serializer = context.newSerializer(Format.JSON, Catalog.class); +serializer.serialize(catalog, Path.of("catalog.json")); +``` + +### Deserialization + +```java +IDeserializer deserializer = context.newDeserializer( + Format.JSON, Catalog.class); +Catalog catalog = deserializer.deserialize(Path.of("catalog.json")); +``` + +## Multiple Modules + +### Generating from Multiple Files + +```xml + + + core_metaschema.xml + extension_metaschema.xml + + +``` + +### Module Dependencies + +If modules import each other: + +```xml + + + ... + + + + + + + + + + + +``` + +All imported modules are processed automatically. + +## Customizing Generation + +### Package Names + +Control package via namespace: + +```xml + + My Model + http://example.com/ns/mymodel + + +``` + +### Abstract Base Classes + +The plugin generates abstract base classes for extension: + +```java +// Generated abstract class +public abstract class AbstractCatalog { + // Generated fields and methods +} + +// Your extension (in src/main/java) +public class Catalog extends AbstractCatalog { + // Additional methods +} +``` + +## Troubleshooting + +### Classes Not Generated + +**Symptom:** No classes in `target/generated-sources/` + +**Fix:** +1. Check `metaschemaDir` path is correct +2. Verify `includes` pattern matches files +3. Run `mvn generate-sources -X` for debug output + +### Compilation Errors + +**Symptom:** Generated classes won't compile + +**Fix:** +1. Ensure all imported modules are accessible +2. Check for circular dependencies +3. Verify Metaschema syntax is valid + +### Missing Dependencies + +**Symptom:** Annotations not found + +**Fix:** Add runtime dependency: + +```xml + + dev.metaschema.java + metaschema-databind + ${project.version} + +``` + +## Best Practices + +1. **Use consistent naming** - Follow `*_metaschema.xml` convention +2. **Organize by module** - One module per domain concept +3. **Document in Metaschema** - Descriptions become Javadoc +4. **Version your schemas** - Track schema changes in VCS +5. **Don't edit generated code** - Use extension classes instead + +## Next Steps + +Continue learning about the Metaschema Java Tools with these related guides: + +- [Loading Modules](loading-metaschema-modules.html) - Load modules at runtime +- [Reading & Writing Data](reading-writing-data.html) - Use generated classes +- [Architecture](architecture.html) - Understand the generation pipeline diff --git a/src/site/markdown/guides/generating-schemas.md.vm b/src/site/markdown/guides/generating-schemas.md.vm new file mode 100644 index 000000000..3ea63fd0a --- /dev/null +++ b/src/site/markdown/guides/generating-schemas.md.vm @@ -0,0 +1,336 @@ +# Generating Schemas + +This guide explains how to generate XML Schema (XSD) and JSON Schema from Metaschema modules. + +## Overview + +The Metaschema tools can generate: + +- **XML Schema (XSD)** - For validating XML documents +- **JSON Schema** - For validating JSON documents + +## Maven Plugin Configuration + +### Basic Configuration + +```xml + + + + dev.metaschema.java + metaschema-maven-plugin + ${project.version} + + + + generate-xsd + + generate-xsd + + + src/main/metaschema + + my-model_metaschema.xml + + ${project.build.directory}/schemas/xsd + + + + + + generate-json-schema + + generate-json-schema + + + src/main/metaschema + + my-model_metaschema.xml + + ${project.build.directory}/schemas/json + + + + + + +``` + +### Running Generation + +```bash +# Generate all schemas +mvn generate-resources + +# Generate only XSD +mvn metaschema:generate-xsd + +# Generate only JSON Schema +mvn metaschema:generate-json-schema +``` + +## Using the CLI + +### Generate XML Schema + +```bash +metaschema-cli generate-schema --as=xsd model_metaschema.xml output.xsd +``` + +### Generate JSON Schema + +```bash +metaschema-cli generate-schema --as=json model_metaschema.xml output.json +``` + +## Programmatic Generation + +### Generate XML Schema + +```java +import dev.metaschema.core.model.IModule; +import dev.metaschema.core.model.MetaschemaLoader; +import dev.metaschema.schemagen.xml.XmlSchemaGenerator; + +import java.io.Writer; +import java.nio.file.Files; +import java.nio.file.Path; + +// Load module +MetaschemaLoader loader = new MetaschemaLoader(); +IModule module = loader.load(Path.of("model_metaschema.xml")); + +// Generate XSD +XmlSchemaGenerator generator = new XmlSchemaGenerator(); +try (Writer writer = Files.newBufferedWriter(Path.of("output.xsd"))) { + generator.generateFromModule(module, writer); +} +``` + +### Generate JSON Schema + +```java +import dev.metaschema.core.model.IModule; +import dev.metaschema.core.model.MetaschemaLoader; +import dev.metaschema.schemagen.json.JsonSchemaGenerator; + +import java.io.Writer; +import java.nio.file.Files; +import java.nio.file.Path; + +// Load module +MetaschemaLoader loader = new MetaschemaLoader(); +IModule module = loader.load(Path.of("model_metaschema.xml")); + +// Generate JSON Schema +JsonSchemaGenerator generator = new JsonSchemaGenerator(); +try (Writer writer = Files.newBufferedWriter(Path.of("output.json"))) { + generator.generateFromModule(module, writer); +} +``` + +## Schema Features + +### XML Schema Output + +Generated XSD includes: + +- Complex types for assemblies +- Simple types for fields and flags +- Element declarations +- Namespace declarations +- Documentation from Metaschema descriptions + +```xml + + + + + + + + A collection of controls. + + + + + + + + + +``` + +### JSON Schema Output + +Generated JSON Schema includes: + +- Object definitions for assemblies +- Property definitions for fields and flags +- Required property arrays +- Type constraints +- Description annotations + +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "definitions": { + "catalog": { + "type": "object", + "description": "A collection of controls.", + "properties": { + "uuid": { + "type": "string", + "format": "uuid" + }, + "metadata": { + "$ref": "#/definitions/metadata" + }, + "controls": { + "type": "array", + "items": { + "$ref": "#/definitions/control" + } + } + }, + "required": ["uuid", "metadata"] + } + } +} +``` + +## Validation with Generated Schemas + +### XML Validation + +```java +import javax.xml.XMLConstants; +import javax.xml.transform.stream.StreamSource; +import javax.xml.validation.Schema; +import javax.xml.validation.SchemaFactory; +import javax.xml.validation.Validator; + +import java.io.File; + +import org.xml.sax.SAXException; + +SchemaFactory factory = SchemaFactory.newInstance( + XMLConstants.W3C_XML_SCHEMA_NS_URI); +Schema schema = factory.newSchema(new File("output.xsd")); +Validator validator = schema.newValidator(); + +try { + validator.validate(new StreamSource(new File("document.xml"))); + System.out.println("Valid!"); +} catch (SAXException e) { + System.err.println("Invalid: " + e.getMessage()); +} +``` + +### JSON Validation + +Using a JSON Schema validator library (e.g., [networknt/json-schema-validator](https://github.com/networknt/json-schema-validator)): + +```java +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.networknt.schema.JsonSchema; +import com.networknt.schema.JsonSchemaFactory; +import com.networknt.schema.SpecVersion; +import com.networknt.schema.ValidationMessage; + +import java.io.File; +import java.util.Set; + +ObjectMapper objectMapper = new ObjectMapper(); +JsonSchemaFactory factory = JsonSchemaFactory.getInstance( + SpecVersion.VersionFlag.V7); +JsonSchema schema = factory.getSchema( + objectMapper.readTree(new File("output.json"))); + +JsonNode document = objectMapper.readTree(new File("document.json")); +Set errors = schema.validate(document); + +if (errors.isEmpty()) { + System.out.println("Valid!"); +} else { + errors.forEach(err -> System.err.println(err.getMessage())); +} +``` + +## Customizing Output + +### XSD Configuration + +```xml + + src/main/metaschema + + model_metaschema.xml + + ${project.build.directory}/schemas/xsd + +``` + +### JSON Schema Configuration + +```xml + + src/main/metaschema + + model_metaschema.xml + + ${project.build.directory}/schemas/json + +``` + +## Schema Generation vs Constraints + +| Aspect | Schema Validation | Constraint Validation | +|--------|-------------------|----------------------| +| Format | XSD/JSON Schema | Metaschema constraints | +| Scope | Structure, types | Business rules | +| Cross-references | Limited | Full support | +| Custom rules | No | Metapath expressions | +| Standard tools | Yes | Metaschema tools | + +**Recommendation:** Use both for comprehensive validation: +1. Schema for structural validation +2. Constraints for semantic validation + +## Common Issues + +### Missing Imports + +**Symptom:** Generated schema references undefined types + +**Fix:** Ensure all imported modules are accessible + +### Namespace Conflicts + +**Symptom:** Multiple schemas with same namespace + +**Fix:** Use unique namespaces per module + +### Output Directory + +**Symptom:** Schemas not found after generation + +**Fix:** Check `outputDirectory` configuration + +## Best Practices + +1. **Generate during build** - Include in Maven lifecycle +2. **Version schemas** - Track schema changes +3. **Publish schemas** - Make available for consumers +4. **Use both schema types** - Support XML and JSON users +5. **Include in documentation** - Reference from API docs + +## Next Steps + +Continue learning about the Metaschema Java Tools with these related guides: + +- [Loading Modules](loading-metaschema-modules.html) - Load modules for generation +- [Validating with Constraints](validating-with-constraints.html) - Beyond schema validation +- [Architecture](architecture.html) - Schema generation internals diff --git a/src/site/markdown/guides/loading-metaschema-modules.md.vm b/src/site/markdown/guides/loading-metaschema-modules.md.vm new file mode 100644 index 000000000..8aac93e59 --- /dev/null +++ b/src/site/markdown/guides/loading-metaschema-modules.md.vm @@ -0,0 +1,241 @@ +# Loading Metaschema Modules + +This guide explains how to load and work with Metaschema module definitions. + +## Overview + +Metaschema modules define the structure of data models. Loading a module allows you to: + +- Generate Java binding classes +- Validate documents against the model +- Execute Metapath expressions +- Generate XML/JSON schemas + +## Loading a Module from a File + +```java +import dev.metaschema.core.model.IModule; +import dev.metaschema.core.model.MetaschemaLoader; + +import java.nio.file.Path; + +// Create a loader +MetaschemaLoader loader = new MetaschemaLoader(); + +// Load a module +IModule module = loader.load(Path.of("my-metaschema.xml")); + +// Access module information +System.out.println("Module: " + module.getName()); +System.out.println("Namespace: " + module.getXmlNamespace()); +``` + +## Loading from Resources + +```java +import java.net.URL; + +URL resource = getClass().getResource("/metaschema/my-model.xml"); +IModule module = loader.load(resource); +``` + +## Loading from URI + +```java +import java.net.URI; + +IModule module = loader.load( + URI.create("https://example.com/schemas/model.xml")); +``` + +## Understanding Module Structure + +A loaded module contains: + +```java +// Get all definitions +module.getAssemblyDefinitions().forEach(assembly -> { + System.out.println("Assembly: " + assembly.getName()); +}); + +module.getFieldDefinitions().forEach(field -> { + System.out.println("Field: " + field.getName()); +}); + +module.getFlagDefinitions().forEach(flag -> { + System.out.println("Flag: " + flag.getName()); +}); +``` + +## Module Hierarchy + +Modules can import other modules: + +```java +// Get imported modules +module.getImportedModules().forEach(imported -> { + System.out.println("Imports: " + imported.getName()); +}); + +// Get the complete module chain (recursive) +module.getImportedModules(); +``` + +## Working with Definitions + +### Assembly Definitions + +Assemblies are complex structures containing other elements: + +```java +import dev.metaschema.core.model.IAssemblyDefinition; + +IAssemblyDefinition assembly = module.getAssemblyDefinitionByName("catalog"); +if (assembly != null) { + System.out.println("Root: " + assembly.isRoot()); + + // Get child definitions + assembly.getModelInstances().forEach(instance -> { + System.out.println(" Contains: " + instance.getName()); + }); +} +``` + +### Field Definitions + +Fields are simple values with optional flags: + +```java +import dev.metaschema.core.model.IFieldDefinition; + +IFieldDefinition field = module.getFieldDefinitionByName("title"); +if (field != null) { + System.out.println("Data type: " + field.getJavaTypeAdapter()); +} +``` + +### Flag Definitions + +Flags are attributes/properties on assemblies and fields: + +```java +import dev.metaschema.core.model.IFlagDefinition; + +IFlagDefinition flag = module.getFlagDefinitionByName("id"); +if (flag != null) { + System.out.println("Required: " + flag.isRequired()); +} +``` + +## Accessing Constraints + +Modules define validation constraints: + +```java +import dev.metaschema.core.model.constraint.IConstraint; + +// Get constraints from a definition +assembly.getConstraints().forEach(constraint -> { + System.out.println("Constraint: " + constraint.getId()); + System.out.println("Level: " + constraint.getLevel()); +}); +``` + +## Module Caching + +The loader caches modules by URI to avoid reloading: + +```java +// Same module, loaded once +IModule module1 = loader.load(Path.of("model.xml")); +IModule module2 = loader.load(Path.of("model.xml")); +// module1 == module2 (same instance) +``` + +## Error Handling + +```java +import dev.metaschema.core.model.MetaschemaException; + +import java.io.IOException; +import java.nio.file.Path; + +try { + IModule module = loader.load(Path.of("invalid.xml")); +} catch (MetaschemaException e) { + System.err.println("Failed to load module: " + e.getMessage()); +} catch (IOException e) { + System.err.println("IO error: " + e.getMessage()); +} +``` + +## Common Use Cases + +### Validate Module Structure + +```java +IModule module = loader.load(path); + +// Check for root assembly (document root) +boolean hasRoot = module.getAssemblyDefinitions().stream() + .anyMatch(IAssemblyDefinition::isRoot); + +if (!hasRoot) { + System.err.println("Module has no root assembly"); +} +``` + +### List All Definitions + +```java +public void listDefinitions(IModule module) { + System.out.println("=== " + module.getName() + " ==="); + + System.out.println("\nAssemblies:"); + module.getAssemblyDefinitions().forEach(a -> + System.out.println(" " + a.getName() + + (a.isRoot() ? " (root)" : ""))); + + System.out.println("\nFields:"); + module.getFieldDefinitions().forEach(f -> + System.out.println(" " + f.getName())); + + System.out.println("\nFlags:"); + module.getFlagDefinitions().forEach(f -> + System.out.println(" " + f.getName())); +} +``` + +### Find Definition by Name + +```java +import dev.metaschema.core.model.IAssemblyDefinition; +import dev.metaschema.core.model.IFieldDefinition; +import dev.metaschema.core.model.IModule; +import dev.metaschema.core.model.INamedModelDefinition; + +public INamedModelDefinition findDefinition(IModule module, String name) { + IAssemblyDefinition assembly = module.getAssemblyDefinitionByName(name); + if (assembly != null) return assembly; + + IFieldDefinition field = module.getFieldDefinitionByName(name); + if (field != null) return field; + + return module.getFlagDefinitionByName(name); +} +``` + +## Best Practices + +1. **Reuse loaders** - The loader caches modules +2. **Handle errors** - Modules may have syntax errors +3. **Check for imports** - Modules may depend on others +4. **Use absolute paths** - Relative paths resolved from loader location + +## Next Steps + +Continue learning about the Metaschema Java Tools with these related guides: + +- [Generating Java Classes](generating-java-classes.html) - Generate code from modules +- [Reading & Writing Data](reading-writing-data.html) - Work with module data +- [Architecture](architecture.html) - Understand the module system diff --git a/src/site/markdown/guides/reading-writing-data.md.vm b/src/site/markdown/guides/reading-writing-data.md.vm new file mode 100644 index 000000000..83308ba1e --- /dev/null +++ b/src/site/markdown/guides/reading-writing-data.md.vm @@ -0,0 +1,304 @@ +# Reading & Writing Data + +This guide explains how to read, write, and convert data using Metaschema-generated classes. + +## Overview + +The Metaschema databind library provides serialization and deserialization for: + +- **XML** - Full XML support with namespace handling +- **JSON** - JSON object serialization +- **YAML** - YAML document support + +## Getting a Binding Context + +```java +import dev.metaschema.databind.IBindingContext; +import dev.metaschema.databind.DefaultBindingContext; + +// For Metaschema-generated classes +IBindingContext context = new DefaultBindingContext(); + +// Or use a pre-configured context (e.g., OscalBindingContext) +``` + +## Reading Data + +### From a File + +```java +import dev.metaschema.databind.io.Format; +import dev.metaschema.databind.io.IDeserializer; + +import java.nio.file.Path; + +// Create deserializer +IDeserializer deserializer = context.newDeserializer( + Format.JSON, MyModel.class); + +// Read from file +MyModel model = deserializer.deserialize(Path.of("data.json")); +``` + +### From Different Formats + +```java +// JSON +IDeserializer jsonReader = context.newDeserializer( + Format.JSON, MyModel.class); +MyModel fromJson = jsonReader.deserialize(Path.of("data.json")); + +// XML +IDeserializer xmlReader = context.newDeserializer( + Format.XML, MyModel.class); +MyModel fromXml = xmlReader.deserialize(Path.of("data.xml")); + +// YAML +IDeserializer yamlReader = context.newDeserializer( + Format.YAML, MyModel.class); +MyModel fromYaml = yamlReader.deserialize(Path.of("data.yaml")); +``` + +### From URL + +```java +import java.net.URL; + +URL url = new URL("https://example.com/data.json"); +MyModel model = deserializer.deserialize(url); +``` + +### From InputStream + +```java +import java.io.InputStream; +import java.net.URI; + +try (InputStream is = getClass().getResourceAsStream("/data.json")) { + MyModel model = deserializer.deserialize(is, + URI.create("classpath:/data.json")); +} +``` + +### Auto-detecting Format + +```java +Path file = Path.of("data.json"); +Format format = Format.valueOf(file); // Detects from extension + +IDeserializer deserializer = context.newDeserializer( + format, MyModel.class); +MyModel model = deserializer.deserialize(file); +``` + +## Writing Data + +### To a File + +```java +import dev.metaschema.databind.io.ISerializer; + +ISerializer serializer = context.newSerializer( + Format.JSON, MyModel.class); +serializer.serialize(model, Path.of("output.json")); +``` + +### To Different Formats + +```java +// JSON +ISerializer jsonWriter = context.newSerializer( + Format.JSON, MyModel.class); +jsonWriter.serialize(model, Path.of("output.json")); + +// XML +ISerializer xmlWriter = context.newSerializer( + Format.XML, MyModel.class); +xmlWriter.serialize(model, Path.of("output.xml")); + +// YAML +ISerializer yamlWriter = context.newSerializer( + Format.YAML, MyModel.class); +yamlWriter.serialize(model, Path.of("output.yaml")); +``` + +### To OutputStream + +```java +import java.io.OutputStream; +import java.nio.file.Files; + +try (OutputStream os = Files.newOutputStream(Path.of("output.json"))) { + serializer.serialize(model, os); +} +``` + +### To String + +```java +import java.io.StringWriter; + +StringWriter writer = new StringWriter(); +serializer.serialize(model, writer); +String json = writer.toString(); +``` + +## Converting Between Formats + +```java +public void convert(Path input, Path output, + Format inputFormat, Format outputFormat, Class clazz) { + + IBindingContext context = new DefaultBindingContext(); + + // Read + IDeserializer reader = context.newDeserializer(inputFormat, clazz); + MyModel model = reader.deserialize(input); + + // Write + ISerializer writer = context.newSerializer(outputFormat, clazz); + writer.serialize(model, output); +} + +// Usage +convert( + Path.of("data.xml"), + Path.of("data.json"), + Format.XML, + Format.JSON, + MyModel.class +); +``` + +## Validation During Read + +### Enable Constraint Validation + +```java +IDeserializer deserializer = context.newDeserializer( + Format.JSON, MyModel.class); + +// Enable validation +deserializer.setConstraintValidationEnabled(true); + +// Read and validate +MyModel model = deserializer.deserialize(Path.of("data.json")); +``` + +### Custom Validation Handler + +```java +import dev.metaschema.core.model.constraint.IConstraintValidationHandler; + +IConstraintValidationHandler handler = new MyValidationHandler(); +deserializer.setConstraintValidationHandler(handler); +``` + +## Error Handling + +```java +import dev.metaschema.databind.io.DeserializationException; +import dev.metaschema.databind.io.SerializationException; + +// Reading +try { + MyModel model = deserializer.deserialize(path); +} catch (DeserializationException e) { + System.err.println("Parse error: " + e.getMessage()); +} catch (IOException e) { + System.err.println("IO error: " + e.getMessage()); +} + +// Writing +try { + serializer.serialize(model, path); +} catch (SerializationException e) { + System.err.println("Serialization error: " + e.getMessage()); +} catch (IOException e) { + System.err.println("IO error: " + e.getMessage()); +} +``` + +## Working with Generic Data + +### Using BoundLoader + +```java +import dev.metaschema.databind.io.DefaultBoundLoader; +import dev.metaschema.databind.model.IBoundObject; + +DefaultBoundLoader loader = new DefaultBoundLoader(context); + +// Load any supported document +IBoundObject document = loader.load(Path.of("unknown-type.json")); + +// Determine type +Class type = document.getClass(); +System.out.println("Loaded: " + type.getSimpleName()); +``` + +## Batch Processing + +```java +import java.io.IOException; +import java.io.UncheckedIOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.stream.Stream; + +public void convertDirectory(Path inputDir, Path outputDir, + Format inputFormat, Format outputFormat) throws IOException { + + IBindingContext context = IBindingContext.newInstance(); + + try (Stream files = Files.list(inputDir)) { + files.filter(p -> Format.valueOf(p) == inputFormat) + .forEach(inputPath -> { + try { + // Read + IDeserializer reader = context.newDeserializer( + inputFormat, MyModel.class); + MyModel model = reader.deserialize(inputPath); + + // Write + String name = inputPath.getFileName().toString(); + String outputName = name.replaceAll( + "\\.[^.]+$", outputFormat.getDefaultExtension()); + Path outputPath = outputDir.resolve(outputName); + + ISerializer writer = context.newSerializer( + outputFormat, MyModel.class); + writer.serialize(model, outputPath); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + }); + } +} +``` + +## Best Practices + +1. **Reuse context** - Create once, use throughout application +2. **Use try-with-resources** - For streams and readers +3. **Handle errors** - Catch specific exceptions +4. **Validate input** - Enable constraint validation for untrusted data +5. **Detect format** - Use `Format.valueOf(Path)` when format unknown + +## Format Comparison + +| Aspect | XML | JSON | YAML | +|--------|-----|------|------| +| Readability | Good | Good | Excellent | +| File size | Larger | Medium | Medium | +| Comments | Yes | No | Yes | +| Namespaces | Full | Via properties | Via properties | +| Parsing speed | Medium | Fast | Slower | + +## Next Steps + +Continue learning about the Metaschema Java Tools with these related guides: + +- [Executing Metapath](executing-metapath.html) - Query loaded data +- [Validating with Constraints](validating-with-constraints.html) - Validate data +- [Generating Java Classes](generating-java-classes.html) - Generate binding classes diff --git a/src/site/markdown/guides/validating-with-constraints.md.vm b/src/site/markdown/guides/validating-with-constraints.md.vm new file mode 100644 index 000000000..5f526b8d8 --- /dev/null +++ b/src/site/markdown/guides/validating-with-constraints.md.vm @@ -0,0 +1,323 @@ +# Validating with Constraints + +This guide explains how to define and validate Metaschema constraints. + +## Overview + +Metaschema constraints provide validation beyond schema structure: + +- **Cardinality** - Required fields, occurrence limits +- **Allowed values** - Enumerated restrictions +- **Patterns** - Regex-based validation +- **Uniqueness** - Key constraints +- **Cross-references** - Reference integrity +- **Custom rules** - Metapath-based assertions + +## Constraint Types + +### Allowed Values + +Restrict values to a defined set: + +```xml + + + + Active status + Inactive status + Pending status + + + +``` + +### Matches (Pattern) + +Validate against a regular expression: + +```xml + + + + + +``` + +### Cardinality + +Control occurrence requirements: + +```xml + + + + + + + + + +``` + +### Index (Uniqueness) + +Ensure unique values within a scope: + +```xml + + + + + + + +``` + +### Index-Has-Key (Reference Integrity) + +Validate references point to existing values: + +```xml + + + + + + + +``` + +### Expect (Assertion) + +Custom Metapath-based validation: + +```xml + + + + + +``` + +## Validation in Java + +### During Deserialization + +```java +import dev.metaschema.databind.IBindingContext; +import dev.metaschema.databind.io.DeserializationFeature; +import dev.metaschema.databind.io.IBoundLoader; + +import java.nio.file.Path; + +IBindingContext context = IBindingContext.newInstance(); +IBoundLoader loader = context.newBoundLoader(); + +// Enable constraint validation during loading +loader.enableFeature(DeserializationFeature.DESERIALIZE_VALIDATE_CONSTRAINTS); + +Object model = loader.load(Path.of("data.json")); +``` + +### Post-load Validation + +```java +import dev.metaschema.core.model.validation.IValidationResult; +import dev.metaschema.databind.IBindingContext; + +import java.net.URI; +import java.nio.file.Path; + +IBindingContext context = IBindingContext.newInstance(); +URI target = Path.of("data.json").toUri(); + +IValidationResult result = context.validateWithConstraints(target, null); + +if (!result.isPassing()) { + result.getFindings().forEach(finding -> { + System.err.println(finding.getSeverity() + ": " + + finding.getMessage() + " at " + finding.getLocation()); + }); +} +``` + +## Working with Validation Results + +```java +import dev.metaschema.core.model.constraint.IConstraint.Level; +import dev.metaschema.core.model.validation.IValidationFinding; +import dev.metaschema.core.model.validation.IValidationResult; +import dev.metaschema.databind.IBindingContext; + +import java.net.URI; +import java.nio.file.Path; + +IBindingContext context = IBindingContext.newInstance(); +URI target = Path.of("data.json").toUri(); + +IValidationResult result = context.validateWithConstraints(target, null); + +// Check overall status +if (result.isPassing()) { + System.out.println("Validation passed"); +} + +// Process findings by severity +for (IValidationFinding finding : result.getFindings()) { + Level severity = finding.getSeverity(); + if (severity == Level.CRITICAL) { + handleCritical(finding); + } else if (severity == Level.ERROR) { + handleError(finding); + } else if (severity == Level.WARNING) { + handleWarning(finding); + } else { + logInfo(finding); + } +} +``` + +## Using FindingCollectingConstraintValidationHandler + +The framework provides `FindingCollectingConstraintValidationHandler` for collecting validation findings: + +```java +import dev.metaschema.core.model.constraint.FindingCollectingConstraintValidationHandler; +import dev.metaschema.core.model.constraint.IConstraint.Level; +import dev.metaschema.core.model.validation.IValidationResult; + +// The handler implements IValidationResult +FindingCollectingConstraintValidationHandler handler = + new FindingCollectingConstraintValidationHandler(); + +// After validation completes, check results +if (!handler.isPassing()) { + handler.getFindings().forEach(finding -> { + System.err.println(finding.getSeverity() + ": " + + finding.getMessage()); + }); +} + +// Check highest severity level +Level highestSeverity = handler.getHighestSeverity(); +if (highestSeverity.ordinal() >= Level.ERROR.ordinal()) { + System.err.println("Validation failed with errors"); +} +``` + +## Constraint Levels + +Control validation behavior with levels: + +```xml + + + + + + + + + + +``` + +## External Constraints + +Define constraints in separate files: + +```xml + + + + + + + + + /catalog//control + + + + + +``` + +## Common Constraint Patterns + +### Required Fields + +```xml + + + +``` + +### Conditional Requirements + +```xml + + + +``` + +### Value Ranges + +```xml + + + +``` + +### Cross-field Validation + +```xml + + + +``` + +### Unique Within Parent + +```xml + + + + + +``` + +## Severity Reference + +| Level | Meaning | Effect | +|-------|---------|--------| +| `CRITICAL` | Severe error | Document unusable | +| `ERROR` | Constraint violation | Validation fails | +| `WARNING` | Potential issue | Logged, doesn't fail | +| `INFORMATIONAL` | Note | Logged only | + +## Best Practices + +1. **Use appropriate levels** - Not every issue is an ERROR +2. **Provide clear messages** - Include context and fix hints +3. **Validate early** - Check on load, not later +4. **Handle all severities** - Don't ignore warnings +5. **Test constraints** - Validate with known-bad data + +## Constraint Validation Status + +> **Note:** Metaschema constraint validation is experimental in some areas. Schema-level validation (structure, types) is stable. Advanced constraint types may have limitations. + +## Next Steps + +Continue learning about the Metaschema Java Tools with these related guides: + +- [Executing Metapath](executing-metapath.html) - Write constraint expressions +- [Reading & Writing Data](reading-writing-data.html) - Load data for validation +- [Generating Schemas](generating-schemas.html) - Generate schema validators diff --git a/src/site/markdown/index.md.vm b/src/site/markdown/index.md.vm index 6f304a664..502e0d728 100644 --- a/src/site/markdown/index.md.vm +++ b/src/site/markdown/index.md.vm @@ -1,20 +1,140 @@ # Metaschema Java Tools -This project provides a Java implementation of the [Metaschema](https://framework.metaschema.dev/) framework that supports format-agnostic information modeling and processing. A set of tools and libraries are provided that allow developers to quickly develop applications using Metaschema-based models. +This project provides a Java implementation of the [Metaschema](https://framework.metaschema.dev/) framework, a modeling language designed to enable format-agnostic information modeling and processing. -This project supports the following features: +## What is Metaschema? -- **Allows developers to generate Java classes from Metaschema definitions.** Using a given Metaschema module, a developer can quickly [generate Java classes](metaschema-databind/) for a Metaschema-based model, which can be used to create, parse, modify, and write XML, JSON, and YAML representations of that model using the generated bound Java objects. This allows a developer to quickly start programming business logic instead of spending hours writing parsing code. This approach is similar to the binding frameworks provided by [Jakarta XML Binding (JAXB)](https://eclipse-ee4j.github.io/jaxb-ri/), [XMLBeans](https://xmlbeans.apache.org/), [Java API for JSON Binding (JSON-B)](https://javaee.github.io/jsonb-spec/), and other class-to-object binding approaches. The [OSCAL Java Library](https://github.com/metaschema-framework/liboscal-java/) is an example of applying this approach to generate Java programming APIs for a set of Metaschema-defined models. -- **Supports generating Metaschema-based Java code during Maven builds.** An [Apache Maven](https://maven.apache.org/) [Metaschema code generation plugin](metaschema-maven-plugin/) is provided that supports Java class generation based on a Metaschema module during Maven builds. -- **Enables validation of data aligned with a set of Metaschema modules using constraints defined within the Metaschema definitions.** Using this framework, format-agnostic validation rules, defined in or external to a Metaschema module, can be enforced over data loaded into bound objects. -- **Allows execution of Metapath queries against data aligned with a Metaschema-based model.** Metapath is an XPath-like expression language that can be used to query XML, JSON, or YAML data aligned with a Metaschema module. This allows data to be queried irrespective of the format it is stored in. +Metaschema addresses a fundamental challenge in data exchange: organizations often need to work with data in multiple formats (XML, JSON, YAML) while maintaining consistent validation and semantics across all formats. Rather than defining separate schemas for each format, Metaschema allows you to define your data model once and automatically generate: -This project contains the following core sub-modules: +- **Format-specific schemas** (XML Schema, JSON Schema) for validation +- **Type-safe data bindings** (Java classes) for programmatic access +- **Documentation** that stays synchronized with the model -- [Metaschema Java API](metaschema-core/): Provides a [Java API](metaschema-core/apidocs/index.html) for interacting with Metaschema modules and executing Metapath expressions in Java programs. Supports loading XML-based Metaschema modules. -- [Metaschema Java Binding](metaschema-databind/): Supports the generation and use of annotated plain old Java objects (POJOs) to create and store Metaschema module-based data. Supports code generation of POJO classes based on a Metaschema module and reading and writing Metaschema module-based data. Can read and write XML, JSON, and YAML content that is valid to the associated Metaschema model. -- [Metaschema Module Bindings](metaschema-databind-modules/): Provides a variety of different Metaschema modules for different types of data, including support for the Static Analysis Results Interchange Format (SARIF). -- [Metaschema Maven Plugin](metaschema-maven-plugin/): A Maven build plugin that automates generation on Java classes and XML and JSON schemas based on a Metaschema module as part of a Maven build. -- [Metaschema Schema Generator](metaschema-schema-generator): An API for generating XML and JSON schemas based on a Metaschema module. +The Metaschema framework originated from [NIST](https://www.nist.gov/) as the foundation for the [OSCAL](https://pages.nist.gov/OSCAL/) (Open Security Controls Assessment Language) project, but its approach to information modeling is applicable to any domain requiring multi-format data exchange. -Please refer to each sub-module for usage instructions. +## Who Should Use This Library? + +This library is designed for Java developers who need to: + +- **Process OSCAL content** - If you're working with OSCAL documents, this library (via [liboscal-java](https://github.com/metaschema-framework/liboscal-java)) provides the foundation for reading, writing, and validating OSCAL in any format. + +- **Define custom data models** - If your organization needs a format-agnostic data model with consistent validation across XML, JSON, and YAML, Metaschema provides a robust framework for defining and implementing that model. + +- **Generate code from models** - The Maven plugin generates type-safe Java classes from Metaschema definitions, eliminating boilerplate serialization code and ensuring your Java objects always match your data model. + +- **Query and validate data** - Metapath, the XPath-like query language included in this library, provides powerful capabilities for querying and validating data regardless of its original format. + +## Quick Start + +Add the dependency to your Maven project: + +```xml + + ${project.groupId} + metaschema-databind + ${project.version} + +``` + +Once you have the dependency, you can read and write data: + +```java +import dev.metaschema.databind.IBindingContext; +import dev.metaschema.databind.io.Format; +import dev.metaschema.databind.io.IDeserializer; +import dev.metaschema.databind.io.ISerializer; +import java.nio.file.Path; + +// Get the binding context for your generated model class +IBindingContext context = IBindingContext.instance(); + +// Read data from JSON (substitute MyModel with your generated class) +IDeserializer deserializer = context.newDeserializer(Format.JSON, MyModel.class); +MyModel data = deserializer.deserialize(Path.of("input.json")); + +// Write data to XML +ISerializer serializer = context.newSerializer(Format.XML, MyModel.class); +serializer.serialize(data, Path.of("output.xml")); +``` + +To generate Java classes from your own Metaschema module, configure the Maven plugin: + +```xml + + ${project.groupId} + metaschema-maven-plugin + ${project.version} + + + + generate-sources + + + + +``` + +See the [Installation](installation.html) and [Generating Java Classes](guides/generating-java-classes.html) guides for complete setup instructions. + +## Key Features + +### Java Class Generation + +Generate type-safe Java classes directly from Metaschema module definitions. The generated classes handle serialization and deserialization for XML, JSON, and YAML formats automatically. This means you write your business logic against Java objects while the library handles all format-specific details. + +### Maven Integration + +The `metaschema-maven-plugin` integrates code generation into your Maven build lifecycle. Define your Metaschema modules, configure the plugin, and generated source code appears in `target/generated-sources/` ready for compilation. Changes to your Metaschema automatically regenerate the Java classes on the next build. + +### Constraint Validation + +Metaschema supports rich constraint definitions that validate data beyond basic schema requirements. Constraints can express business rules, cross-field dependencies, and complex conditions using Metapath expressions. These constraints are defined once in the Metaschema and enforced consistently regardless of whether your data is in XML, JSON, or YAML. + +### Metapath Query Language + +Metapath is an XPath-like expression language designed for querying Metaschema-based data. Unlike XPath which is specific to XML, Metapath works identically across all supported formats. Use Metapath to extract data, define constraints, and navigate complex document structures. + +## Getting Started + +| Task | Guide | +|:-----|:------| +| Add to your project | [Installation](installation.html) | +| Build from source | [Building](building.html) | +| Generate Java classes | [Generating Java Classes](guides/generating-java-classes.html) | +| Load Metaschema modules | [Loading Modules](guides/loading-metaschema-modules.html) | +| Read and write data | [Reading & Writing Data](guides/reading-writing-data.html) | +| Query with Metapath | [Executing Metapath](guides/executing-metapath.html) | +| Validate constraints | [Validating with Constraints](guides/validating-with-constraints.html) | +| Generate schemas | [Generating Schemas](guides/generating-schemas.html) | + +## Core Modules + +| Module | Description | +|--------|-------------| +| [metaschema-core](metaschema-core/) | Core APIs for Metaschema modules and Metapath expressions | +| [metaschema-databind](metaschema-databind/) | Data binding for XML, JSON, and YAML serialization | +| [metaschema-databind-modules](metaschema-databind-modules/) | Pre-built Metaschema modules (including SARIF) | +| [metaschema-maven-plugin](metaschema-maven-plugin/) | Maven plugin for code and schema generation | +| [metaschema-schema-generator](metaschema-schema-generator/) | XML and JSON schema generation | + +## Related Projects + +This framework is part of a larger ecosystem of Metaschema and OSCAL tools: + +| Project | Description | +|:--------|:------------| +| [liboscal-java](https://github.com/metaschema-framework/liboscal-java) | OSCAL Java library built on this framework | +| [oscal-cli](https://github.com/metaschema-framework/oscal-cli) | Command-line tool for OSCAL operations | +| [Metaschema Specification](https://framework.metaschema.dev/) | The Metaschema modeling language specification | + +## Using with Claude Code + +This project includes plugins for [Claude Code](https://claude.ai/code) that provide AI-assisted development. See the [Claude Integration](claude-integration.html) guide for details. + +## Support + +Have questions or found an issue? Here's how to get help: + +- [GitHub Issues](${project.issueManagement.url}) - Report bugs or request features +- [GitHub Discussions](${project.scm.url}/discussions) - Ask questions and discuss ideas +- [Contributing](${project.scm.url}/blob/develop/CONTRIBUTING.md) - Contribution guidelines diff --git a/src/site/markdown/installation.md.vm b/src/site/markdown/installation.md.vm new file mode 100644 index 000000000..e00005fe3 --- /dev/null +++ b/src/site/markdown/installation.md.vm @@ -0,0 +1,165 @@ +# Installation + +This guide explains how to add the Metaschema Java libraries to your project. + +## Choosing the Right Module + +The Metaschema Java project consists of several modules that serve different purposes. Most projects will only need one or two: + +| If you want to... | Use this module | +|:------------------|:----------------| +| Read/write data in XML, JSON, or YAML | `metaschema-databind` | +| Generate Java classes from Metaschema modules | `metaschema-maven-plugin` | +| Work with Metaschema modules directly | `metaschema-core` | +| Generate XML or JSON schemas | `metaschema-schema-generator` | +| Work with OSCAL documents | Consider [liboscal-java](https://github.com/metaschema-framework/liboscal-java) instead | + +For most use cases, `metaschema-databind` provides everything you need. It includes the core Metaschema functionality plus the data binding layer for serialization. If you're specifically working with OSCAL content, [liboscal-java](https://github.com/metaschema-framework/liboscal-java) bundles this library with pre-built OSCAL model classes. + +## Prerequisites + +Before adding the library to your project, ensure your development environment meets these requirements: + +| Requirement | Minimum Version | Recommended | +|:------------|:----------------|:------------| +| Java JDK | 11 | 17 or later | +| Maven | 3.9.0 | Latest | +| Gradle | 7.0 | Latest | + +The library targets Java 11 for broad compatibility, but building from source requires Java 17. See [Building from Source](building.html) if you need to compile the library yourself. + +## Maven + +### Release Versions + +Stable releases are published to Maven Central and require no additional repository configuration. Add the following dependency to your `pom.xml`: + +```xml + + ${project.groupId} + metaschema-databind + LATEST_VERSION + +``` + +Replace `LATEST_VERSION` with the current release from [Maven Central](https://central.sonatype.com/artifact/dev.metaschema.java/metaschema-databind). + +### Snapshot Versions + +Snapshot versions contain the latest changes from the `develop` branch, including new features and bug fixes that haven't yet been released. These are useful for testing upcoming changes or accessing features before they're officially released, but they may be less stable than release versions. + +To use snapshots, add the project's snapshot repository: + +```xml + + + metaschema-snapshots + https://raw.githubusercontent.com/metaschema-framework/maven2/refs/heads/main + + true + + + + + + + ${project.groupId} + metaschema-databind + ${project.version} + + +``` + +## Gradle + +Gradle users can add the library using either Kotlin DSL or Groovy DSL syntax. The examples below show `metaschema-databind`, but you can substitute any module from the [choosing the right module](#choosing-the-right-module) section. + +### Kotlin DSL + +Add to your `build.gradle.kts`: + +```kotlin +dependencies { + implementation("dev.metaschema.java:metaschema-databind:LATEST_VERSION") +} +``` + +For snapshot versions, add the repository: + +```kotlin +repositories { + mavenCentral() + maven { + url = uri("https://raw.githubusercontent.com/metaschema-framework/maven2/refs/heads/main") + } +} +``` + +### Groovy DSL + +Add to your `build.gradle`: + +```groovy +dependencies { + implementation 'dev.metaschema.java:metaschema-databind:LATEST_VERSION' +} +``` + +For snapshot versions: + +```groovy +repositories { + mavenCentral() + maven { + url 'https://raw.githubusercontent.com/metaschema-framework/maven2/refs/heads/main' + } +} +``` + +## Building from Source + +If you need to build the library from source—for example, to contribute changes, debug issues, or access unreleased features—see the [Building](building.html) guide. Building from source requires Java 17 or later. + +## IDE Setup + +Modern Java IDEs automatically recognize Maven and Gradle projects, downloading dependencies and configuring the classpath. After adding the dependency to your build file: + +1. **IntelliJ IDEA**: Open the project folder or `pom.xml`/`build.gradle` file. IntelliJ will import the project and download dependencies automatically. + +2. **Eclipse**: Import as a Maven or Gradle project using the built-in import wizard. The [M2Eclipse](https://www.eclipse.org/m2e/) plugin handles Maven projects. + +3. **VS Code**: Install the [Extension Pack for Java](https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-pack), then open the project folder. VS Code will detect the build system and configure accordingly. + +For more detailed IDE setup, refer to your IDE's documentation: + +- [IntelliJ IDEA - Maven Projects](https://www.jetbrains.com/help/idea/maven-support.html) +- [Eclipse - Maven Integration](https://www.eclipse.org/m2e/) +- [VS Code - Java Extension Pack](https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-pack) + +## Verifying Installation + +After adding the dependency, verify the installation by creating a simple test class: + +```java +import dev.metaschema.core.metapath.MetapathExpression; + +public class VerifyInstallation { + public static void main(String[] args) { + MetapathExpression expr = MetapathExpression.compile("1 + 1"); + System.out.println("Metaschema installed successfully!"); + System.out.println("Metapath result: " + expr.evaluateAs(null, + MetapathExpression.ResultType.NUMBER)); + } +} +``` + +If this compiles and runs without errors, the library is correctly installed. + +## Next Steps + +Once installed, explore these guides to start using the library: + +- [Architecture](guides/architecture.html) - Understand the module structure and how components interact +- [Loading Modules](guides/loading-metaschema-modules.html) - Load and work with Metaschema definitions +- [Generating Java Classes](guides/generating-java-classes.html) - Use the Maven plugin to generate code +- [Reading & Writing Data](guides/reading-writing-data.html) - Serialize and deserialize data in multiple formats diff --git a/src/site/resources/css/custom.css b/src/site/resources/css/custom.css index 2c3aa8f86..f03f5258b 100644 --- a/src/site/resources/css/custom.css +++ b/src/site/resources/css/custom.css @@ -19,4 +19,11 @@ footer { code { color: #5f879b; +} + +/* Fix table header alignment - flexmark doesn't apply alignment to cells */ +/* Use higher specificity to override Bootstrap's .table th styles */ +.table th, +table th { + text-align: left !important; } \ No newline at end of file diff --git a/src/site/site.xml b/src/site/site.xml index 53dde4325..9db1441a6 100644 --- a/src/site/site.xml +++ b/src/site/site.xml @@ -110,6 +110,20 @@ + + + + + + + + + + + + + +