Add CBOR Support to libyang (#2130)

[MeherRushi]

Add CBOR Support to libyang

Attempt to resolve #2130

Overview

This PR adds comprehensive CBOR support to libyang, enabling parsing, validating, and printing instance data in CBOR format as specified in RFC 9254. The implementation uses libcbor as the underlying CBOR library and follows libyang's existing patterns and conventions.

Key Features

• Optional Dependency: CBOR support is controlled via the compile-time flag ENABLE_CBOR_SUPPORT, allowing builds with or without CBOR capabilities
• Full Parsing Support: Complete CBOR parser that constructs libyang data trees and links them with schema trees for validation
• Full Printing Support: CBOR printer with parallel structure to printer_json.c, supporting all YANG node types (leaf, leaf-list, list, container, anydata, anyxml, RPC, action, notification)
• Consistent API: Mirrors existing JSON functionality for ease of use and maintenance

Implementation Details

New Files

• parser_cbor.c / parser_cbor.h - High-level CBOR parser
• printer_cbor.c / printer_cbor.h - CBOR printer implementation
• lcbor.c / lcbor.h - Consistent wrapper over libcbor following libyang coding style

Changes

• Added ENABLE_CBOR_SUPPORT CMake flag to conditionally enable CBOR functionality
• Integrated libcbor dependency with CMake find_package and conditional compilation
• Introduced LY_VALUE_CBOR format with corresponding case statements throughout the codebase
• Added lyd_parse_data_mem_len() high-level function for parsing CBOR from memory (required because CBOR binary data may contain null bytes, making strlen() unreliable)
• Implemented consistent function structure mirroring JSON counterparts for parsing (name parsing, prefix handling, schema node lookup, metadata, opaque nodes, etc.)
• Added proper error handling, null handling, array processing, and metadata support

TODO / Known Limitations

• Refactor input handling - Move lyd_parse_data_mem_len to input handler (in.c) for better architecture
• Clean up debug statements - Remove debug print statements from production code
• Add more comments - Improve code documentation
• CBOR SID support - Currently only named identifiers are supported; add SID (Schema Item iDentifier) support
• Finalize input API - Determine optimal API design for CBOR input handling
• Handle missing switch cases - Complete remaining switch statement cases
• RPC/anydata edge cases - Address remaining edge cases for RPC and anydata handling

Dependencies

• libcbor - Required when ENABLE_CBOR_SUPPORT=ON

Build Instructions

cmake -DENABLE_CBOR_SUPPORT=ON ..
make

---

Testing :

Considered an example.json file :

{
  "ietf-interfaces:interfaces": {
    "interface": [
      {
        "name": "eth0",
        "description": "Loopback",
        "type": "iana-if-type:ethernetCsmacd",
        "enabled": true,
        "oper-status": "up",
        "statistics": {
          "discontinuity-time": "2024-01-01T00:00:00Z"
        }
      }
    ]
  }
}

Converted the same into cbor notation and output in example.cbor using standard cbor libraries in python (below code - json_to_cbor.py):

import json
import cbor2
import time

# Read the JSON file
with open("example.json", "r", encoding="utf-8") as f:
    json_data = json.load(f)

# Write CBOR-encoded output
with open("example.cbor", "wb") as f:
    cbor2.dump(json_data, f)

print("✅ example.cbor written from example.json")

time.sleep(5)

with open("example.cbor", "rb") as f:
    obj = cbor2.load(f)

print(json.dumps(obj, indent=2))

Now ran the following json_test.c on the json file with the necessary yang modules iana-if-type.yang and ietf-interfaces.yang in the directory to see functionality on the json data

#include <stdlib.h>
#include <libyang/libyang.h>

int main(void) {
    struct ly_ctx *ctx = NULL;
    struct lyd_node *data = NULL;

    // Context: search current dir only, no auto-loading
    if (ly_ctx_new(".", LY_CTX_NO_YANGLIBRARY, &ctx) != LY_SUCCESS) {
        fprintf(stderr, "Context creation failed.\n");
        return 1;
    }

    // Load only required official modules
    if (lys_parse_path(ctx, "iana-if-type.yang", LYS_IN_YANG, NULL) != LY_SUCCESS ||
        lys_parse_path(ctx, "ietf-interfaces.yang", LYS_IN_YANG, NULL) != LY_SUCCESS) {
        fprintf(stderr, "Module loading failed.\n");
        ly_ctx_destroy(ctx);
        return 1;
    }

    // Parse data
    if (lyd_parse_data_path(ctx, "example.json", LYD_JSON, LYD_PARSE_STRICT, 0, &data) != LY_SUCCESS) {
        fprintf(stderr, "JSON parsing failed.\n");
        ly_ctx_destroy(ctx);
        return 1;
    }

    // Print parsed data
    lyd_print_file(stdout, data, LYD_JSON, 0);

    // Cleanup
    lyd_free_all(data);
    ly_ctx_destroy(ctx);
    return 0;
}

Repeated the same with the cbor_test.c file (in the same directory) utilizing the cbor features introduced to test the newly built library

#include <stdlib.h>
#include <libyang/libyang.h>

int main(void) {
    struct ly_ctx *ctx = NULL;
    struct lyd_node *data = NULL;

    // Context: search current dir only, no auto-loading
    if (ly_ctx_new(".", LY_CTX_NO_YANGLIBRARY, &ctx) != LY_SUCCESS) {
        fprintf(stderr, "Context creation failed.\n");
        return 1;
    }

    // Load only required official modules
    if (lys_parse_path(ctx, "iana-if-type.yang", LYS_IN_YANG, NULL) != LY_SUCCESS ||
        lys_parse_path(ctx, "ietf-interfaces.yang", LYS_IN_YANG, NULL) != LY_SUCCESS) {
        fprintf(stderr, "Module loading failed.\n");
        ly_ctx_destroy(ctx);
        return 1;
    }

    // Parse data
    if (lyd_parse_data_path(ctx, "example.cbor", LYD_CBOR, LYD_PARSE_STRICT, 0, &data) != LY_SUCCESS) {
        fprintf(stderr, "CBOR parsing failed.\n");
        ly_ctx_destroy(ctx);
        return 1;
    }

    // Print parsed to file named "output.cbor"
    // output.cbor should be a file pointer opened in binary write mode
    FILE *output_file = fopen("output.cbor", "wb");
    if (!output_file) {
        fprintf(stderr, "Failed to open output.cbor for writing.\n");
        ly_ctx_destroy(ctx);
        return 1;
    }
    lyd_print_file(output_file, data, LYD_CBOR, 0);
    fclose(output_file);
    // lyd_print_file(stdout, data, LYD_JSON, 0);


    // Cleanup
    lyd_free_all(data);
    ly_ctx_destroy(ctx);
    return 0;
}

Following output:

$ gcc cbor_test.c -lyang -o cbor
$./cbor 
DEBUG: Parsed CBOR data:
{"ietf-interfaces:interfaces": {"interface": [{"name": "eth0", "description": "Loopback", "type": "iana-if-type:ethernetCsmacd", "enabled": true, "oper-status": "up", "statistics": {"discontinuity-time": "2024-01-01T00:00:00Z"}}]}}

And we can change the input file from example.cbor to output.cbor to see the parsed data being parsed properly

$ gcc cbor_test.c -lyang -o cbor
(base) meherrushi@MeherRushi:~/Meher/learning/c/cbor/test$ ./cbor 
DEBUG: Parsed CBOR data:
{"ietf-interfaces:interfaces": {"interface": [{"name": "eth0", "description": "Loopback", "type": "iana-if-type:ethernetCsmacd", "enabled": true, "oper-status": "up", "statistics": {"discontinuity-time": "2024-01-01T00:00:00+00:00"}}]}}

note: shall upload the test files on another repo

[michalvasko]

Thanks a lot for this effort and I will try to provide some feedback today or tomorrow. But I can immediately tell you to change the libyang base branch to devel, PRs are never merged into master directly.

[michalvasko]

Generally looks fine but some improvements are possible and the coding style is somewhat different to ours.

[michalvasko]

Actually, an option is probably redundant. You can just search for libcbor and if found, the support will be added, otherwise not.

[michalvasko]

Is pkg-config really required? We have used it only when necessary, the else() branch should be fine except may be better to add a new FindCBOR.cmake module file.

[michalvasko]

Seems redundant if the define is around the whole file, the file should just not be compiled at all if disabled. In all the new files.

[michalvasko]

Just cbor.c would probably fit better, like lyb.c and lyb.h.

[michalvasko]

Redundant if and maybe the whole function.

[michalvasko]

Not really needed, I think, if you just declare struct lycbor_ctx;.

[michalvasko]

Again not necessary, the function can exist and fail. I would like to keep these macros to a minimum.

[michalvasko]

Leftovers.

[michalvasko]

This changes how JSON opaque nodes are stored, what is the reason?

[michalvasko]

LYD_ANYDATA_LYB was removed in the devel branch and I think for the same reason LYD_ANYDATA_CBOR is not required. Only objects are allowed in anydata, so it should always be possible to parse them into LYD_ANYDATA_DATATREE.