The Interface Definition Language (IDL) is used to define interfaces, structures, and data types that Canopy uses to generate type-safe proxy and stub code.
Canopy IDL uses a C++-like syntax with additional attributes for RPC-specific metadata.
// Comments use C++ style
#include "other_file.idl" // Copy/paste content (caution - causes duplicate symbols)
#import "other_file.idl" // Reference external IDL types (preferred)
namespace my_namespace
{
[attribute=value, ...]
interface my_interface
{
[attribute=value]
return_type method_name(param_type param_name, [out] param_type& out_param);
};
[attribute=value]
struct my_struct
{
type member_name;
type member_name = default_value;
};
}IDL files should be organized logically:
idl/
├── my_project/
│ ├── main.idl // Main file with interfaces
│ ├── types.idl // Shared struct definitions
│ └── import.idl // External imports
Namespaces organize interfaces and structs into logical groups.
namespace calculator
{
interface i_calculator
{
error_code add(int a, int b, [out] int& result);
};
}namespace outer
{
namespace inner
{
interface i_foo
{
error_code method();
};
}
}Inline namespaces allow seamless API evolution by treating nested namespaces as a single namespace. This is essential for version management.
Key Benefits:
- C++ access without version prefix:
calculator::i_calculatorinstead ofcalculator::v1::i_calculator - Seamless version upgrades: When adding
v2, existing code continues to work - Explicit version access available: Can still use
calculator::v1::i_calculatororcalculator::v2::i_calculator
namespace calculator
{
[inline] namespace v1
{
interface i_calculator
{
error_code add(int a, int b, [out] int& result);
};
}
}C++ Usage:
// Direct access (inline namespace means v1 is transparent)
calculator::i_calculator calc;
// Explicit version access (still works)
calculator::v1::i_calculator calc_v1;Version Migration Example:
// Original version
namespace comprehensive
{
[inline] namespace v1
{
interface i_foo { ... };
}
}
// Adding v2 with new features
namespace comprehensive
{
[inline] namespace v2
{
interface i_foo : v1::i_foo
{
error_code new_feature();
};
}
}[status=production] // development, production, deprecated
namespace stable
{
interface i_foo { };
};Structs define complex data types with member variables.
namespace xxx
{
struct person
{
std::string name;
int age;
std::vector<std::string> hobbies;
};
}struct config
{
std::string name = "default_name";
int timeout = 30;
bool enabled = true;
};struct constants
{
static std::string prefix = "RPC_";
static int max_value = 1000;
};struct address
{
std::string street;
std::string city;
int zip_code;
};
struct employee
{
std::string name;
address home_address;
address work_address;
};struct collection
{
std::vector<int> numbers;
std::vector<std::string> strings;
std::map<std::string, int> name_to_id;
std::map<int, std::string> id_to_name;
};template<typename T>
struct box
{
T value;
};
template<typename T>
struct result
{
T data;
bool success;
std::string error_message;
};Interfaces define RPC service contracts with method signatures.
namespace yyy
{
interface i_example
{
error_code add(int a, int b, [out] int& c);
error_code multiply(int a, int b, [out] int& c);
};
}[status=production, description="Calculator service for mathematical operations"]
interface i_calculator
{
[description="Adds two integers"] error_code add(int a, int b, [out] int& result);
[description="Subtracts two integers"] error_code subtract(int a, int b, [out] int& result);
[description="Multiplies two integers"] error_code multiply(int a, int b, [out] int& result);
};namespace yyy
{
interface i_host
{
error_code look_up_app(const std::string& name, [out] rpc::shared_ptr<i_example>& app);
error_code set_app(const std::string& name, [in] const rpc::shared_ptr<i_example>& app);
};
interface i_example
{
error_code create_foo([out] rpc::shared_ptr<xxx::i_foo>& target);
error_code receive_interface([out] rpc::shared_ptr<xxx::i_foo>& val);
error_code give_interface([in] const rpc::shared_ptr<xxx::i_baz> val);
};
}namespace xxx
{
interface i_foo
{
error_code do_something(int val);
};
interface i_bar
{
error_code do_something_else(int val);
};
// Multiple inheritance
interface i_baz : i_foo, i_bar
{
error_code callback(int val);
};
}The [tag=...] attribute passes metadata to the service proxy or transport for special processing:
// Define tag values (typically an enum)
namespace comprehensive::v1
{
enum class tags
{
none = 0,
include_certificate,
require_auth,
high_priority
};
}
interface i_demo_service
{
// Normal call - no special processing
error_code get_name([out] std::string& name);
// Tagged call - transport/proxy can handle specially
[tag=comprehensive::v1::tags::include_certificate]
error_code create_object([out] rpc::shared_ptr<i_managed_object>& obj);
[tag=comprehensive::v1::tags::require_auth]
error_code delete_object(uint64_t object_id);
}How Tags Work:
- The
[tag=...]attribute is stored in the generated interface metadata - When the function is called via the service proxy, the tag is passed to
i_marshaller::send()andi_marshaller::post() - The transport or marshaller can inspect the tag and apply special processing
Common Use Cases:
- Authentication: Tag sensitive operations requiring extra verification
- Priority: High-priority vs normal calls in network transport
- Encryption: Request certificate inclusion for certain operations
- Logging: Tag operations for audit trails
- Routing: Direct specific calls through different paths
Accessing Tags in Generated Code:
// Tags are accessible via function_info
auto& info = i_demo_service::function_info<create_object>();
auto tag = info.tag; // comprehensive::v1::tags::include_certificateThe [post] attribute creates fire-and-forget methods that send data without waiting for a response. These one-way methods return immediately after queuing the message, making them ideal for notifications, logging, and event streaming where response synchronization is not needed.
Key Characteristics:
- No Response Wait: Method returns immediately after sending
- Ordering Guarantee: Messages are received in the order they were sent
- One-Way Communication: No return values or output parameters allowed
- Performance: Lower latency for operations that don't need confirmation
namespace websocket_demo::v1
{
interface i_context_event
{
// Fire-and-forget notification - returns immediately
[post] error_code piece(const std::string& data);
};
interface i_logger
{
// Fire-and-forget log message
[post] error_code log_event(const std::string& message, int severity);
// Fire-and-forget metrics update
[post] error_code record_metric(const std::string& metric_name, double value);
};
}Implementation Example:
class context_event : public rpc::base<context_event, i_context_event>
{
std::vector<std::string> received_pieces_;
std::mutex mutex_;
public:
// [post] methods still return error_code but caller doesn't wait for it
CORO_TASK(error_code) piece(const std::string& data) override
{
std::lock_guard<std::mutex> lock(mutex_);
received_pieces_.push_back(data);
RPC_INFO("Received piece: {}", data);
CO_RETURN rpc::error::OK();
}
};Usage Example:
// Caller sends multiple messages without waiting
for (int i = 0; i < 100; ++i)
{
// Returns immediately - doesn't wait for remote processing
CO_AWAIT event->piece("Message " + std::to_string(i));
}
// All messages guaranteed to arrive in order sentRestrictions:
- Cannot have
[out]or[in, out]parameters - all parameters must be[in]only - Cannot pass interface parameters (
rpc::shared_ptrorrpc::optimistic_ptr) - posting interfaces is not currently supported - Return type should be
error_code - No response data can be returned to caller
- Errors may not be immediately visible to caller
Invalid Examples:
interface i_invalid_post
{
// ERROR: [post] cannot have [out] parameters
[post] error_code send_data(const std::string& data, [out] int& status);
// ERROR: [post] cannot pass interface parameters
[post] error_code notify_observer(const rpc::shared_ptr<i_observer>& observer);
// ERROR: [post] cannot have [in, out] parameters
[post] error_code modify([in, out] std::string& value);
}Valid Examples:
interface i_valid_post
{
// OK: Only [in] parameters (primitive types and structs)
[post] error_code log_message(const std::string& message, int severity);
// OK: Multiple [in] parameters
[post] error_code record_metric(const std::string& name, double value, uint64_t timestamp);
// OK: Complex struct as [in] parameter
[post] error_code send_event(const event_data& event);
}Use Cases:
- Event Streaming: Send continuous stream of events or sensor data
- Logging: Send log messages without blocking the main thread
- Notifications: Push notifications where acknowledgment isn't critical
- Metrics Collection: Send performance metrics asynchronously
- UI Updates: Send incremental UI update commands
Performance Considerations:
- Reduces latency by eliminating wait for response
- May increase throughput for notification-heavy workloads
- Still subject to transport buffering and flow control
- Message ordering guaranteed but delivery timing is asynchronous
Error Handling: Since the caller doesn't wait for a response, error handling is limited:
- Transport-level errors (connection lost) may be reported
- Application-level errors in the implementation are not visible to caller
- Consider complementing with status query methods if error visibility is needed
interface i_async_logger
{
// Fire-and-forget log write
[post] error_code write_log(const std::string& message);
// Query method to check logger health (normal bidirectional call)
error_code get_status([out] std::string& status);
}Parameters in Canopy have direction attributes that control data marshalling across the transport.
If no attribute is specified, the parameter is assumed to be [in]:
// All three are equivalent - data sent TO the object
error_code process_value(int value);
error_code process_value([in] int value);
error_code process_value([in] const int& value);Data is marshalled FROM the caller TO the remote object:
// Pass by value (copy)
error_code process_value(int value);
// Pass by const reference
error_code process_ref([in] const int& value);
// Pass by move
error_code process_move([in] int&& value);If the IDL declares an rvalue reference, the generated interface and proxy method signatures keep that rvalue reference:
error_code process_move([in] payload&& value);Generates an interface-level signature of the same shape:
virtual CORO_TASK(error_code) process_move(payload&& value) = 0;This is intentional. The shared interface must reflect the IDL contract exactly so callers can see that the API is expressed as an rvalue-reference-taking operation.
Internally, the generated marshalling helpers may use non-consuming parameter forms when they serialise request data. That is an implementation detail of the generated transport layer, not a change to the public IDL contract. The reason is that proxy-side marshalling may need to inspect or serialise the same logical input more than once during encoding fallback or retry handling, while the stub still reconstructs a temporary and passes it to the implementation using the IDL-declared shape.
In practice:
- The generated shared interface keeps
T&&if the IDL saysT&&. - Calling a proxy with
std::move(x)does not imply the RPC transport takes ownership ofxin the same way as a local move-only call. - The stub side is where a deserialised temporary is created and supplied to the target function.
If you need different move semantics, express that choice explicitly in the IDL because the generated interface is treated as the source of truth.
Data is marshalled FROM the remote object BACK to the caller:
// Output by reference (caller provides storage)
error_code get_value([out] int& value);Note: This feature has limited testing, particularly with interface parameters. Use with caution and verify behavior in your specific use case.
Data is marshalled IN BOTH DIRECTIONS - first to the object, then back:
// Modify in place
error_code modify([in, out] int& value);Recommendation: Avoid using raw pointer types (T*, T*&) in IDL interfaces.
Reason: Pointers represent memory addresses in a specific address space. When marshalling data between different processes or machines, these addresses have no meaning in the remote address space.
// Not recommended - pointers only valid in shared memory scenarios
error_code get_optional([out] int*& value); // Address not valid remotely
error_code process_ptr([in] const int* value); // Address not valid remotelyUse instead: Value types, references, or smart pointers:
// Good - values are copied across address spaces
error_code get_value([out] int& value);
error_code get_values([out] std::vector<int>& values);
// Use rpc::shared_ptr for interface references
error_code get_service([out] rpc::shared_ptr<i_foo>& service);Exception: Pointer types may be useful only when both objects share the same memory address space (e.g., shared memory regions between processes).
Use Cases (rare): Marshalling pointers only use within zones that share the same address space, not recommended for handles
[in]parameters: For pointer types (T*) where you need to serialize the pointer address[out]parameters: For double pointers (T**) or pointer references (T*&) to receive an address
Example:
// Pointer to single value - serializes the address
error_code process_value([in] const int* value);
// Pointer reference - serializes the address
error_code allocate_value([out] int*& value);Security Warning: Raw pointer values are memory addresses that should never be used for unrestricted environments (e.g., web clients, untrusted networks). Pointer serialization only makes sense when both caller and callee exist in the same address space or have carefully controlled shared memory access.
Important: rpc::shared_ptr and rpc::optimistic_ptr can only be [in] OR [out], never [in, out]:
// Valid - shared_ptr only in
error_code set_app([in] const rpc::shared_ptr<i_foo> app);
// Valid - shared_ptr only out
error_code get_app([out] rpc::shared_ptr<i_foo>& app);
// Invalid - shared_ptr cannot be [in, out]
// error_code transfer_app([in, out] rpc::shared_ptr<i_foo>& app); // ERROR!IDL interfaces define how data is marshalled between address spaces. Consider your transfer patterns:
// Get entire blob at once - good for bulk operations
error_code get_config([out] std::vector<char>& config_data);
// Navigate and extract - good for selective access
error_code get_value(const std::string& key, [out] std::string& value);
error_code get_count([out] int& count);Trade-offs:
- Single large transfer: One round-trip, may transfer unused data
- Selective access: Multiple round-trips, but transfers only needed data
Choose based on your use case and network characteristics.
namespace rpc
{
enum encoding : uint64_t
{
yas_binary = 1,
yas_compressed_binary = 2,
yas_json = 8,
protocol_buffers = 16
};
enum status : uint32_t
{
success = 0,
pending = 1,
failed = 2
};
}Attributes provide metadata for interfaces, methods, and structs.
[status=production] // development, production, deprecated
[description="Service description"]
interface i_foo
{
// ...
};[description="What the method does"]
[deprecated="Use new_method instead"]
error_code old_method(int input, [out] int& output);[status=production, description="Data structure"]
struct my_data
{
int value;
};Use #import to reference types from other IDL files:
// Main IDL file
#import "shared/types.idl"
#import "common/interfaces.idl"
namespace my_project
{
// Uses types and interfaces from imports
}Canopy IDL supports two ways to include external definitions:
| Directive | Behavior | Use Case |
|---|---|---|
#import |
Makes IDL aware of types without regenerating them | Preferred - Use for referencing external IDL definitions |
#include |
Copies and pastes IDL content (like C++ #include) | Useful for #defines etc - Can cause duplicate symbol errors |
Using #include can lead to duplicate symbol problems:
// types.idl
namespace shared
{
struct data { int value; };
}
// Avoid this - causes duplicate symbols when types.idl is included multiple times
#include "types.idl"
// Use this instead - safe reference without duplication
#import "types.idl"Best Practice: Always use #import to reference types from other IDL files. Only use #include if you specifically need to inline content and understand the duplication risks.
When you #import another IDL file:
- The generator recognizes the types defined in that file
- No code is regenerated for the imported types
- The imported types can be used in your interfaces and structs
- Marshalling logic for imported types is reused from their original definition
Define a return type for methods that indicates success or failure:
// Define error_code as a simple typedef (typically 0 = OK, non-zero = error)
typedef int error_code;
interface i_calculator
{
error_code add(int a, int b, [out] int& result);
// Common error code values (defined in your IDL or C++):
// 0 = OK
// Non-zero = error
};Note: error_code is not a built-in Canopy type. Define it in your IDL using typedef int error_code; or a similar pattern that works for your use case.
Canopy provides a flexible error code system designed for seamless integration with legacy applications:
// Configure Canopy error codes to avoid conflicts with your application
rpc::error::set_offset_val(10000); // Base offset
rpc::error::set_offset_val_is_negative(false); // Positive offset direction
rpc::error::set_OK_val(0); // Success value (typically 0)Key Points:
- RPC error codes are reserved for internal RPC operations (transport errors, object not found, etc.)
- Application error codes are defined by you in your IDL (
typedef int error_code;) - Canopy error codes can be offset to coexist with existing application error codes
- Use RPC error codes for inspection purposes only, not as your application error codes
Standard RPC Error Codes (may be offset):
| Code | Meaning |
|---|---|
| 0 | OK (success) |
| 1 | OUT_OF_MEMORY |
| 4 | INVALID_DATA |
| 5 | TRANSPORT_ERROR |
| 12 | OBJECT_NOT_FOUND |
| 23 | OBJECT_GONE |
Inject raw C++ code using #cpp_quote:
namespace rpc
{
// Template specialization that can't be expressed in IDL
#cpp_quote(R^__(
template<typename T>
class id
{
static constexpr uint64_t get(uint64_t rpc_version);
};
)__^)
}Note: The R^__(...)__^) syntax is equivalent to C++ raw string literals R"__(...) __". The caret notation (^) is used instead of quotes to enable proper syntax highlighting and colorization of embedded C++ code within IDL files.
// First define error_code (typically at top of IDL file)
typedef int error_code;
#import "example_shared/example_shared.idl"
namespace yyy
{
[status=production]
interface i_example
{
[description="Adds two integers and returns the result"]
error_code add(int a, int b, [out] int& c);
[description="Creates a new foo object instance"]
error_code create_foo([out] rpc::shared_ptr<xxx::i_foo>& target);
[description="Creates an example instance in a subordinate zone"]
error_code create_example_in_subordinate_zone(
[out] rpc::shared_ptr<yyy::i_example>& target,
const rpc::shared_ptr<i_host>& host_ptr);
[description="Receives an interface object (can be null)"]
error_code receive_interface([out] rpc::shared_ptr<xxx::i_foo>& val);
[description="Gets the current host instance"]
error_code get_host([out] rpc::shared_ptr<i_host>& app);
};
[status=production]
interface i_host
{
[description="Creates a new enclave instance"]
error_code create_enclave([out] rpc::shared_ptr<i_example>& target);
[description="Looks up an application by name from the registry"]
error_code look_up_app(const std::string& name, [out] rpc::shared_ptr<i_example>& app);
[description="Sets an application instance in the registry with the given name"]
error_code set_app(const std::string& name, [in] const rpc::shared_ptr<i_example>& app);
[description="Unloads an application by name from the registry"]
error_code unload_app(const std::string& name);
};
}When you compile IDL files, Canopy generates:
| File | Purpose |
|---|---|
{name}.h |
Interface declarations |
{name}_proxy.cpp |
Client-side proxy implementation |
{name}_stub.cpp |
Server-side stub implementation |
{name}_stub.h |
Stub declarations |
{name}.json |
JSON schema for introspection |
// Generated header
namespace yyy {
class i_example : public rpc::interface<i_example>
{
public:
virtual CORO_TASK(error_code) add(int a, int b, int& c) = 0;
virtual ~i_example() = default;
static std::vector<rpc::function_info> get_function_info();
};
} // namespace yyy// Generated proxy implementation
class i_example_proxy : public rpc::interface_proxy<i_example>
{
public:
virtual CORO_TASK(error_code) add(int a, int b, int& c) override
{
// Serialization and network send
// Deserialization of response
}
};- Use descriptive names:
i_calculator_servicenotcalc - Add descriptions: Always include
[description="..."]attributes - Version with namespaces: Use
[inline] namespace v1for API versioning - Keep interfaces focused: Single responsibility per interface
- Use output parameters: For large data, use
[out]to avoid copying - Document error codes: Explain what each method returns on error
- Organize in files: Group related interfaces in separate files
- Use #import for external types: Avoid
#includeto prevent duplicate symbols - Understand marshalling:
[in]sends to object,[out]receives back - No [in,out] with pointers:
rpc::shared_ptrandrpc::optimistic_ptrcannot be[in, out] - Avoid raw pointers: Use references, value types, or smart pointers instead
- Use [post] for one-way operations: Fire-and-forget methods with
[post]reduce latency for notifications and events where responses aren't needed
When using the Canopy IDL generator, be aware of the following behaviors and workarounds:
Note: Parameter attributes ([in], [out], [in, out]) are instructions to the IDL generator about marshalling direction. They are reflected in the generated proxy/stub code but do not appear in the C++ method signature as attributes.
interface i_example
{
int process([in] const std::string& input, [out] std::string& output);
};
// C++ implementation - attributes determine marshalling but are not C++ attributes
CORO_TASK(int) process(const std::string& input, std::string& output) override
{
output = "Processed: " + input;
CO_RETURN rpc::error::OK();
}Issue: Method names like get_id may conflict with interface ID getter methods.
Workaround: Use alternative names:
// Avoid
interface i_object
{
int get_id([out] uint64_t& id); // Conflicts with interface ID getter
};
// Use instead
interface i_object
{
int get_object_id([out] uint64_t& id); // Unique method name
};Issue: Some struct patterns may not serialize correctly due to generator limitations.
Workaround: Test struct serialization early and avoid complex nested patterns. If serialization fails, simplify the struct or use alternative data representation.
Note: Multiple methods in the same interface can use the same parameter names (e.g., result) without issue. The IDL generator should handle this correctly.
If you encounter conflicts with duplicate parameter names, this indicates a bug in the code generator that should be reported.
// This is valid IDL - parameter names can repeat across methods
interface i_calculator
{
int add(int a, int b, [out] int& result);
int multiply(int a, int b, [out] int& result); // 'result' is fine here
int divide(int a, int b, [out] int& result); // 'result' is fine here
};- Transports and Passthroughs - Learn about communication channels
- Getting Started - Follow a tutorial
- API Reference - Complete API documentation