-
Couldn't load subscription status.
- Fork 0
Add support for builtin types for get_value and set_value #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 36 commits
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
6643e6b
First draft of builtin type marshalling
jfriedri-ni 8d97456
Use a brute-force explicit helper for type-hinting the convertible na…
jfriedri-ni 02c1cdd
Use the StreamlitPanel as the main entrypoint
jfriedri-ni ec9e760
Add tests for unopened-set, unopened-get, and scalar round trips
jfriedri-ni 4407668
Remove unused import
jfriedri-ni 86c22aa
Add test showing an unset value raises when clients attempt to get it
jfriedri-ni b3e504a
Merge remote-tracking branch 'origin/main' into users/jfriedri-ni/add…
jfriedri-ni b63ea73
Add type hints that are compatible with Python 3.9
jfriedri-ni a60c17f
Do not add type hints when the analyzer can infer it
jfriedri-ni b3f1314
Parameterize the scalar value test
jfriedri-ni 4d73ca6
Add roundtrip tests for IntFlag, IntEnum, StrEnum
jfriedri-ni 518dd2f
Add tests for mixin-enums and bare-enums
jfriedri-ni 345e89a
Shorten typename imports
jfriedri-ni 3a02372
Rename fixture 'grpc_channel_for_fake_panel_service' to 'fake_panel_c…
jfriedri-ni ca9b8a3
Use grcpio's logging thread pool
jfriedri-ni 91ee96b
Remove static storage for the fake servicer
jfriedri-ni 63296e1
Remove class members from the fake service
jfriedri-ni 5969e92
Simplify class and storage for ConvertibleType
jfriedri-ni 3fb2b9b
Use pyproject.toml to suppress type checker
jfriedri-ni 62f808c
Clarify docstring
jfriedri-ni 9a5ae3c
Do not use 'as' for imports because it confuses tooling
jfriedri-ni eabcb0f
Make types compatible with Python 3.9 and 3.10
jfriedri-ni 6457e41
Remove redundant namespacing and front-load building the type convers…
jfriedri-ni fde87f6
Use the Python types rather than typenames for to_any
jfriedri-ni 905e315
Shortest path to custom initializers
jfriedri-ni b73b7b0
Use a Protocol to allow converters to have custom conversions
jfriedri-ni e07a2c0
Apply formatter
jfriedri-ni 3285505
Make protobuf_message return a Message so the default implementation …
jfriedri-ni e9fe18c
Simple naive type-fix
jfriedri-ni dc0c552
Refactor converters as an ABC family
jfriedri-ni 4f3a208
Converting builtins with a shared function, still mypy errors
jfriedri-ni 9115100
Do not share code between the builtin converters because type hinting…
jfriedri-ni 6d8318b
Address the linters
jfriedri-ni 3313691
Follow naming convention for covariant typevars
jfriedri-ni c1324ae
Rely on set_value throwing for test___unopened_panel___set_value___se…
jfriedri-ni da0a6d7
Merge remote-tracking branch 'origin/main' into users/jfriedri-ni/add…
jfriedri-ni d5d4c51
converters: Refactor message->Any conversion
bkeryan ae54c82
converters: Refactor to_python any handling
bkeryan 6ff26ea
Clarify some docstrings
jfriedri-ni 072a3ac
Use object in the to_any() and from_any() signatures
jfriedri-ni 83e6363
converters: Use type variables more consistently
bkeryan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,58 @@ | ||
| """Placeholder example for the package.""" | ||
|
|
||
| import enum | ||
jfriedri-ni marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| import nipanel | ||
|
|
||
|
|
||
| class MyIntFlags(enum.IntFlag): | ||
jfriedri-ni marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| """Example of an IntFlag enum.""" | ||
|
|
||
| VALUE1 = 1 | ||
| VALUE2 = 2 | ||
| VALUE4 = 4 | ||
|
|
||
|
|
||
| class MyIntEnum(enum.IntEnum): | ||
jfriedri-ni marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| """Example of an IntEnum enum.""" | ||
|
|
||
| VALUE10 = 10 | ||
| VALUE20 = 20 | ||
| VALUE30 = 30 | ||
|
|
||
|
|
||
| class MyStrEnum(str, enum.Enum): | ||
| """Example of a mixin string enum.""" | ||
|
|
||
| VALUE1 = "value1" | ||
| VALUE2 = "value2" | ||
| VALUE3 = "value3" | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| my_panel = nipanel.StreamlitPanel( | ||
| panel_id="placeholder", | ||
| streamlit_script_uri=__file__, | ||
| ) | ||
|
|
||
| my_types = { | ||
| "my_str": "im justa smol str", | ||
| "my_int": 42, | ||
| "my_float": 13.12, | ||
| "my_bool": True, | ||
| "my_bytes": b"robotext", | ||
| "my_intflags": MyIntFlags.VALUE1 | MyIntFlags.VALUE4, | ||
| "my_intenum": MyIntEnum.VALUE20, | ||
| "my_strenum": MyStrEnum.VALUE3, | ||
| } | ||
|
|
||
| print("Setting values") | ||
| for name, value in my_types.items(): | ||
| print(f"{name:>15} {value}") | ||
| my_panel.set_value(name, value) | ||
|
|
||
| print() | ||
| print("Getting values") | ||
| for name in my_types.keys(): | ||
| the_value = my_panel.get_value(name) | ||
jfriedri-ni marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| print(f"{name:>15} {the_value}") | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,230 @@ | ||
| """Functions to convert between different data formats.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import logging | ||
| from abc import ABC, abstractmethod | ||
| from typing import Any, Generic, Type, TypeVar | ||
|
|
||
| from google.protobuf import any_pb2, wrappers_pb2 | ||
| from google.protobuf.message import Message | ||
|
|
||
| _TPythonType_co = TypeVar("_TPythonType_co", covariant=True) | ||
| _TProtobufType_co = TypeVar("_TProtobufType_co", bound=Message, covariant=True) | ||
|
|
||
| _logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class Converter(Generic[_TPythonType_co, _TProtobufType_co], ABC): | ||
| """A class that defines how to convert between Python objects and protobuf Any messages.""" | ||
|
|
||
| @property | ||
| @abstractmethod | ||
| def python_type(self) -> Type[_TPythonType_co]: | ||
| """The Python type that this converter handles.""" | ||
|
|
||
| @property | ||
| @abstractmethod | ||
| def protobuf_message(self) -> Type[_TProtobufType_co]: | ||
| """The type-specific protobuf message for the Python type.""" | ||
|
|
||
| @property | ||
| def protobuf_typename(self) -> str: | ||
| """The protobuf name for the type.""" | ||
| return self.protobuf_message.DESCRIPTOR.full_name # type: ignore[no-any-return] | ||
|
|
||
| @abstractmethod | ||
| def to_protobuf(self, python_value: Any) -> any_pb2.Any: | ||
| """Convert the Python object to its type-specific message and pack it as any_pb2.Any.""" | ||
|
|
||
| @abstractmethod | ||
| def to_python(self, protobuf_value: any_pb2.Any) -> _TPythonType_co: | ||
| """Convert the protobuf message to its matching Python type.""" | ||
|
|
||
|
|
||
| class BoolConverter(Converter[bool, wrappers_pb2.BoolValue]): | ||
| """A converter for boolean types.""" | ||
|
|
||
| @property | ||
| def python_type(self) -> Type[bool]: | ||
| """The Python type that this converter handles.""" | ||
| return bool | ||
|
|
||
| @property | ||
| def protobuf_message(self) -> Type[wrappers_pb2.BoolValue]: | ||
| """The type-specific protobuf message for the Python type.""" | ||
| return wrappers_pb2.BoolValue | ||
|
|
||
| def to_protobuf(self, python_value: bool) -> any_pb2.Any: | ||
| """Convert a bool to a protobuf Any.""" | ||
| wrapped_value = self.protobuf_message(value=python_value) | ||
| as_any = any_pb2.Any() | ||
| as_any.Pack(wrapped_value) | ||
| return as_any | ||
|
|
||
| def to_python(self, protobuf_value: any_pb2.Any) -> bool: | ||
| """Convert the protobuf message to a Python bool.""" | ||
| protobuf_message = self.protobuf_message() | ||
| did_unpack = protobuf_value.Unpack(protobuf_message) | ||
| if not did_unpack: | ||
| raise ValueError(f"Failed to unpack Any with type '{protobuf_value.TypeName()}'") | ||
| return protobuf_message.value | ||
|
|
||
|
|
||
| class BytesConverter(Converter[bytes, wrappers_pb2.BytesValue]): | ||
| """A converter for byte string types.""" | ||
|
|
||
| @property | ||
| def python_type(self) -> Type[bytes]: | ||
| """The Python type that this converter handles.""" | ||
| return bytes | ||
|
|
||
| @property | ||
| def protobuf_message(self) -> Type[wrappers_pb2.BytesValue]: | ||
| """The type-specific protobuf message for the Python type.""" | ||
| return wrappers_pb2.BytesValue | ||
|
|
||
| def to_protobuf(self, python_value: bytes) -> any_pb2.Any: | ||
| """Convert bytes to a protobuf Any.""" | ||
| wrapped_value = self.protobuf_message(value=python_value) | ||
| as_any = any_pb2.Any() | ||
| as_any.Pack(wrapped_value) | ||
| return as_any | ||
|
|
||
| def to_python(self, protobuf_value: any_pb2.Any) -> bytes: | ||
| """Convert the protobuf message to Python bytes.""" | ||
| protobuf_message = self.protobuf_message() | ||
| did_unpack = protobuf_value.Unpack(protobuf_message) | ||
| if not did_unpack: | ||
| raise ValueError(f"Failed to unpack Any with type '{protobuf_value.TypeName()}'") | ||
| return protobuf_message.value | ||
|
|
||
|
|
||
| class FloatConverter(Converter[float, wrappers_pb2.DoubleValue]): | ||
| """A converter for floating point types.""" | ||
|
|
||
| @property | ||
| def python_type(self) -> Type[float]: | ||
| """The Python type that this converter handles.""" | ||
| return float | ||
|
|
||
| @property | ||
| def protobuf_message(self) -> Type[wrappers_pb2.DoubleValue]: | ||
| """The type-specific protobuf message for the Python type.""" | ||
| return wrappers_pb2.DoubleValue | ||
|
|
||
| def to_protobuf(self, python_value: float) -> any_pb2.Any: | ||
| """Convert a float to a protobuf Any.""" | ||
| wrapped_value = self.protobuf_message(value=python_value) | ||
| as_any = any_pb2.Any() | ||
| as_any.Pack(wrapped_value) | ||
| return as_any | ||
|
|
||
| def to_python(self, protobuf_value: any_pb2.Any) -> float: | ||
| """Convert the protobuf message to a Python float.""" | ||
| protobuf_message = self.protobuf_message() | ||
| did_unpack = protobuf_value.Unpack(protobuf_message) | ||
| if not did_unpack: | ||
| raise ValueError(f"Failed to unpack Any with type '{protobuf_value.TypeName()}'") | ||
| return protobuf_message.value | ||
|
|
||
|
|
||
| class IntConverter(Converter[int, wrappers_pb2.Int64Value]): | ||
| """A converter for integer types.""" | ||
|
|
||
| @property | ||
| def python_type(self) -> Type[int]: | ||
| """The Python type that this converter handles.""" | ||
| return int | ||
|
|
||
| @property | ||
| def protobuf_message(self) -> Type[wrappers_pb2.Int64Value]: | ||
| """The type-specific protobuf message for the Python type.""" | ||
| return wrappers_pb2.Int64Value | ||
|
|
||
| def to_protobuf(self, python_value: int) -> any_pb2.Any: | ||
| """Convert an int to a protobuf Any.""" | ||
| wrapped_value = self.protobuf_message(value=python_value) | ||
| as_any = any_pb2.Any() | ||
| as_any.Pack(wrapped_value) | ||
| return as_any | ||
|
|
||
| def to_python(self, protobuf_value: any_pb2.Any) -> int: | ||
| """Convert the protobuf message to a Python int.""" | ||
| protobuf_message = self.protobuf_message() | ||
| did_unpack = protobuf_value.Unpack(protobuf_message) | ||
| if not did_unpack: | ||
| raise ValueError(f"Failed to unpack Any with type '{protobuf_value.TypeName()}'") | ||
| return protobuf_message.value | ||
|
|
||
|
|
||
| class StrConverter(Converter[str, wrappers_pb2.StringValue]): | ||
| """A converter for text string types.""" | ||
|
|
||
| @property | ||
| def python_type(self) -> Type[str]: | ||
| """The Python type that this converter handles.""" | ||
| return str | ||
|
|
||
| @property | ||
| def protobuf_message(self) -> Type[wrappers_pb2.StringValue]: | ||
| """The type-specific protobuf message for the Python type.""" | ||
| return wrappers_pb2.StringValue | ||
|
|
||
| def to_protobuf(self, python_value: str) -> any_pb2.Any: | ||
| """Convert a str to a protobuf Any.""" | ||
| wrapped_value = self.protobuf_message(value=python_value) | ||
| as_any = any_pb2.Any() | ||
| as_any.Pack(wrapped_value) | ||
| return as_any | ||
|
|
||
| def to_python(self, protobuf_value: any_pb2.Any) -> str: | ||
| """Convert the protobuf message to a Python string.""" | ||
| protobuf_message = self.protobuf_message() | ||
| did_unpack = protobuf_value.Unpack(protobuf_message) | ||
| if not did_unpack: | ||
| raise ValueError(f"Failed to unpack Any with type '{protobuf_value.TypeName()}'") | ||
| return protobuf_message.value | ||
|
|
||
|
|
||
| # FFV -- consider adding a RegisterConverter mechanism | ||
| _CONVERTIBLE_TYPES = [ | ||
| BoolConverter(), | ||
| BytesConverter(), | ||
| FloatConverter(), | ||
| IntConverter(), | ||
| StrConverter(), | ||
| ] | ||
|
|
||
| _CONVERTER_FOR_PYTHON_TYPE = {entry.python_type: entry for entry in _CONVERTIBLE_TYPES} | ||
| _CONVERTER_FOR_GRPC_TYPE = {entry.protobuf_typename: entry for entry in _CONVERTIBLE_TYPES} | ||
| _SUPPORTED_PYTHON_TYPES = _CONVERTER_FOR_PYTHON_TYPE.keys() | ||
|
|
||
|
|
||
| def to_any(python_value: Any) -> any_pb2.Any: | ||
| """Convert a Python object to a protobuf Any.""" | ||
| underlying_parents = type(python_value).mro() # This covers enum.IntEnum and similar | ||
|
|
||
| best_matching_type = next( | ||
| (parent for parent in underlying_parents if parent in _SUPPORTED_PYTHON_TYPES), None | ||
| ) | ||
| if not best_matching_type: | ||
| raise TypeError( | ||
| f"Unsupported type: {type(python_value)} with parents {underlying_parents}. Supported types are: {_SUPPORTED_PYTHON_TYPES}" | ||
| ) | ||
| _logger.debug(f"Best matching type for '{repr(python_value)}' resolved to {best_matching_type}") | ||
|
|
||
| converter = _CONVERTER_FOR_PYTHON_TYPE[best_matching_type] | ||
| return converter.to_protobuf(python_value) | ||
|
|
||
|
|
||
| def from_any(protobuf_any: any_pb2.Any) -> Any: | ||
bkeryan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| """Convert a protobuf Any to a Python object.""" | ||
| if not isinstance(protobuf_any, any_pb2.Any): | ||
| raise ValueError(f"Unexpected type: {type(protobuf_any)}") | ||
|
|
||
| underlying_typename = protobuf_any.TypeName() | ||
| _logger.debug(f"Unpacking type '{underlying_typename}'") | ||
|
|
||
| converter = _CONVERTER_FOR_GRPC_TYPE[underlying_typename] | ||
| return converter.to_python(protobuf_any) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.