Replies: 5 comments 3 replies
-
|
Some docs related comments from me: Developing and using inventory and execution connectors is not obvious to me ; i've started documentation for that at https://github.com/pyinfra-dev/pyinfra/compare/3.x...medeopolis:pyinfra:3.x-documentation-updates?expand=1 and will share it once i've confirmed its correct. Outstanding questions which i haven't added to that doc (because i don't have the answers yet)
Edit: on the subject of developing connectors: |
Beta Was this translation helpful? Give feedback.
-
|
Hi - Firstly, I must thank you and all the contributors to the project. Out of the various configuration automation systems out there, this fills an important role. It's approachable, uses an actual scripting language for specifying what is to be done, only executes shell commands on the remote systems (agentless), and puts idempotency and declarative end state up-front. Thank you! I just started with pyinfra; for me, below are some missing, confusing, and ambiguous pieces in the docs. If you are familiar with other config automation systems, you figure most of this out, but it shouldn't be expected or required. Warning: this is a long post. Top-levelThe homepage mentions a lot of positive traits/differentiators pyinfra possesses, but to see what that really means, you have to go all over the docs. Two ideas for improvement:
Getting Started
Using Operations
Inventory & Data
Global Arguments
FAQ
Deploy Reference -> Operations Index
How pyinfra works
If you read this far, thank you. At least for me, if the above were addressed, I'd be a lot clearer on how pyinfra works and how best to use/not use it. |
Beta Was this translation helpful? Give feedback.
-
|
It would also help if the descriptions of the api functions would match the function prototypes. |
Beta Was this translation helpful? Give feedback.
-
|
Are there.. any modules anywhere for doing common things? I'm looking for better examples as the examples repo is kind of weak. I'm trying to find things like:
|
Beta Was this translation helpful? Give feedback.
-
|
Another couple:
|
Beta Was this translation helpful? Give feedback.
-
|
Another - can't seem to get the SwapEnabled fact that is demonstrated to be of use.. from pyinfra import host
from pyinfra.facts.server import Arch
from pyinfra.operations import apt, files, server, ssh
from pyinfra.api import FactBase
class SwapEnabled(FactBase):
'''
Returns a boolean indicating whether swap is enabled.
'''
command = 'swapon --show'
def process(self, output):
return len(output) > 0 # we have one+ lines
server.shell(
name="Disable Swap",
commands=["swapoff -a"],
_if=host.get_fact(SwapEnabled),
)The fact returns a bool, so is there some sort of hidden EDIT: Worked out that we pass the bool through a lambda, would be good to demo that for facts docs. server.shell(
name="Disable Swap",
commands=["swapoff -a"],
_if=lambda: host.get_fact(SwapEnabled),
) |
Beta Was this translation helpful? Give feedback.
-
|
I've been working a lot with Pyinfra recently (almost 9 months now) while building an IaC framework of my own on top of it, as I do believe It is the best IaC API out there. Due to my current experience, I felt I should write a comment here! Init FlowOne of my biggest issues is that, while the docs explain the individual parts relatively well, it fails on building a complete mental model on how they work together. I had to dig through the source code quite a few times to actually understand things, and even with AI it was a pain. For instance, when using the API, the docs show a code snippet with a specific flow: inventory -> config -> state -> connect -> add_op. HOWEVERThe actual flow is something more like inventory -> config -> state - > state.current_stage = StateStage.Prepare -> ctx_state.set(state) -> connect -> add_op. The two parts that are important here are StateStage and ctx_state, which are not documented. About API interfaceAnother small friction point was how op results behave op = add_op(state, ...)
# To get the stdout for a specific host
op[host].stdout
# to get if all hosts changed
op.changedI understand the reasoning behind op.changed representing a global change state, but the interface feels a bit inconsistent because the object is partially dictionary-like and partially object-like. something like op["all"].changed or op.host(host).changed would be more consistent (I'd say the second one is better because of LSPs) About Ops and FactsAnother area that deserves significantly more attention is creating custom Facts and Operations. Pyinfra simply does not have the same number of modules as Ansible. That's perfectly fine, but it means that the ability to create your own extensions becomes extremely important. The project already makes this quite painless, but the documentation doesn't highlight it enough. Examples showing how to build:
would greatly improve discoverability and long-term maintainability of the ecosystem. Callback system (bro, fr)Something similar happens with the callback system. It's incredibly powerful but not very discoverable in the documentation. Using callbacks I was able to build a telemetry module that captures detailed execution data, including:
This allows building quite advanced observability around Pyinfra-based tooling. However, this capability isn't very visible in the docs today. Additionally (bigass request, but this could be cool)the add_op function works and is great, I DO NOT have an issue with it. What I do have an issue with is the fact that my LSP doesn't like it (cause of static analysis). I think that, instead of sunsetting the add_op implementation, an object could be made, something like OpAdder, that simply runs add_op's based on pyinfra objects passed onto it. OpAdder.add(pacman.package(...))why? Because like this we could have an type safe, easy to call and LSP compliant method to simply add ops. This is one of the main drawbacks in Pyinfra in API mode, we simply lose one of the project's main features, the LSP, because of the use of **kwargs in add_op summaryOverall, I’d say the main friction point for me was understanding the initialization/setup stage of the API and discovering some of the more powerful extension points. That said, once I got past that learning curve, pyinfra has become one of my favorite IaC tools (I’m literally building my own infrastructure framework on top of it.) |
Beta Was this translation helpful? Give feedback.
-
|
I think OpAdder could look something like so? from typing import TYPE_CHECKING
if TYPE_CHECKING:
from typing import Callable, ParamSpec, TypeVar
from pyinfra.api.host import Host
from pyinfra.api.state import State
from pyinfra.api.operation import add_op
P = ParamSpec("P")
T = TypeVar("T")
class OpAdder:
"""
Type-safe wrapper around pyinfra's add_op.
Preserves LSP autocomplete and type checking for operations.
"""
def __init__(self, state: State, host: Host):
self.state = state
self.host = host
def add(self, op_func: Callable[P, T], *args: P.args, **kwargs: P.kwargs) -> dict:
"""
Adds an operation to the state. The LSP will automatically
enforce that *args and **kwargs match `op_func`.
"""
if "host" not in kwargs:
kwargs["host"] = self.host
return add_op(self.state, op_func, *args, **kwargs) |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I’d like to improve pyinfra’s documentation, I think there’s a few things that open come up and clearly need documenting better. Off the top of my head some of these are:
server.Commandfact)Would be amazing to hear what other things are unclear to people, and any ideas on how to better document or structure the docs would be much appreciated! 🙏
Beta Was this translation helpful? Give feedback.
All reactions