Migrate the ansible-consul role to a collection

[nre-ableton]

The ansible-consul role is one of the few roles that still lives under the ansible-community GitHub organization, and this has been a increasing point of pain and confusion in regards to the role's irregular name and namespace. In particular, there are two main annoyances:

1. Molecule tests no longer run because of the role's name and lack of a namespace attribute in meta/main.yml
2. Including the role is very awkward in requirements.yml files because it isn't available on Ansible Galaxy due to the current namespace

Therefore, I'm starting to wonder if we should migrate this role to be part of a collection (I would suggest community.consul). I'm not really sure where to reach out to the greater Ansible community, but it seems that this discussion board might be a good place. Please forgive me if there is a better place where we should discuss this matter.

---

Ansible Galaxy: N/A
GitHub: https://github.com/ansible-community/ansible-consul
Issues Tracker: https://github.com/ansible-community/ansible-consul/issues
GitHub Handles: @nre-ableton, @bbaassssiiee
Is the collection part of Automation Hub: No
We meet Collection Requirements: No (due to Ansible Galaxy, as mentioned above)

[bbaassssiiee]

ansible-consul, ansible-vault and ansible-nomad should IMHO be maintained by Hashicorp. If they don't pickup on this I suggest to delete these roles from ansible-community or to move them to a collection. Any volunteers to maintain them going forward?Bas ***@***.*** Hashicorp Ambassador, no longer involved.On 4 Sep 2023, at 18:02, Nik Reiman ***@***.***> wrote: The ansible-consul role is one of the few roles that still lives under the ansible-community GitHub organization, and this has been a increasing point of pain and confusion in regards to the role's irregular name and namespace. In particular, there are two main annoyances: Molecule tests no longer run because of the role's name and lack of a namespace attribute in meta/main.yml Including the role is very awkward in requirements.yml files because it isn't available on Ansible Galaxy due to the current namespace Therefore, I'm starting to wonder if we should migrate this role to be part of a collection (I would suggest community.consul). I'm not really sure where to reach out to the greater Ansible community, but it seems that this discussion board might be a good place. Please forgive me if there is a better place where we should discuss this matter. Ansible Galaxy: https://galaxy.ansible.com/NAMESPACE/COLLECTION GitHub: https://github.com/ansible-community/ansible-consul Issues Tracker: https://github.com/ansible-community/ansible-consul/issues GitHub Handles: @nre-ableton, @bbaassssiiee Is the collection part of Automation Hub: No We meet Collection Requirements: No (due to Ansible Galaxy, as mentioned above) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[nre-ableton]

ansible-consul, ansible-vault and ansible-nomad should IMHO be maintained by Hashicorp

I can't say that I disagree with this. I'll try to reach out to Hashicorp and see if they have any capacity to maintain these roles.

[briantist]

As the maintainer of community.hashi_vault (and hvac), I can say that in my experience HashiCorp is not well put-together in terms of supporting open source projects that aren't their own. There are people there who are smart and well-meaning but from the outside the company does not seem to be organized in a way where this is possible. I say this as a paying customer of HashiCorp, and someone who attends conferences and has talked to HC employees at conferences and events and my efforts to get more involvement from them have not been very successful.

The point of me saying this isn't to focus on HashiCorp though, it's to say that while asking them to take over maintenance would be convenient, it's unlikely to happen and to say that it isn't necessary for them to take over either.

If the roles in question have active maintainers, I see nothing wrong with having community supported collections for them.

(if you do get traction on these and find a good contact within HC, I am interested to talk with them too 🙂)

[nre-ableton]

@briantist Thanks for the information. Now I kind of wish we had a community.hashicorp collection where we could put all the OSS HC roles. 😄

[briantist]

The products are different enough and have such different requirements that I don't think it would make much sense to put them in a single collection, especially due to testing requirements. A collection is also released (versioned) as a unit which can be weird when you have unrelated roles put together. I would segregate them by product.

A collection doesn't have so much overhead (imo), so I don't think it's too crazy to make them separate collections even if they only have a single role. community.hashi_vault started with a single lookup plugin and remained that way for a while.

I think you'll find a lot of help in the community for setting up a new collection (CI, releases, versioning, documentation, etc.).

I took a quick look at the ansible-consul role and (unsurprisingly for a published role) it's pretty huge, and looks like it could probably be broken up into multiple smaller roles.

Breaking up a large role is awkward with standalone roles because there's no grouping to help people find the related or dependent roles, but it's quite natural in a collection since everything is together, so it could be a great opportunity for that.

[nre-ableton]

A bit of a delayed follow-up here, but due to the holiday downtime I've had some spare time to mess around with this a bit. I think that a collection would be a good solution for this role. The directory organization would then look like this:

.
├── meta
└── roles
    └── consul
        ├── defaults
        ├── examples
        ├── files
        ├── handlers
        ├── molecule
        │   ├── (various platforms by name)
        │   ├── _shared
        │   │   └── tests
        ├── tasks
        ├── templates
        └── vars

The various markdown/text files in the root directory would be moved to the new root directory, and the only code change that is necessary is in the shared converge.yml file:

--- a/molecule/_shared/converge.yml
+++ b/molecule/_shared/converge.yml
@@ -2,7 +2,7 @@
 - name: Converge
   hosts: all
   roles:
-    - role: ansible-consul
+    - role: consul

After doing this, I can finally run molecule test locally. 🎉

There is a question about the collection name. For my experiment, I used community.consul which seemed fine in isolation until I realized that using this role in a playbook would mean including community.consul.consul, which feels redundant. Should we follow @briantist's example and use community.hashi_consul?

However, if we proceed with this migration, It would mean that users of the current ansible-consul role would need to do the following:

1. Replace ansible-consul in their requirements.yml file with the new collection
2. Change the role name in their playbooks from ansible-consul to community.consul.consul

We should probably clear out the README.md file in ansible-consul before archiving the repo and replace it a link to the new repo. We would then have migration documentation in the new README, not the old one, since the old repo will be read-only and if we discover any errors in these instructions it will be difficult to fix them.

We'd also need to migrate issues from the old repo (51 open, I'm not sure how many are still relevant) and PRs (13, though at least a few of them are related to fixing ansible-lint and CI). For the issues, I would propose whipping up a simple shell script that uses gh to iterate over the existing issues and create issues in the new repo with the same title/body, and also add a "Migrated from ..." comment to link to the prior discussion. I don't think we should migrate comments, because the script won't be able to create those comments as the original users who posted them. Linking to the original discussions ought to be enough.

As for the PRs, I'm happy to handle them on a case-by-case basis, but this process would probably involve disabling CI on ansible-consul, locally testing the code for each PR in the new collection, and then merging. Otherwise, we should close the PRs or if they are otherwise unmergeable to notify the contributors. Obviously, we should take care of this before completing the migration.

Is anyone strictly opposed to the above? Are there any other concerns (such as the name etc)? Otherwise, what are the next steps? I'm happy to volunteer for most of the migration work, but obviously I'd need someone from the Ansible team to create the actual repo once we nail down the details. Ping @bbaassssiiee

[bbaassssiiee]

In 2020 I made former maintainer Brain a temporary member of my GitHub organisation. He transferred ownership of this role to that. This has the advantage of preserving Issues, PRs, etc under new stewardship. Then I transferred ownership to ansible-community. My permissions are too limited now to help transfer to ansible-collections.

[nre-ableton]

If we could migrate the repo, that would be even better. I wonder if we could arrange a similar transfer with ansible-collections.

But, if we go that route, we'd need to migrate to the collection layout pretty quickly. We could have a PR ready and merge it after the migration was completed.

[bbaassssiiee]

I guess @gundalow has the super-powers to move this.

[Andersson007]

Hello, i scanned the conversation, could you folks please briefly summarize what you'd like to achieve? If you want to request a brand new collection under ansible-collections, please open an issue in ansible-collections/overview, see examples in closed issues. We could also maybe fork the repo to ansible-collections with a preferred name and then you could try to convert it to a collection w/o changing the original repo, preserving commit history, etc. Please open an issue.

[nre-ableton]

I think that the best option would be to transfer our current repo to ansible-collections; that way we can preserve all issues and PRs. If this isn't possible/feasible, then forking would at least preserve git history, and we could migrate the issues by way of a script as described above.

If it's not possible to transfer the repo, then I'll open up an issue in the overview project as you mentioned.

[Andersson007]

@beechesII @rndmh3ro @ksatirli @lanefu @groggemans @jacobmammoliti are you, as maintainers, also OK with transferring the repo to the ansible-collections org?

[rndmh3ro]

+1 from me.

Aside: we should also do this with the nomad (and vault) role.

[beechesII]

+1

[Andersson007]

The repo has been transferred: https://github.com/ansible-collections/ansible-consul
People who had write have it there automatically. I also created a hashicorp-team there, they have admin. FYI
If it's OK now, please close the discussion.

[nre-ableton]

Thanks! I'll write up a note in the README today about the new location and start the migration process.

[Andersson007]

folks you're welcome!