ansible-collections / overview

Collections overview, how to request a namespace

Ansible Collections Overview

This repository is not used anymore.

If you are here to discuss Ansible community collections or request a repository for a community collection under the ansible-collections GitHub organization, see the communication guide.

overview's Issues

Stop/guide the incoming flow of PRs to ansible/ansible

PRs are still being raised against ansible/ansible. Some/lots of these PRs are against content that's a

Can we put something in the PULL_TEMPLATE to put people into a holding pattern?

This is particularly useful between now and devel being emptied out (23rd March) https://github.com/ansible-collections/overview/blob/master/README.rst#timeline

https://github.com/ansible/ansible/blob/devel/.github/PULL_REQUEST_TEMPLATE.md

CI for Collections: Part 1

For the repos we look after community.(aws,general,vmware,windows)

Define and implement what is out minimum level of CI.

• Passing on most PRs
• Nightly job (where do those results go?
• codecov.io (maybe nightly only?)

Whatever isn't done, can be delayed moved this into Part 2

Porting Guide entry for Collections

First stab: acozine/ansible@8b5388b

Fix module._name comparisons to also consider FQCNs

SUMMARY

See ansible-collections/community.general#230 for example.

Need to review pre-ansible-base/ and see which other modules need updating in different collections.

Also see if dev_guide need updating

ISSUE TYPE

• Bug Report

Guide to running your own Collection Community

Every Collection Repo is it's own (mini) Community.
Some of these Collection Repos are managed by individuals or Partners that maybe new to helping building and maintaining Open Source Communities, we should give them some guidance as well as a clear way to speak to us.

repo: community.zabbix

SUMMARY

(Official)? Zabbix module namespace by community for the:
https://github.com/ansible/ansible/tree/devel/lib/ansible/modules/monitoring/zabbix

Near future plans would be to deploy custom CI so we can test modules against every zabbix LTS release.

https://www.zabbix.com/life_cycle_and_release_policy

Name of collection

community.zabbix

GitHub maintainers

@D3DeFi
@sky-joker

Add repository link to galaxy page for collection

SUMMARY

On the GitHub landing page for this collection, it would be nice if the repo metadata links at the top of the page (which are currently empty—no description or website) had a link to https://galaxy.ansible.com/community/general

I like to be able to bounce between Galaxy and the source repo, and right now I can only go from Galaxy to the source repo and not vice-versa.

ISSUE TYPE

• Documentation Report

COMPONENT NAME

readme.md

ANSIBLE VERSION

N/A

repo: scaleway

SUMMARY

Name of collection

scaleway

GitHub maintainers

@scaleway/devtools

If assigning team does not work, you can use the individual GitHub profile:

@remyleone
@jerome-quere
@kindermoumoute
@QuentinBrosse
@loicbourgois

Issue Mover

We need some nicer to allow movement of issues from gh/ansible/ansible to gh/ansible-collections/REPO.

Built in GitHub functionality only allows Issues to be moved within the same GH Org.

PR mover is solved by #5

I've contacted GitHub to see if they have any ideas

• Add migrated_to_collection_repo https://github.com/ansible/ansible/labels/migrated_to_collection_repo to moved items.

Development non-blocking known issues and Limitations

At the time of the move to Collections there will be some differences and limitations in contributor workflow, we should keep an uptodate "Known issues and Limitations" list.

For example:

• Bot Limitations (labels, rebuild_merge)
• When CI may fail (and what to do about it)

repo: rabbitmq

SUMMARY

Name of collection

rabbitmq

GitHub maintainers

@chrishoffman @h-matsuo @jgkirschbaum @manuel-sousa @retr0h

move contrib/ to new home

There are various files in contrib + integration tests that need moving to their correct home

  grep 'needs/file/contrib/inventory' test/integration/targets/*/aliases
test/integration/targets/inventory_aws_conformance/aliases:needs/file/contrib/inventory/ec2.py
test/integration/targets/inventory_foreman_script/aliases:needs/file/contrib/inventory/foreman.py
test/integration/targets/vmware_inventory/aliases:needs/file/contrib/inventory/vmware_inventory.py
test/integration/targets/vmware_inventory/aliases:needs/file/contrib/inventory/vmware_inventory.ini

Remove use of ANSIBLE_METADATA in collection modules

SUMMARY

ANSIBLE_METADATA has two fields of use:

• supported_by - this field is no longer relevant as the collection itself is either certified or not.
• status - deprecated and removed are covered in the collectionsmeta/routing.yml

That leaves status of preview vs stable. Do collection owners still see relevance in that information per module? If so, let's create a DOCUMENTATION field to cover that.

ISSUE TYPE

• Bug Report

[1d] Remove `tests/sanity/requirements.txt` from collections

Some collections have a tests/sanity/requirements.txt file copied from the ansible/ansible repo.

This file is not used by ansible-test and should be removed to avoid confusion. I've seen it in two collections, but others which copied these may also have it:

• community.general
• community.zabbix

Call to action

• Please review your collections for this file and remove it if present. In the PR please reference this issue, so we can track where the fix has been applied.

Delta between ansible/ansible:devel and already existing collections

How can we review this?

Update collection tests from RHEL 8.1 to 8.2

Collections which are tested on Shippable against RHEL 8.1 should update to RHEL 8.2.

This requires an update to shippable.yml to replace T=rhel/8.1 with T=rhel/8.2. Here's one such line in ansible/ansible:

ansible/ansible@1c5c89b#diff-033d042b39454e8e066ab38fd89610a5L33-R33

Some tests may need to be updated.

Call to action

• If your collection uses Shippable for CI please update shippable.yml as detailed above. In the PR please reference this issue, so we can track where the fix has been applied.

[DOCS] Module Examples vs FQCN

SUMMARY

Need to document this:

1. Modules that were in ansible/ansible before the mass migration can continue to be used by the short form thanks to routing.yml
2. New modules (ie ones that have only existed in a Collection) will not work in the short form unless Playbook authors add the collection: form
3. EXAMPLES for new modules MUST have FQCN. This means Playbook authors can copy-paste examples without having to set collections:
4. FQCN and namespaces are a feature, we want to make more people aware of them. Though we MUST NOT break 2.9 Playbooks
5. For smaller collections like Crypto I think it's OK to bulk update all examples to use FQCN
6. EXCEPTION: I'm not sure if it makes sense to bulk update EXISTING modules docs in community,general. I expect we will continue to move content out of community.general into smaller Collections (like we've just done with Network, rabbitmq, etc). Adding in FQCN may confuse that matter.
7. No more entries (modules or plugins) will be added to routing.yml. The entries was frozen in time when pre-ansible-base was tagged. Hence 3.

Places to update

• Module & plugin docs
• Examples repos
• Training materials

repo: dellemc_networking.os10

SUMMARY

This shall be Ansible collection for Dell EMC Networking OS10
Currently, code is available at https://github.com/ansible-collection-migration/dell.os10

Name of collection

dellemc_networking.os10

GitHub maintainers

https://github.com/skg-net
https://github.com/parkrish

Document deprecation process

SUMMARY

Now that ansible/ansible#68646 has been merged we need to view and possibly correct how they are defined.

• Collections don't use a leading _ for deprecation, they use
  routing.yml instead

When raising PRs to fix this, please mention #58 in the PR, so we can track where this has been fixed

• community.general
• community.network

ISSUE TYPE

• Bug Report

repo: mongodb

SUMMARY

Name of collection

mongodb

GitHub maintainers

@rhysmeister

Community messaging

• Write overview doc #3
• Reddit https://www.reddit.com/r/ansible/comments/f6d4ut/community_update_on_ansible_collections/
• Ansible Twitter https://twitter.com/the_gundalow/status/1230170710900977664
• Partner email - Done
• Changes impacting Contributors
• Google Group: ansible-announce - Subset, in particular running from devel. link to ansible-project & Reddit
• Google Group: ansible-project
• Google Group: ansible-devel
• Google Group: ansible-outreach
• Update Collections Blog?
• Twitter Did you know that the @ansible devel branch is about to get a lot smaller (wordsmith to be much scarier)

How to set the develop branch as the default branch for my repo

SUMMARY

Hi admin,

My repo is https://github.com/ansible-collections/ibm.spectrum_virtualize
In the setting button, I didn't find any setting can be used to set the default branch, would you please help to guide me to do it? Thank you very much.

ISSUE TYPE

• Bug Report

repo: cassandra

SUMMARY

Name of collection

cassandra

GitHub maintainers

@rhysmeister

repo: libvirt

SUMMARY

Name of collection

libvirt

GitHub maintainers

@joshainglis @karmab @drybjed @odyssey4me

repo: dellemc_networking.sonic

SUMMARY

This shall be the Ansible collection repository for Dell EMC Networking SONiC

Name of collection

dellemc_networking.sonic

GitHub maintainers

https://github.com/skg-net
https://github.com/javeedf

Update collection tests from RHEL 7.6 to 7.8

Collections which are tested on Shippable against RHEL 7.6 should update to RHEL 7.8.

This requires an update to shippable.yml to replace T=rhel/7.6 with T=rhel/7.8. Here's one such line in community.general:

https://github.com/ansible-collections/community.general/blob/6dd5cb5dea38553a236656592d423ec258cf3297/shippable.yml#L26

Some tests may need to be updated. For example, the name of the RHEL extras repository changed in RHEL 7.7, which is known to affect at least one test. See:

https://github.com/ansible/ansible/blob/a681810b2f271abcc504629f944a7529d376791b/test/integration/targets/incidental_setup_docker/tasks/RedHat-7.yml#L15-L16

Call to action

• If your collection uses Shippable for CI please update shippable.yml as detailed above. In the PR please reference this issue, so we can track where the fix has been applied.

[?w] changelog process for collections

• How do we notify users of (potentially) breaking changes (aka porting_guide)
• How do we notify users of new features?
• Do we need to document bug fixes

Once we've answered the above, we can look at tooling that exists.

Migration.py limitations and manual steps

THE place to detail what happens after migrate.py gets run.

If you know of things that I've missed, please ping me, or add comments and I'll update this list.

• Clear out ansible-collection-migration (apart from ansible-base?)
• run migrate.py (FIXME what exact flags)
  • FIXME how do we know it's correct?
  • Store the full log somewhere
• For repos that aren't being moved
  • Archive (which makes them read only)
• If they are being moved, move them

Community Team Steps

Part 1

• Move repo or archive if it already exists.
• Settings app https://github.com/organizations/ansible-collections/settings/installations/5727515
• git checkout -b initial-conf
• git rm .github/workflows/collection-continuous-integration.yml
• Update galaxy.yml
  • repo +s -migration ssh->https
  • Issue link
  • Comment out dependencies
  • version 0.1.1
• Git tag 0.1.0
• Publish 0.1.0

Part 2

• Enable Settings GH App
• PR1
  • galaxy.yml uncomment dependencies
  • settings.yml (to enable branch protections)
  • Shippable
    • Enable Shippable https://app.shippable.com/subs/github/ansible-collections/enable
    • Shippable scripts - executible
    • shippable.yml
    • GH Action (Update checkout location x3) inc coverage
  • GitHub Action
    • Enable codecod
    • .github/workflows/ansible-test.yml - Update checkout location
    • Code coverage
  • OPTIONAL: Generate BOTMETA.yml

Collection Owner

• README.md
• galaxy.yml
• GitHub Owners
• Branch protects
• CI setup
  • Shippable
    • Copy tests/utils/shippable from FIXME
    • CodeCov Key Not needed for GitHub action
  • GitHub actions
    • copy .github/workflows/ansible-test.yml form FIXME
    • Update checkout location x3
  • Zuul - Who apart from Networking is using this from day one?
• For repos that already exist
  • Diff against generated repo against existing - protection against changes that's gone into devel since the repo was created.
• Generate BOTMETA.yml (for Ansibulbot repos)
• Galaxy Tags?

Things migrate.py doesn't do:

Questions

• How do we know where each of the generated GH Repos should move to.
• Which repos should be archived (network, grafana, mongodb, whatelse?)

repo: dellemc_networking.os6

SUMMARY

This shall be Ansible collection for Dell EMC Networking OS6

Currently, code is in https://github.com/ansible-collection-migration/dell.os6

Name of collection

dellemc_networking.os6

GitHub maintainers

https://github.com/skg-net
https://github.com/javeedf

Tidy Up Existing Collections

Description TBD

review Shippable usage and Ansible-tests change detection.

Once most of the repos are up an running we need to review Shippable configuration

ansible-test change detection

• Modify every file, see what is listed as unknown -ve

For each repo under gh/ansible-collections that uses Shippable

• Remove unused targets (ie OS). Review against alises files
• Review sanity durations and rebalance.

To rebalance https://github.com/ansible/ansible/blob/devel/hacking/shippable/rebalance.py

[1d] Add tags to all the galaxy.yml files

We need to add Galaxy Tags to the community. Collections

Some thought and care will be needed for community.general and community.network

Review community.general

Description TBD

PR mover workflow

• Define PR mover use-cases
• Host PR mover
• Ensure it works for various module and plugin types
• Document (and bot should link to) how to rewrite Python imports.
• Document to whatever else migrate.py rewrote in plugins
• Define what happens to PRs that modify ansible-base AND a collection
  • Does the bot know how to split PRs?
• Define what happens to PRs that modify MULTIPLE a collection
• Add migrated_to_collection_repo https://github.com/ansible/ansible/labels/migrated_to_collection_repo to moved items.
• Brad said we can test this on
  Code: https://github.com/ansible/prmove/tree/collections

Questions

• Does it know which repo to PRs to (BOTMETA lookup?)
• Should ansibulbot close PRs with a pre-populated PR mover URL?

Python imports
from ansible_collections.community.grafana.plugins.module_utils.base import grafana_argument_spec

Docs updates (personas master List)

SUMMARY

It's hopefully easier to track what improvements to the documentation are needed if we break it down by who, then secondly what.

Once we have an understanding of what needs doing we can link to specific PRs from here.

We may move content from https://github.com/ansible-collections/overview/blob/master/README.rst into the main doc site (though we must keep the sections in README.rst as we have given those links out)

Persona: User

• Installing and using
  • Using Collections alongside ansible-base
  • Updating to a newer version of a Collection

Persona: Contributor

• Where has my code gone (that was in a/a)
• What do I do with my a/a issues and PRs
• I want to contribute to a collection, how can I help
  • FIXME: list of ways they can help (review, docs, etc)
• How can I download and test a PR

Persona: Maintainer

Persona: Collection Owner

• How to setup CI

Persona: Proven Packager, Elders, Community Team

ISSUE TYPE

• Bug Report

DRAFT: Project Plan

Project Plan

Detail the remaining steps to get us to Ansible 2.10.

This is just a proposal, the order and contents of the milestones may change.

As a reminder ansible-base (gh/ansible/ansible) has it's own roadmap

How

1. Define the buckets
2. Ensure every thing in https://github.com/orgs/ansible-collections/projects/1 is an issue (not a note)
3. Create bucket labels in all repos
4. Assign buck labels
5. Issue titles to include [1d] ie [3d] Migrated Modules Messaging #48
6. Create filter links for Project Board for each bucket. Filter for no bucket labels

bucket1: Each Collection open for contributors

bucket2: Each collection published to Galaxy

AIM: Get the Collections into a state so people can test and give us feedback

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket2

• Foreach collection included in ansible
  • Automated Publishing on Git Tag for all community. collections
  • README
  • galaxy.yml (tags, dependencies, etc)

bucket3: Issues and PRs from a/a closed

AIM: Allow people to contribute

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket3

• Clear directions on what to do next
• Correctly direct people to their new location
  • Collections still to move: a10, AVI, Dell, Infoblox

bucket4: Versioning, deprecation

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket4

• What
• How
• Actions
• CI to enforce correct structure

ansible/ansible#66920

bucket5 pre-release ansible-2.10

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket5

AIM: A basic build of ansible-2.10 on PyPi to allow people to check that the new process works. Allows us to confirm that Ansible 2.9 playbooks work without modification

FIXME: Should this be split into multiple milestones

• collection redirection/tombstoning ansible/ansible#67684
• Collection loader loads and executes module code, causing problems with old libraries in path ansible/ansible#68701
• Collection loader: importing from module_utils/foo/init.py does not work ansible/ansible#68872
• Requires ansible-base build on PyPi

bucket6: No more content shuffling

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket6

• No more moving content from community.general or community.network into dedicated collections
• Validate that ansible-base's routing and botmeta is correct
• validate acd.in
• Ensure deprecated content is in the right collection

bucket7: Developer & Contributor guides

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket7

bucket8: feature freeze for Collections

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket8

• No new modules or plugins in any collection
• No new features

...

bucket9: Docs for each collection

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket9

bucket10: ansible published to all Packagers

https://github.com/orgs/ansible-collections/projects/1?card_filter_query=label%3Abucket10

Other bits

• CI improvements
• Governance
• Build questions
  • Single vs multiple packages
  • Launchpad PPA (need to find a contact: relrod has been talking to a Debian maintainer)

repo: dellemc_networking.os9

SUMMARY

This shall be Ansible collection for Dell EMC Networking OS9
Currently, code is in https://github.com/ansible-collection-migration/dell.os9

Name of collection

dellemc_networking.os9

GitHub maintainers

https://github.com/skg-net
https://github.com/javeedf

Changes impacting Collection Contributors and maintainers

NO LONGER IN USE

Please subscribe to ansible-collections/news-for-maintainers

Details of what's changed

In accordance with the Community decision, we have created the news-for-maintainers repository for announcements of changes impacting collection maintainers (see the examples) instead of this issue.

• To keep yourself well-informed and, therefore, things in your collection working, please subscribe to the repository by using the Watch button in the upper right corner on the repository's home page.
• If you do not want to get notifications about related discussions, please subscribe only to Issues.
• Please read the brief guidelines on how the repository should be used.
• Please avoid unnecessary discussions in issues, use the Discussions feature. Every comment posted will notify a lot of folks!

CI for Collections: Part 2

Following on from Part 1

Docs to use FQCN?

We need to confirm, though the thought is that we should update documentation to use FQCN.

Module & plugin docs
Examples repos
Training materials

Infra for maintainers of collections which are part of ansible-2.10+

• Create a list of contact information for all collection owners.
• Create a mailing list for all collection owners.

This infra should include both people who are maintaining what's in ansible-2.10+ and people who want to maintain collections in 2.10+

GitHub teams vs Bot

Define the ansible-collections version of Core & external contributors. These people will have powers on all of the Community Repos under gh/ansible-collection.

• Community Team (GDK's people)

• Community Leadership (external)

• Core Team?

• Update Bot to honor these teams for Collections (rather than gh/ansible teams)

• Reduce Bot user account so it no longer needs gh/ansible/ powers

• Update settings.yml to give these people powers?

Update collection tests from Fedora 30 to 32

Collections which are tested on Shippable against Fedora 30 should now test against Fedora 32 (so they are testing on 31 and 32), and at their discretion, drop support for Fedora 30 which is going end-of-life soon.

This requires an update to shippable.yml to add T=linux/fedora/32. Here's one such line in ansible/ansible:

ansible/ansible@de59b17#diff-033d042b39454e8e066ab38fd89610a5R40

Some tests may need to be updated.

Call to action

• If your collection uses Shippable for CI please update shippable.yml as detailed above. In the PR please reference this issue, so we can track where the fix has been applied.

https://ansible.sivel.net/pr/byfile.html for community.general (move to eng.ansible.com?)

Yep, this has been a useful tool and people rely upon it.

[3d] Migrated Modules Messaging

SUMMARY

It is not entirely clear where modules have moved to, while a google search may clear up some things it is still hard to see which collection a module has moved to.

I am proposing some documentation to list out where the modules have gone. I think a single page with the relevant information will go a long way.

Module	Galaxy Name	Collection	Repo
ios_*.	cisco.ios	https://galaxy.ansible.com/cisco/ios	https://github.com/ansible-collections/cisco.ios/
digital_ocean_*	community.general	https://galaxy.ansible.com/community/general	https://github.com/ansible-collections/community.general

Less of a bug, and more of a feature, but there was not ability to create a feature.

ISSUE TYPE

• Bug Report

CI Testing CI dependency

How to test proposed changes to a collection you depend on.

Speak to Paul

repo: jpmens

SUMMARY

Name of collection

jpmens

GitHub maintainers

jpmens

repo: IBM Spectrum Virtualize

SUMMARY

Name of collection

I want to use "IBM Spectrum Virtualize" as my repo name, but in my galaxy.yaml, the collection name is "spectrum_virtualize"

GitHub maintainers

wangpww

[3w] Ansible test cases (2.9 -> 2.10 without user impact)

SUMMARY

One of the primary design goals of Ansible 2.10 is that Ansible 2.9 Playbook should work without modification, to ensure that's the case we need to define some test cases

Playbooks

• OpenStack installer
• Ansible Tower installer
• debops
• https://github.com/ansible/ansible-examples
• https://github.com/ansible/workshops

Specific tests

• Plugin Types
  • become
  • cache
  • callback
  • cliconf
  • connection
  • httpapi
  • inventory
  • lookup
  • netconf
  • shell
  • module
  • strategy
  • vars
• Commands
  • ansible
  • ansible-playbook
  • ansible-doc

[2d] Galaxy build ignore file - tests/output? Others from gitignore

ansible-galaxy collection build has support for an ignore list to prevent files ending up in the built tarball.

Although ideally Collection builds should be happening in a sandbox (CI) their is still the chance that they may include incorrect files.

Also people do build and publish collections locally, where their checkouts maybe dirty.

Posisble entries

• tests/ (check ansible-base's .gitignore
• ignore .gitignore

Once agreed

• update collections under https://github.com/ansible-collections/
• Raise PR to update the default ignore file in https://github.com/ansible/ansible/tree/devel/lib/ansible/galaxy/data/default