aristanetworks / ansible-cvp Goto Github PK

View Code? Open in Web Editor NEW

65.0 24.0 61.0 4.91 MB

Ansible modules for Arista CloudVision

Home Page: http://cvp.avd.sh

License: Apache License 2.0

Python 95.43% Shell 0.47% Makefile 1.64% Dockerfile 0.01% CSS 0.60% Jinja 1.64% HTML 0.17% JavaScript 0.03%

devops ansible networking arista ansible-modules arista-cloudvision cv-device cvprac ansible-collection

ansible-cvp's People

Contributors

Stargazers

Watchers

Forkers

b-abadie jc83419 mpergament ajnouri apollocreed fredhsu mnfrankie cdorryarista ankudinov-arista-forks jrecchia1029 rongw22 jeremyblank saileshgiri navisidhu sashisekaran 4ever7 guillaumevilar spyton clausholbecharista bjmeuer noredistribution anshulbehl tonyli71 blemmenes mbalagot12 th3architect emilotep manuwelakanade elizabethabraham2018 ankudinov carlbuchmann konikachaurasiya-gslab pantunex colinmacgiolla sugetha24 mtache mthiel117 gmuloc pvinci-arista juliopdx pmerlo2022 kmueller68 alisonlhart shivani-chourasiya adosztal gmuloc-org vibhu-gslab jorisc90 ksarista edwinrg00 dboy1016 steffessonatype aetherrealm lsarista stephenamaki alexeygorbunov gravitasse tuarista

ansible-cvp's Issues

cv_device - Update to be idempotent

Data structure to use as module inputs:

---
 devices_list:
     veos1
     veos02:
        configlets:
          - Test_Configlet
        images:
          - 4.22.1F

Update cv_container for intend topology approach

Update cv_container to allow intend approach instead of a declarative approach.

Data model example:

---
containers:
  - name: Fabric
    parent_container: Tenant
  - name: Spines
    parent_container: Fabric
  - name: Leaves
    parent_container: Fabric

Logic

Compare {{containers}} structure with {{ansible_facts}} from cv_facts module
If a container is part of {{containers}} and not within {{ansible_facts}} then container will create it by cv_container
if a container is part of {{ansible_facts}} and not part of {{containers}}, then container will be deleted by cv_container

Feature Request - Allow cv_facts to only target a subgroup of facts

Is your feature request related to a problem? Please describe.

In some scenario, it might be important to rerun cv_facts to refresh a list of objects.

Current implementation requires to update everything and it might be time consuming in large deployments.

Describe the solution you'd like

Like gather_subset implemented in PR #92 , an option to target only a subset of facts would be useful. An approach might be:

  tasks:
    - name: '#01 - Collect devices facts from {{inventory_hostname}}'
      cv_facts_v2:
        facts:
          devices
      register: FACTS_DEVICES

In the meantime, it would be interesting to get access to fields sent by CVP as stated in issue #30

Describe alternatives you've considered

Alternative could be to update modules to refresh facts on the fly when needed by module. This approach might lead to non-optimum behavior as every time, module may have to resolve dependencies and execute more API calls than cv_facts do.

Add EXAMPLE block in cv_facts module

In order to provide auto-generated documentation for every module, cv_facts should implement EXAMPLES=""" block to demonstrate how to consume this module

Add fact filter to cv_facts

One more thing that might be useful is to be able to filter what type of facts you want to collect. for example, if you expect only tasks to have changed, then it should be possible to run cv_facts with something like filter: tasks to only make it update tasks.

[BUG] task not running after using cv_task to execute

Issue Type

Bug Report

Module Name
cv_tasks

`arista.cvp` collection 1.0.0
Python libraries version
Package                          Version
-------------------------------- -------
ansible                          2.9.0
Babel                            0.9.6
backports.ssl-match-hostname     3.5.0.1
cffi                             1.6.0
chardet                          2.2.1
cloud-init                       0.7.9
configobj                        4.7.2
cryptography                     1.7.2
cvprac                           1.0.1
decorator                        3.4.0
enum34                           1.0.4
futures                          3.1.1
fuzzywuzzy                       0.17.0
httplib2                         0.9.2
idna                             2.4
iniparse                         0.4
ipaddress                        1.0.16
IPy                              0.75
Jinja2                           2.7.2
jmespath                         0.9.0
jsonpatch                        1.2
jsonpointer                      1.9
kitchen                          1.1.1
MarkupSafe                       0.11
meld3                            0.6.10
mercurial                        2.6.2
paramiko                         2.1.1
passlib                          1.6.5
perf                             0.1
pip                              19.3.1
ply                              3.4
policycoreutils-default-encoding 0.1
prettytable                      0.7.2
psycopg2                         2.5.1
pyasn1                           0.1.9
pycparser                        2.14
pycurl                           7.19.0
pygobject                        3.22.0
pygpgme                          0.3
pyliblzma                        0.5.3
pyserial                         2.6
python-linux-procfs              0.4.9
pyudev                           0.15
pyxattr                          0.5.1
PyYAML                           3.10
rcvpapi                          1.5.31
requests                         2.6.0
schedutils                       0.4
seobject                         0.1
sepolicy                         1.1
setuptools                       0.9.8
six                              1.9.0
supervisor                       3.1.4
urlgrabber                       3.10
urllib3                          1.10.2
virtualenv                       15.1.0
yum-metadata-parser              1.1.4

OS / Environment

CentOS Linux release 7.5.1804 (Core)

Summary

i created a playbook to update the configlet of leaf

---
- name: Test cv_device
  hosts: all
  connection: local
  gather_facts: no
  collections:
    - arista.cvp
    - arista.cvp
  vars:
    configlet_list:
      t_a_configlet: "!\nvlan {{ vlan_no }}\n   name {{ tenant_name }}\n!\nvrf instance {{ tenant_name }}\n!\ninterface Vlan{{ vlan_no }}\n   vrf {{ tenant_name }}\n   ip address virtual {{ gateway_no }}\n!\ninterface Vxlan1\
    # Device inventory for provision activity: bind configlet
    devices_inventory:
      Leaf01:
        name: Leaf01
        configlets:
          - MLAG-LEAF1
          - eBGP_Underlay_LEAF01
          - RECONCILE_Leaf1
          - t_a_configlet
  tasks:
      # Collect CVP Facts as init process
    - name: "Gather CVP facts from {{inventory_hostname}}"
      cv_facts:
      register: cvp_facts
    - name: 'Create configlets on CVP {{inventory_hostname}}.'
      cv_configlet:
        cvp_facts: "{{cvp_facts.ansible_facts}}"
        configlets: "{{configlet_list}}"
        configlet_filter: ["{{ tenant_name}}"]
      register: cvp_configlet

    # Display information for device before a container's move
    - name: "Configure devices on {{inventory_hostname}}"
      cv_device:
        devices: "{{devices_inventory}}"
        cvp_facts: '{{cvp_facts.ansible_facts}}'
        device_filter: ['Leaf']
      register: cvp_device

    - name: Display cv_device
      debug:
        msg: "{{cvp_device}}"

    - name: 'RUN 1: Execute tasks registered in the previous task'
      cv_task:
        tasks: "{{ cvp_device.data.tasks }}"
      register: cvp_tasks

the tasks output:

TASK [RUN 1: Execute tasks registered in the previous task] ************************************************************************************************************
task path: /opt/arista/cvp_configlet-apply.yml:123
[WARNING]: No actionable tasks found on CVP

ok: [xxxxxx.xxxxxxxx.xxxxxxxxx] => {
    "changed": false,
    "data": {},
    "invocation": {
        "module_args": {
            "state": "executed",
            "tasks": [
                {
                    "actionStatus": "ACTIVE",
                    "currentAction": "Submit",
                    "description": "Ansible Configlet Update: on Device Leaf03",
                    "displayedStutus": "Pending",
                    "name": "",
                    "note": "",
                    "status": "ACTIVE",
                    "taskNo": "74"
                },
                {
                    "actionStatus": "ACTIVE",
                    "currentAction": "Submit",
                    "description": "Ansible Configlet Update: on Device Leaf02",
                    "displayedStutus": "Pending",
                    "name": "",
                    "note": "",
                    "status": "ACTIVE",
                    "taskNo": "75"
                },
                {
                    "actionStatus": "ACTIVE",
                    "currentAction": "Submit",
                    "description": "Ansible Configlet Update: on Device Leaf01",
                    "displayedStutus": "Pending",
                    "name": "",
                    "note": "",
                    "status": "ACTIVE",
                    "taskNo": "76"
                },
                {
                    "actionStatus": "ACTIVE",
                    "currentAction": "Submit",
                    "description": "Ansible Configlet Update: on Device Leaf07",
                    "displayedStutus": "Pending",
                    "name": "",
                    "note": "",
                    "status": "ACTIVE",
                    "taskNo": "77"
                },
                {
                    "actionStatus": "ACTIVE",
                    "currentAction": "Submit",
                    "description": "Ansible Configlet Update: on Device Leaf08",
                    "displayedStutus": "Pending",
                    "name": "",
                    "note": "",
                    "status": "ACTIVE",
                    "taskNo": "78"
                }
            ],
            "wait": 0
        }
    }
}

Steps to reproduce

Please find the playbook code above

Expected results

Pending task to be executed.

Actual results

Unable to run the task.

[WARNING]: No actionable tasks found on CVP

Get partial facts if a device is not accessible.

Hey @titom73

Might be related to #95

Issue Type

Question/feature request

Module Name

facts

OS / Environment

2018.2.5

Summary

We are testing the current modules on an IPFabric lab testbed, but one of the devices has authentication mode changed (through tacacs), so no more available with CVP username/login credentials; which makes the module facts fails with the device MAC.

Is it possible to still get the JSON output for all other devices and mention the ones failed? In case we want to exclude the failed machine from the deployment.

Thanks.

cannot execute tasks with cv_task module

Might be related to #79

Issue Type

Bug Report

Module Name

cv_task

arista.cvp collection and Python libraries version

ansible 2.9.2
  config file = /scripts/test_unitaires_aristacvp/ansible.cfg
  configured module search path = [u'/scripts/plugins/modules', u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15 (default, Jan 31 2019, 00:15:21) [GCC 6.3.0]

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
ansible==2.9.2
certifi==2019.11.28
cffi==1.13.2
chardet==3.0.4
cryptography==2.8
enum34==1.1.6
idna==2.8
ipaddress==1.0.23
Jinja2==2.10.3
MarkupSafe==1.1.1
pycparser==2.19
PyYAML==5.2
requests==2.22.0
six==1.13.0
treelib==1.5.5
urllib3==1.25.7

OS / Environment

CloudVision version
2018.2.5
UI version1.11.0

Summary

The tasks are not executed. The pending tasks are been identified but the module cv_task returns error

Steps to reproduce

CVP before :

Playbook

---
- name: Playbook to validate and demonstrate cv_container module.
  hosts: cvp
  connection: local
  gather_facts: no
  collections:
    - arista.cvp
  
  vars:
    containers_provision:
      IPFabric:
        parent_container: Tenant
        devices:
        - eos-1
        - eos-2
        - eos-3
        - eos-4

  tasks:
    - name: "Gather CVP facts from {{inventory_hostname}}"
      arista.cvp.cv_facts:
      register: cvp_facts
      tags:
        - always

    - name: "Build Container topology on {{inventory_hostname}}"
      arista.cvp.cv_container:
        topology: '{{containers_provision}}'
        cvp_facts: '{{cvp_facts.ansible_facts}}'
        save_topology: true
        mode: override
 #######################################################################""   
    - name: Display tasks
      tags:
        - provision
      debug:
        msg: "{{tasks}}" 

    - name: Display cvp_facts.ansible_facts.tasks
      tags:
        - provision
      debug:
        msg: "{{tasks}}"  

    - name: 'Execute all pending tasks and wait for completion for 60 seconds'
      arista.cvp.cv_task:
        tasks: "{{ tasks }}"
        wait: 60

CVP after execution the playbook

Expected results
the tasks shown in cvp should have been executed

Actual results

Tasks created but in pending

cv_facts - Error when collecting device config from Folsom

When collecting facts from a 2018.2.x server, ansible-playbook times out and display this error:

module_utils/cv_client.py\", line 520, in _make_request
requests.exceptions.ReadTimeout: 
HTTPSConnectionPool(host='10.90.224.122', port=443): 
Read timed out. (read timeout=30)

Server is reachable and HTTPS session is established.

Multiple behavior can be seen:

Timeout if devices are part of undefined container. (Manual GET is working)
API error if device is not streaming with error message above

GET https://10.83.28.164/web/inventory/getInventoryConfiguration.do?netElementId=0c:d7:09:6a:48:c1

{"errorCode":"172602","errorMessage":"An EAPI request failure occurred"}

Function in charge of collecting configuration from CVP: cv_api2018.py#L163

Implement state=absent to reset devices in cv_device

Like cv_configlet, it might be useful to get an option in cv_device to reset devices and move them back to undefined container.

Playbook example

- name: Build Switch configuration
  hosts: DC1_FABRIC
  connection: local
  gather_facts: no
  vars:
      CVP_DEVICES:
        DC1-SPINE1:
          name: DC1-SPINE1
          parentContainerName: DC1_SPINES
          configlets:
              - AVD_DC1-SPINE1
          imageBundle: []
  tasks:
    - name: Get facts
      cv_facts
      register: FACTS
    - name: reset devices
      cv-device:
        devices: "{{CVP_DEVICES}}"
        cvp_facts: '{{FACTS.ansible_facts}}'
        device_filter: ['DC1']
        state: absent

Expected results

Devices moves back to undefined.
Clean configuration.
Restart ZTP process to onboard device.

Implement mode for cv_container

Implement mode for `cv_container`

CLI Implementation

  tasks:
    - name: "Gather CVP facts from {{inventory_hostname}}"
      cv_facts:
      register: cvp_facts
    - name: "Build Container topology on {{inventory_hostname}}"
      cv_container:
        topology: '{{containers_provision}}'
        cvp_facts: '{{cvp_facts.ansible_facts}}'
        mode: merge
        save_topology: true

Supported keywords:

□ merge
□ override
□ delete

Keywords description:

B(override) - Discard the entire existing container topology and replace it
with the new topology. If devices are attached to containers, these ones are not deleted.
B(merge) - Combine the new container topology with the existing
topology. If containers in the new toplogy conflict with
containers in the existing toplogy, the containers definition in
the new topology replace those in the existing
Topology (mainly used for device attachement).
B(delete) - Discard the entire container topology provided in B(topology) and keep other existing containers untouched.

Implement module_utils to package python code

Implement module_utils to package all embeded python code related to modules:

Create a modue_utils folder
Update ansible.cfg to point to correct module_utils folder
Add content and update modules accordingly

Allow for a more granular approach to idempotence

Hello,

I am integrating ansible-cvp's modules for a mutual customer of ours and have a request : is it possible to have a more granular approach to the idempotence concept ?
In the current approach with fully idempotent modules, we have to feed them a full topology of containers, devices, images and configlets. They would then proceed and add or modify what needs to be added or modified, but also delete what is found on CVP and not in the topology.

While I like this approach and see the value in it, it causes some issues with the workflow/use-case I am currently implementing.

In particular, I see two topics where it is the case :

cv_container and device deletion

My use case requires to have all switches in the Ansible inventory, each with its group vars and host vars that will ultimately be used to build the final config, while using CVP to bootstrap and configure them. It also requires to leverage the --limit feature of Ansible to only affect a subset of the switches at a time (for slow release and redundancy considerations).

This is currently achieved by having two plays : the first one targeting the switches to compute all the necessary vars without actually making any outgoing connection, and the second one targeting CVP to actually create containers and move devices around.

With the current fully idempotent approach, having the playbook executed on two separate sets of switches would have the second pass delete the result of the first one. I know I could build a complete list of devices with the inventory, but that would defeat the point of the --limit option. Also, some devices way be configured in the inventory but not booted yet.

My request is to have an option to cv_container that would prevent the deletion or move of a device that is in CVP but not in the topology. Another possibility is to have cv_container only create empty containers and have a parameter container: to cv_device.

I know it looks like a setback from the "full idempotence" model, but here is my rationale : in a classic use of Ansible, if I remove a server from the inventory, Ansible will not do any cleanup, VM deletion or bare metal poweroff. Yet this is acceptable.

cv_device and configlet deletion

In the same way, CVP assigns configlets to devices. I'd like to be able to add and edit only a subset of the configlets applied to a particular switch without changing or deleting the others.

The idea behind this is that if I have a playbook to configure a switch's VNIs, I would not want it to touch the underlay or AAA config, no matter what. This is similar to having a playbook configure Apache's virtual hosts that would uninstall an unrelated package, change the allowed SSH keys or upgrade the kernel.

In the end, it is only a matter of how you consider CVP : as a central point that always describes the complete infrastructure (device list, full config and EOS versions), or as a proxy to access EOS switches (and add a whole lot of cool features, I know).

The point of my request for "granular idempotence" is to let the user choose between the two viewpoints, with some modules parameters of something.
Or, even better, have the best of the two : in my workflow I plan on also using the modules with the full topology, in forced dry-run, to hunt for lost devices or rogue config.

[Feature] - Enable CI with Github Actions

Since Github Actions is available for the repository, it should be interesting to enable CI for the following tasks:

Python code linting for any commit in issues/* and features/*
Ansible Code Validation for any PR in releases/*
Publication and release management for any Push in master

Some other actions could be implemented later for different needs.

Implement filter for cv_container

Implement filter option

CLI Implementation

  tasks:
    - name: "Gather CVP facts from {{inventory_hostname}}"
      cv_facts:
      register: cvp_facts
    - name: "Build Container topology on {{inventory_hostname}}"
      cv_container:
        topology: '{{containers_provision}}'
        cvp_facts: '{{cvp_facts.ansible_facts}}'
        filter: ['DC1', 'DC2']
        save_topology: true

Supported keywords:

□ List of string to match

Keywords description:

If B(filter) is set, then, cv_container will only manage containers started with any of these prefixes or attached to a container using these prefixes.
If B(filter) is unset, behavior will be applied to all containers on CVP and within expected topology

When root container is not Tenant, cv_container fails

Issue Type

Bug Report

Module Name

cv_container

Description

Module assumes root container is always named Tenant as it is the default value. But some design have this value changed.

When root container is not Tenant then, module starts an infinite loop as it is looking for a non-existing key.

Instead of assuming root is Tenant, module should identify the root container and use it as base for the tree representation

Copy CVP JSON into facts as is

I've noticed we've been filtering what data is being copied from JSON received from CVP into CVP facts, e.g. https://github.com/aristanetworks/ansible-cvp/blob/grant-release/library/cv_configlet.py#L281
This creates several problems:

Our objects (facts, registered variables) now have a different structure from CVP, which users may not expect
The filters and remapping of keys (as in the case of tasks) creates confusion as the same piece of code tasksField dict now needs to be updated in multiple modules, e.g. cv_fact, cv_configlet, cv_device, i.e. anything that can generate a task object.

I may not know the initial reason for doing this filtering but can I suggest we discuss going back to just copying objects as-is, without modifications. This way we don't create unnecessary circular dependencies between modules and reduce some of the code complexity.

[BUG] - ansible-test raise many sanity errors

Issue Type

Code Sanity

Summary

Update repository to pass all tests run by ansible-test sanity

List of tests
Complete command outputs: https://gist.github.com/titom73/c9ae3e05dcc2e5f061c9887943265471

Steps to reproduce

$ mkdir ansible_collections
$ cp -r arista/ ansible_collections
$ cd ansible_collections/arista/cvp/
$ ansible-test sanity --docker --python 2.7 --requirements requirements.txt $MODULE_NAME$
[...]
Tests Results
[...]

Tests use quay.io/ansible/default-test-container (version: 1.10.1 / size:4.03GB)

When collecting facts, only last configlet appears and as many times as there are devices

Issue Type

Bug Report

Module Name

cv_facts

Versions Info

ansible 2.9.0rc4
python version = 2.7.15
arista-cvp-1.0.0
CVP  2018.2.5

Summary

When running the tasks below, in containers > configlets, there is only one configlet appearing and it appears as many times as there are devices (in my case 6 times). See below the configlet named Vlan_test3.

tasks:
    - name: "Gather CVP facts {{inventory_hostname}}"
      cv_facts:
      register: cv_facts

    - name: "Print out facts from CVP"
      debug:
        msg: "{{cv_facts}}"
      when: verbose|bool

Actual results

ok: [cvp2] => {
    "msg": {
        "ansible_facts": {
            "configlets": [
                {
                    "config": "",
                    "containers": [],
                    "devices": [],
                    "key": "SYS_TelemetryBuilderV2_2404547584890",
                    "name": "SYS_TelemetryBuilderV2",
                    "type": "Builder"
                },
                {
                    "config": "vlan 2\n  name test2",
                    "containers": [],
                    "devices": [],
                    "key": "configlet_4_1976073421368445",
                    "name": "Vlan_test2",
                    "type": "Static"
                },
                {
                    "config": "vlan 3\n  name test3",
                    "containers": [],
                    "devices": [],
                    "key": "configlet_72_1986529749166153",
                    "name": "Vlan_test3",
                    "type": "Static"
                }
            ],
            "containers": [
                {
                    "childContainerKey": null,
                    "configlets": [],
                    "devices": [],
                    "imageBundle": "",
                    "key": "root",
                    "name": "Tenant",
                    "parentName": null
                },
                {
                    "childContainerKey": null,
                    "configlets": [
                        "Vlan_test3",
                        "Vlan_test3",
                        "Vlan_test3",
                        "Vlan_test3",
                        "Vlan_test3",
                        "Vlan_test3"
                    ],
                    "devices": [
                        "DC4-Leaf1a",
                        "DC4-Leaf1b",
                        "DC4-Leaf2a",
                        "DC4-Leaf2b",
                        "DC4-Spine1",
                        "DC4-Spine2"
                    ],
                    "imageBundle": "EOS-4.20.11M",
                    "key": "undefined_container",
                    "name": "Undefined",
                    "parentName": "Tenant"
                },
                {
                    "childContainerKey": null,
                    "configlets": [],
                    "devices": [],
                    "imageBundle": "",
                    "key": "container_54_1983658155049676",
                    "name": "staging",
                    "parentName": "Tenant"
                }
            ]

cv_container - Update module for Grant

Align cv_container with new information provided by cv_facts module.

It also has to use following input structure:

---
container_topology: 
   Fabric:
      parent_container: Tenant
   Spines:
      parent_container: Fabric
      configlets:
        - container_configlet
      images:
        - 4.22.0F
      devices:
        - veos01

Module features:

Container creation / deletion for 2018.2.x
Container creation / deletion for 2019.1.x
Device move to the correct container for 2018.2.x
Device move to the correct container for 2019.1.x
Configlet attachement (previously created by cv_configlet) for 2018.2.x
Configlet attachement (previously created by cv_configlet) for 2019.1.x
Module output definition: list of actions & taskIds

Create VSCode devcontainer for easy setup

Is your feature request related to a problem? Please describe.

In order to allow user to start quickly with the repository, it might be interesting to provide materials to automatically build a development environment.

Describe the solution you'd like

Visual Code is a well known text editor with a function to open a project in a specific container to make environment independent from local workstation.

Describe alternatives you've considered

Build a VM with pre-installed tools and library
Make documentation to install virtual-environment

Feature Request - Build example use-cases

Provide some example use-cases under examples folder to demonstrate how to use modules in a lab environment.

Examples should be done on a per folder approach with documentation to explain how to build CVP/EOS environment.

.
├── Example01
├── Example02
├── Example03
├── Makefile
├── README.md
...

Create a Dockerfile for the repository

Create a dockerfile to build an image to run test locally and reduced list of requirements to install on user environment.

Features:

Install repository requirements.
Build and install collection with the current repository version.
Export examples folder to ship content to test to users.

Inactive devices causing get facts to fail

verbose-output.txt

[BUG] cv_configlet will execute ALL tasks, instead of only the ones affected

Issue Type

Bug Report

cv_configlet

The error can be found here:
https://github.com/aristanetworks/ansible-cvp/blob/master/arista/cvp/plugins/modules/cv_configlet.py#L253

This approach results in all tasks being picked up and executed. This should be changed to only return tasks that have resulted from the configlet change.

Add cv_facts documentation in README

Add documentation about cv_facts module in README file like any other modules.

Todos:

Module description
Module options
Usage in Example playbook section

cv_container - module fails if topology is empty

If to make a container's clearnup, we provide an empty topology structure, module fails with following error message:

line 227, in create_new_containers\nTypeError: 'NoneType' object is not iterable

Example playbook:

---
- name: Build Topology.
  hosts: cvp
  connection: local
  gather_facts: no
  vars:
    verbose: False
    containers_cleanup:
  tasks:
    - name: "Gather CVP facts {{inventory_hostname}}"
      cv_facts:
        host: '{{ansible_host}}'
        username: '{{cvp_username}}'
        password: '{{cvp_password}}'
        protocol: https
      register: cvp_facts
      tags:
        - always

    - name: "Build Container topology on {{inventory_hostname}}"
      tags:
        - cleanup
      cv_container:
        host: '{{ansible_host}}'
        username: '{{cvp_username}}'
        password: '{{cvp_password}}'
        topology: '{{containers_cleanup}}'
        cvp_facts: '{{cvp_facts.ansible_facts}}'

Feature Request - Support option to gather device config in cv_fact

Is your feature request related to a problem? Please describe.

In large environment, collecting facts is getting more and more time because of collecting device configuration for every streaming device in CVP.

These configurations are not necessary in most of scenario as modules only focus on CVP provisionning and not on EOS device configuration.

As facts could be collected at different times in playbooks, overall execution time can be very important.

Describe the solution you'd like

An example of implementation might be:

- name: collect default set of facts
  cv_fact:

- name: collect default set of facts and eos configuration
  cv_fact:
    gather_subset: config

With this approach it is easy to extend this option to any other additional content like lldp or any command output.

Import missing API function from Hugh's repository

Missing functions to import:

Prepare repository to support Ansible Collection

As Ansible is moving to a new method to package and deliver 3rd part content, repository should start to adopt new collections structure as described on their website

Another reason to apply this model is to embed our own version of cvprac with no patch to fix current limitation. So there is no interaction with Python's setup on user side.

Structure to build a collection

as described on Ansible website:

collection/
├── docs/
├── galaxy.yml
├── plugins/
│   ├── modules/
│   │   └── module1.py
│   ├── inventory/
│   └── .../
├── README.md
├── roles/
│   ├── role1/
│   ├── role2/
│   └── .../
├── playbooks/
│   ├── files/
│   ├── vars/
│   ├── templates/
│   └── tasks/
└── tests/

Resources:

First iteration will only provide module packaging and example playbooks. Additional roles and playbook for specific use cases might come in a second iteration.

As it is in development, everything could change and this issue should be developed using ansible@devel branch to get all fixes.

To build package:

$ cd arista/cvp/
$ ansible-galaxy collection build --force
$ ansible-galaxy collection install *.tar.gz -p collections/ansible_collections/

Complete example ci file

Update cv_facts and cv_configlet to be idempotent

Update modules with content from CVP_Ansible_Modules

Use module_utils as referenced in issue #17

Test modules to provide same structure
Build documentation for these 2 modules: DOCUMENTATION and EXAMPLE blocks
Protect sensitive data like password
Update README file
Update installation process
Create How to use modules

Acceptance tests must be done against 2018 & 2019

cv_facts multiple minor bugs

Duplicate key definition
https://github.com/aristanetworks/ansible-cvp/blob/grant-release/library/cv_facts.py#L112
Unused variable
https://github.com/aristanetworks/ansible-cvp/blob/grant-release/library/cv_facts.py#L202
Not sure about this one - we don't have to register any variable since cv_facts changes ansible_facts, effectively settings those variables directly on inventory_hostname object. I get why you'd need this for debugging, but we can safely remove it from examples to reduce the visual clutter.
https://github.com/aristanetworks/ansible-cvp/blob/grant-release/library/cv_facts.py#L73

Feature Request - Build project documentation

Create Wiki pages to document project outside of the release management for critical topics:

Installation.
User guide.
Frequently Asked Question.
External resources.

Manage device registration within undefined_container

Have cv_device or cv_task support check mode and return a diff

Hello,
Currently, cv_device applies a configlet to a device, and cv_task actually changes the config on EOS (by executing the pending task).

It can be useful to have one of these two tasks to return a diff of what was applied (the result of the reconcile, I guess), similar to what eos_config does.
In the same spirit, it would be useful to have a dry run mode, to test the playbook execution without touching the switches.

Add explicit delete option via absent=True

With the new idempotent approach the only way to delete an object is by removing a reference to it. This works fine for some cases, but sometimes it may be required to explicitly remove large number of objects (e.g. cleanup, migration) or undo the previously created state. A lot of Ansible's core modules support an optional state parameter, which can be passed to tasks and modules to explicitly modify state.

I think, once we get the idempotent behaviour sorted for all our main resources, we can add this fairly easily.
For Example :

    - name: 'Create configlets on CVP {{inventory_hostname}}.'
      cv_configlet:
        configlets: "{{configlet_list}}"
        configlet_filter: ["New", "Test","base-chk","base-firewall"]
      state: absent

Will delete all configlets from "{{configlet_list}}" from CVP.

Enable python 3 support for all modules execution

Issue Type

Bug Report

Module Name

arista.cvp collection and Python libraries version

$ python --version                                                       
Python 3.7.3

$ ansible --version         
ansible 2.9.1
  config file = ansible-cvp/ansible_collections/arista/cvp/tests/ansible.cfg
  configured module search path = [~/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = ~/.venv/venv3.0-cvp-ansible/lib/python3.7/site-packages/ansible
  executable location = ~/.venv/venv3.0-cvp-ansible/bin/ansible
  python version = 3.7.3 (default, Mar 27 2019, 09:23:15) [Clang 10.0.1 (clang-1001.0.46.3)]

Summary

Modules are not python 3 compliant and still require usage of python 2.7 which is going to be end of support 1st January 2020

# cv_facts
TypeError: 'dict_values' object does not support indexing

# cv_container
File \"/var/folders/bj/x0pdlg4s46vbk9t8916xkfb00000gn/T/ansible_cv_container_payload_38idtqv_
/ansible_cv_container_payload.zip/ansible_collections/arista/cvp/plugins/
modules/cv_container.py\", line 152, in tree_to_list\n  File \"/var/folders/bj/x0pdlg4s46vbk9t8916xkfb00000gn/T/ansible_cv_container_payload_38idtqv_/ansible_cv_container_payload.zip/ansible_collections/arista/cvp/plugins/
modules/cv_container.py\", line 152, in tree_to_list\n  File \"/var/folders/bj/x0pdlg4s46vbk9t8916xkfb00000gn/T/ansible_cv_container_payload_38idtqv_/ansible_cv_container_payload.zip/ansible_collections/arista/cvp/plugins
/modules/cv_container.py\", line 152, in tree_to_list\n  File \"/var/folders/bj/x0pdlg4s46vbk9t8916xkfb00000gn/T/ansible_cv_container_payload_38idtqv_/ansible_cv_container_payload.zip/ansible_collections/arista/cvp/plugins/
modules/cv_container.py\", line 138, in tree_to_list\n

File \"/usr/local/
Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/
json/__init__.py\", line 348, in loads\n    return _default_decoder.decode(s)\n

File \"/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py\", line 337, in decode\n
obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  

File \"/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/json/decoder.py\", line 355, in raw_decode\n
raise JSONDecodeError(\"Expecting value\", s, err.value) from None\njson.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

when 'save_topology' = false, cv_container fails with exception

Issue Type

Bug Report

Module Name

arista.cvp collection and Python libraries version

Traceback (most recent call last):\n  File \"/Users/hugh/.ansible/tmp/ansible-local-16973IX_vgO/ansible-tmp-1578666343.39-221383191659629/AnsiballZ_cv_container.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/Users/hugh/.ansible/tmp/ansible-local-16973IX_vgO/ansible-tmp-1578666343.39-221383191659629/AnsiballZ_cv_container.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/Users/hugh/.ansible/tmp/ansible-local-16973IX_vgO/ansible-tmp-1578666343.39-221383191659629/AnsiballZ_cv_container.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.arista.cvp.plugins.modules.cv_container', init_globals=None, run_name='__main__', alter_sys=False)\n  File \"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 180, in run_module\n    fname, loader, pkg_name)\n  File \"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_RPRn6z/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 1077, in <module>\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_RPRn6z/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 1026, in main\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_RPRn6z/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 865, in attached_configlet_to_container\nTypeError: list indices must be integers, not str\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

OS / Environment

2019.1.1
Summary

when creating or updating a container if the 'save_topology' option is set to 'false' an exception is thrown by the module
Steps to reproduce

---
- name: Playbook to demo cv_container module.
  hosts: CloudVision
  connection: local
  gather_facts: no
  collections:
    - arista.cvp

Vars:
   TOPOLOGY:
      CTL2_TempTest:
         parent_container: Arista_CTL2
         configlets:
            - shared_Test
            - shared_LoginBanner
            - Static_Demo_LocalUserPolicy

- name: collect CVP Facts
  arista.cvp.cv_facts:
  register: CVP_FACTS

- name: 'Create Shared configlets in Containers.'
  arista.cvp.cv_container:
    cvp_facts: "{{CVP_FACTS.ansible_facts}}"
    topology: "{{TOPOLOGY}}"
    save_topology: false
    mode: 'merge'

Expected results

This play should add shared_Test, shared_LoginBanner, Static_Demo_LocalUserPolicy to a container called CTL2_TempTest

Actual results

TASK [cvp.provision : Create Shared configlets in Containers.] **************************************************************************************************************************************************
task path: /Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:80
Friday 10 January 2020  15:09:36 +0000 (0:00:00.189)       0:00:51.359 ******** 
<10.83.30.100> attempting to start connection
<10.83.30.100> using connection plugin httpapi
<10.83.30.102> attempting to start connection
<10.83.30.102> using connection plugin httpapi
<10.83.30.100> found existing local domain socket, using it!
<10.83.30.100> updating play_context for connection
<10.83.30.100> 
<10.83.30.100> local domain socket path is /Users/hugh/.ansible/pc/d6a0aa7b8a
<10.83.30.100> ESTABLISH LOCAL CONNECTION FOR USER: ha
<10.83.30.100> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187 `" && echo ansible-tmp-1578668978.05-150881215866187="` echo /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187 `" ) && sleep 0'
<10.83.30.102> found existing local domain socket, using it!
<10.83.30.102> updating play_context for connection
<10.83.30.102> 
<10.83.30.102> local domain socket path is /Users/hugh/.ansible/pc/0aef998aa9
<10.83.30.102> ESTABLISH LOCAL CONNECTION FOR USER: ha
<10.83.30.102> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943 `" && echo ansible-tmp-1578668978.1-129219225104943="` echo /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943 `" ) && sleep 0'
Using module file /Users/hugh/Desktop/Demo/Innovate_2019/collections/ansible_collections/arista/cvp/plugins/modules/cv_container.py
Using module file /Users/hugh/Desktop/Demo/Innovate_2019/collections/ansible_collections/arista/cvp/plugins/modules/cv_container.py
<10.83.30.100> PUT /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/tmpvJTM8B TO /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187/AnsiballZ_cv_container.py
<10.83.30.102> PUT /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/tmpEk_62Z TO /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943/AnsiballZ_cv_container.py
<10.83.30.100> EXEC /bin/sh -c 'chmod u+x /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187/ /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187/AnsiballZ_cv_container.py && sleep 0'
<10.83.30.102> EXEC /bin/sh -c 'chmod u+x /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943/ /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943/AnsiballZ_cv_container.py && sleep 0'
<10.83.30.102> EXEC /bin/sh -c '$(which python) /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943/AnsiballZ_cv_container.py && sleep 0'
<10.83.30.100> EXEC /bin/sh -c '$(which python) /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187/AnsiballZ_cv_container.py && sleep 0'
<10.83.30.102> EXEC /bin/sh -c 'rm -f -r /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943/ > /dev/null 2>&1 && sleep 0'
fatal: [cvp_server_2]: FAILED! => {
    "changed": false, 
    "module_stderr": "/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\nTraceback (most recent call last):\n  File \"/Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943/AnsiballZ_cv_container.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943/AnsiballZ_cv_container.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.1-129219225104943/AnsiballZ_cv_container.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.arista.cvp.plugins.modules.cv_container', init_globals=None, run_name='__main__', alter_sys=False)\n  File \"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 180, in run_module\n    fname, loader, pkg_name)\n  File \"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_RYPhCh/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 1077, in <module>\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_RYPhCh/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 1026, in main\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_RYPhCh/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 865, in attached_configlet_to_container\nTypeError: list indices must be integers, not str\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", 
    "rc": 1
}
<10.83.30.100> EXEC /bin/sh -c 'rm -f -r /Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187/ > /dev/null 2>&1 && sleep 0'
fatal: [cvp_server_1]: FAILED! => {
    "changed": false, 
    "module_stderr": "/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\n/Users/hugh/Desktop/Demo/Innovate_2019/virtEnv/lib/python2.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning,\nTraceback (most recent call last):\n  File \"/Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187/AnsiballZ_cv_container.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187/AnsiballZ_cv_container.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/Users/hugh/.ansible/tmp/ansible-local-188269lSNK6/ansible-tmp-1578668978.05-150881215866187/AnsiballZ_cv_container.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.arista.cvp.plugins.modules.cv_container', init_globals=None, run_name='__main__', alter_sys=False)\n  File \"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 180, in run_module\n    fname, loader, pkg_name)\n  File \"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_5P0BkC/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 1077, in <module>\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_5P0BkC/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 1026, in main\n  File \"/var/folders/h5/9pv_v6fj4yq1_gswh1mv2ns00000gn/T/ansible_arista.cvp.cv_container_payload_5P0BkC/ansible_arista.cvp.cv_container_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_container.py\", line 865, in attached_configlet_to_container\nTypeError: list indices must be integers, not str\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", 
    "rc": 1
}

PLAY RECAP ******************************************************************************************************************************************************************************************************
cvp_server_1               : ok=8    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   
cvp_server_2               : ok=8    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

Friday 10 January 2020  15:09:40 +0000 (0:00:04.042)       0:00:55.401 ******** 
=============================================================================== 
cvp.provision : collect CVP Facts ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 42.85s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:23 -----------------------------------------------------------------------------------------------------------------------------------
cvp.provision : Create Shared configlets in Containers. -------------------------------------------------------------------------------------------------------------------------------------------------- 4.04s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:80 -----------------------------------------------------------------------------------------------------------------------------------
cvp.provision : Create Device configlets on CVP Servers. ------------------------------------------------------------------------------------------------------------------------------------------------- 3.53s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:38 -----------------------------------------------------------------------------------------------------------------------------------
cvp.provision : Create Shared configlets on CVP Servers. ------------------------------------------------------------------------------------------------------------------------------------------------- 2.63s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:63 -----------------------------------------------------------------------------------------------------------------------------------
cvp.provision : Build configlet Device Associations for CVP ---------------------------------------------------------------------------------------------------------------------------------------------- 1.03s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:3 ------------------------------------------------------------------------------------------------------------------------------------
cvp.provision : Build Shared configlets for CVP ---------------------------------------------------------------------------------------------------------------------------------------------------------- 0.55s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:13 -----------------------------------------------------------------------------------------------------------------------------------
cvp.provision : Load CVP device information -------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.23s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:30 -----------------------------------------------------------------------------------------------------------------------------------
cvp.provision : Load CVP shared Configlet information ---------------------------------------------------------------------------------------------------------------------------------------------------- 0.21s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:55 -----------------------------------------------------------------------------------------------------------------------------------
cvp.provision : Load CVP Container information ----------------------------------------------------------------------------------------------------------------------------------------------------------- 0.19s
/Users/hugh/Desktop/Demo/Innovate_2019/roles/cvp.provision/tasks/main.yml:72 -----------------------------------------------------------------------------------------------------------------------------------

Create makefiles for common tasks like testing, linting

It would be handy to have a makefile that would run tests, lint and execute test playbooks against our lab both for personal dev and ci/cd pipelines.

Refactor how CVP connection details are passed around

We should make use of https://docs.ansible.com/ansible/latest/reference_appendices/special_variables.html#connection-variables
to provider url, username and password to CVP modules.
This will reduce the visual clutter and make playbooks look cleaner.

Disable collecting config in cv_facts

Under certain circumstances, collecting device configuration in cv_facts dramatically increase execution time:

Device configured in provisioning but not accessible.

Gather CVP facts from cvp_foster --- 198.20s

Device not configured in provisioning and accessible:

Gather CVP facts from cvp_foster --- 13.92s

because device configuration is not a must have for CVP provisionning, we might disable collecting configuration for devices or configure a timeout to not increas too much time execution.

Tested with

CVP foster
cv_facts from grant-release

cv_configlet add multiple time task to list.

Issue Type

Bug Report

Module Name

cv_container

Summary

Under certain circumstances, cv_container will append multiple times same taskIds to the list raising issue in cv_task when module tries to execute a task already completed.

Steps to reproduce

---
- name: Playbook to validate and demonstrate cv_container module.
  hosts: cvp
  connection: local
  gather_facts: no
  vars:
    verbose: False
    configlet_list:
          ANSIBLE_TESTING_CONTAINER: "alias a{{ 999 | random }} show version"
          ANSIBLE_TESTING_VEOS: "alias a{{ 999 | random }} show version"
    device_provision:
      veos01:
        name: veos01
        configlets:
          - ANSIBLE_TESTING_VEOS
          - SYS_TelemetryBuilderV2_172.23.0.2_1
          - veos01-basic-configuration
          - SYS_TelemetryBuilderV2
      veos02:
        name: veos02
        configlets:
          - ANSIBLE_TESTING_VEOS
          - SYS_TelemetryBuilderV2_172.23.0.3_1
          - SYS_TelemetryBuilderV2
          - veos02-basic-configuration
      veos03:
        name: veos03
        parentContainerName: Spines
        configlets:
          - SYS_TelemetryBuilderV2_172.23.0.4_1
          - SYS_TelemetryBuilderV2
          - veos03-basic-configuration
    containers_provision:
        Fabric:
          parent_container: Tenant
        Spines:
          parent_container: Fabric
        Leaves:
          parent_container: Fabric
          configlets:
              - ANSIBLE_TESTING_CONTAINER
        MLAG01:
          parent_container: Leaves
          devices:
            - veos01
            - veos02

  tasks:
    - name: '#01 - Collect initial facts from {{inventory_hostname}}'
      cv_facts:
      register: FACTS
    
    - name: '#02 - Configure Configlets on {{inventory_hostname}}'
      cv_configlet:
        cvp_facts: "{{FACTS.ansible_facts}}"
        configlets: "{{configlet_list}}"
        configlet_filter: ["all"]
      register: CONFIGLETS

    - name: '#03 - Configure Container topology on {{inventory_hostname}}'
      cv_container:
        cvp_facts: "{{FACTS.ansible_facts}}"
        topology: '{{containers_provision}}'
        save_topology: true
      register: CONTAINERS

    - name: '#04 - Display Containers Information'
      debug:
        msg: "{{CONTAINERS}}"

    - name: '#05 - Execute pending tasks -- timeout 60sec'
      cv_task:
        tasks: "{{ CONTAINERS.cv_container.tasks }}"
        wait: 60

Expected results

All tasks generated by cv_container to be executed.

Actual results

"taskIds": [
    "244", 
    "245", 
    "244", 
    "245", 
    "246", 
    "246", 
    "244", 
    "245"
]

Update license to Apache

License file should reflect Apache license

In cv_container configlet_attached is updated even if not used

Issue Type

Bug Report

Module Name

arista.cvp collection and Python libraries version

arista.cvp==1.0.2
cv-container

Summary

Steps to reproduce

group_vars

---
CVP_CONTAINERS:
  TEAM01:
    parent_container: Tenant
  TEAM01_DC:
    parent_container: TEAM01
  TEAM01_LEAFS:
    parent_container: TEAM01_DC

Playbook

---
- name: lab04 - cv_container lab
  hosts: CloudVision
  connection: local
  gather_facts: no

  tasks:
    - name: "Gather CVP facts {{inventory_hostname}}"
      arista.cvp.cv_facts:
      register: CVP_FACTS

    - name: "Configure containers on {{inventory_hostname}}"
      arista.cvp.cv_container:
        cvp_facts: "{{CVP_FACTS.ansible_facts}}"
        topology: "{{CVP_CONTAINERS}}"
        save_topology: true
      register: CVP_CONTAINERS_RESULT

    - name: 'Display cv_configlet result'
      debug:
        msg: '{{CVP_CONTAINERS_RESULT}}'

Expected results

"data": {
            "attached_configlet": {
                "configlet_attached": 0, 
                "list": [], 
                "taskIds": []
            }, 
            "changed": true, 
            "creation_result": {
                "containers_created": "2"
            }, 
            "moved_result": {
                "devices_moved": 0, 
                "list": [], 
                "taskIds": []
            }, 
            "taskIds": [], 
            "tasks": []
        }, 
        "failed": false
    }

Actual results

"data": {
            "attached_configlet": {
                "configlet_attached": 3, 
                "list": [
                    [], 
                    [], 
                    []
                ], 
                "taskIds": []
            }, 
            "changed": true, 
            "creation_result": {
                "containers_created": "2"
            }, 
            "moved_result": {
                "devices_moved": 0, 
                "list": [], 
                "taskIds": []
            }, 
            "taskIds": [], 
            "tasks": []
        }, 
        "failed": false
    }

Implementing Pure CVP JSON in CV Facts broke cv_device

Issue Type

Bug Report

Module Name

cv_device

Summary

After using complete JSON reply from CVP, cv_device was not able to run as fields where renamed to hostname instead of name

Steps to reproduce

$ git checkout 67dce893199af5f4e2a3faf8edcd455a4bcde922
$ cd ansible_collections/arista/cvp/tests
$ ansible-playbook playbook.reset.topology.yml --limit cvp_2019

Expected results

changed: [cvp_2019]

Actual results

fatal: [cvp_2019]: FAILED! => {
    "changed": false, 
    "module_stderr": "/Users/tgrimonet/.venv/venv2.7-cvp-ansible/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning)\n/Users/tgrimonet/.venv/venv2.7-cvp-ansible/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning)\nTraceback (most recent call last):\n  File \"/Users/tgrimonet/.ansible/tmp/ansible-local-16979YcJaZM/ansible-tmp-1575718258.87-83850111967733/AnsiballZ_cv_device.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/Users/tgrimonet/.ansible/tmp/ansible-local-16979YcJaZM/ansible-tmp-1575718258.87-83850111967733/AnsiballZ_cv_device.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/Users/tgrimonet/.ansible/tmp/ansible-local-16979YcJaZM/ansible-tmp-1575718258.87-83850111967733/AnsiballZ_cv_device.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.arista.cvp.plugins.modules.cv_device', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 188, in run_module\n    fname, loader, pkg_name)\n  File \"/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n  File \"/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/var/folders/bj/x0pdlg4s46vbk9t8916xkfb00000gn/T/ansible_cv_device_payload_z4yHmy/ansible_cv_device_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_device.py\", line 594, in <module>\n  File \"/var/folders/bj/x0pdlg4s46vbk9t8916xkfb00000gn/T/ansible_cv_device_payload_z4yHmy/ansible_cv_device_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_device.py\", line 584, in main\n  File \"/var/folders/bj/x0pdlg4s46vbk9t8916xkfb00000gn/T/ansible_cv_device_payload_z4yHmy/ansible_cv_device_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_device.py\", line 169, in device_action\n  File \"/var/folders/bj/x0pdlg4s46vbk9t8916xkfb00000gn/T/ansible_cv_device_payload_z4yHmy/ansible_cv_device_payload.zip/ansible_collections/arista/cvp/plugins/modules/cv_device.py\", line 169, in <genexpr>\nKeyError: 'name'\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", 
    "rc": 1
}

cv_configlet - module fails if list of configlet is empty

Should support an empty list for CVP cleanup like for cv_container with Issue #14

---
- name: Test cv_configlet_v2
  hosts: cvp
  connection: local
  gather_facts: no
  vars:
    configlets_delete:
  tasks:
    - name: 'Use Empty configlet list to deploy on CVP {{inventory_hostname}}.'
      tags:
        - cleanup
      cv_configlet:
        host: "{{ansible_host}}"
        username: '{{cvp_username}}'
        password: '{{cvp_password}}'
        protocol: https
        port: '{{cvp_port}}'
        cvp_facts: "{{cvp_facts.ansible_facts}}"
        configlets: "{{configlets_delete}}"
        # cvp_filter: ["Test","base-chk","base-firewall"]
      register: cvp_configlet

Enhancement - Move ansible/cvp under ansible_collections

Issue Type

Feature Idea

Summary

Creating a top folder named ansible_collections would be helpful to develop and test on the fly without building collection for every run.

Using this path, ansible.cfg file within Examples should look like something like:

collections_paths=$PWD/../

Indeed, collections folder shall be follow this structure: custom_collection_dir/ansible_collections/namespace/collection

This approach will also allow CI to run test directly instead of building correct tree directory.

This issue requires to relocate all files impacted in open PRs

Allow cv_device to add configlet without listing all configlets

Use case:

When devices are all deployed with their configilet, it might be useful to add a new configlet to device using cv_device module.

Current module implementation requires to list all configlet and add new configlet to the list. This approach means user has to maintain list of configlets per device and run a workflow close to the rollout targeting a larger scope than just 1 device + 1 configlet

Change behaviour:

To allow user to target only one device with one configlet, it can be useful to add an option related to configlets in cv_device to support addition or deletion of a set of configlets.

Current implementation use the following syntax:

vars:
  devices_inventory:
    veos01:
    name: veos01
    configlets:
        - cv_device_test01
        - SYS_TelemetryBuilderV2_172.23.0.2_1
        - veos01-basic-configuration
        - SYS_TelemetryBuilderV2
    parentContainerName: DC1_VEOS
tasks:
    - name: "Configure devices on {{inventory_hostname}}"
      cv_device:
        devices: "{{devices_inventory}}"
        cvp_facts: '{{cvp_facts.ansible_facts}}'
        device_filter: ['veos']
      register: cvp_device

To support such change, 2 options could be implemented and both are already present in other modules

Use a configlet filter:

With this approach, configlet filter allows user to target only configlets with a specific string. So as for cv_container it is easy to add only one configlet with following example:

vars:
  devices_inventory:
    veos01:
    name: veos01
    configlets:
        - veos01-basic-configuration
    parentContainerName: DC1_VEOS
tasks:
    - name: "Configure devices on {{inventory_hostname}}"
      cv_device:
        devices: "{{devices_inventory}}"
        cvp_facts: '{{cvp_facts.ansible_facts}}'
        device_filter: ['veos']
        configlet_filter: ['veos01-basic-configuration']
      register: cvp_device

But in case we want to remove it from this device, it is required to fallback to original definition with all configlets.

This approach is similar to filters implemented in cv_configlet

Use a definition of mode

With implemnentation of a mode keyword, we can define 3 different behaviours:
- override (default): Completely intend based approach and will apply the full list of configlet to device and remove unlisted configlets.
- merge: Only add new configlets to existing list.
- delete: Delete list of configlets from device.

vars:
  devices_inventory:
    veos01:
    name: veos01
    configlets:
        - veos01-basic-configuration
    parentContainerName: DC1_VEOS
tasks:
    - name: "Configure devices on {{inventory_hostname}}"
      cv_device:
        devices: "{{devices_inventory}}"
        cvp_facts: '{{cvp_facts.ansible_facts}}'
        device_filter: ['veos']
        configlet_mode: merge
      register: cvp_device

This approach is similar to cv_container with its mode option

cv_configlet - Creation of configlet on container raises error

When creating a configlet on CVP, ansible raises an error even if configlet is correctly rendered and created on server:

Task:

    - name: Create a configlet on CVP.
      tags: 
        - configlet
        - provision
      cv_configlet:
        host: '{{ansible_host}}'
        username: '{{cvp_username}}'
        password: '{{cvp_password}}'
        protocol: https
        container: "vLeaves"
        parent: "Ansible_Fabric"
        configletName: "ansible-fabric-vlans"
        template: "{{configlet_template}}"
        data: "{{configlet_data}}"
        action: add

Error message:

Following line cv_configlet.py/L297 raise an error:

ansible_cv_configlet_payload_IxU9kQ/main.py", line 297, in
process_configlet\nUnboundLocalError: local variable 'device_data'
referenced before assignment

Ansible version: 2.8.3

ansible 2.8.3
  config file = /Users/tgrimonet/Projects/ansible-devel/github/titom73/ansible-cvp-old-private/tests/ansible.cfg
  configured module search path = [u'/Users/tgrimonet/Projects/ansible-devel/github/titom73/ansible-cvp-old-private/library', u'/Users/tgrimonet/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/tgrimonet/Projects/ansible-devel/github/titom73/ansible-cvp-old-private/.venv/lib/python2.7/site-packages/ansible
  executable location = /Users/tgrimonet/Projects/ansible-devel/github/titom73/ansible-cvp-old-private/.venv/bin/ansible
  python version = 2.7.10 (default, Feb 22 2019, 21:55:15) [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)]

[BUG] - cv_device fail when multiple device in undefined

Issue Type

Bug Report (initially reported by @b-abadie)

Module Name

cv_device

Reported in a specific configuration using module and ansible 2.8

OS / Environment

N/A

Summary

cv_device fails when trying to provision multiple devices from undefined container

Assumptions:

All devices are in pure ZTP mode
All devices are still part of undefined container

Workflow

[Per switch]: create configlets on CVP
[Run_once]: create topology using cv_container to build container topology and assign devices to containers
[Run_once]: assign configlet to devices using cv_device
[Run_once]: execute all pending tasks with cv_task

Within cv_device when trying to assign configlet to a device, API returns error message:

Exception: provsion_device-update_configlets:update_configlets_on_device:\
POST: https://192.168.122.2:443/web/provisioning/addTempAction.do?format=topology&queryParam=&nodeId=root\
 : Request Error: Cannot assign configlet, since device is under undefined container"

cv_device call a /provisioning/saveTopology.do each time it assign a configlet to a device. This behavior is related to create_task=True which is the default and not override by device_action in cv_device.py

new_device_action = module.client.api.provision_device('Ansible',device['cvp_device'],
                                                       device['container'],
                                                       add_configlets,
                                                       add_imageBundle
)

aristanetworks / ansible-cvp Goto Github PK

ansible-cvp's People

Contributors

Stargazers

Watchers

Forkers

ansible-cvp's Issues

Issue Type

OS / Environment

Summary

Steps to reproduce

Actual results

Issue Type

Module Name

OS / Environment

Summary

Issue Type

Module Name

OS / Environment

Summary

Steps to reproduce

Actual results

Implement mode for cv_container

cv_container and device deletion

cv_device and configlet deletion

Implement filter option

Issue Type

Module Name

Description

Issue Type

Summary

Steps to reproduce

Issue Type

Module Name

Versions Info

Summary

Actual results

Issue Type

cv_configlet

Issue Type

Module Name

Summary

Issue Type

Module Name

OS / Environment

2019.1.1 Summary

when creating or updating a container if the 'save_topology' option is set to 'false' an exception is thrown by the module Steps to reproduce

Actual results

Issue Type

Module Name

Summary

Steps to reproduce

Actual results

Issue Type

Module Name

Summary

Steps to reproduce

Actual results

Issue Type

Module Name

Summary

Steps to reproduce

Expected results

Actual results

Issue Type

Summary

Issue Type

Module Name

OS / Environment

Summary

Recommend Projects

Recommend Topics

Recommend Org

Implement mode for `cv_container`

2019.1.1
Summary

when creating or updating a container if the 'save_topology' option is set to 'false' an exception is thrown by the module
Steps to reproduce