aoscx_upload_firmware 401 Authorization Required Aruba CX 6200f / 6300M

[mrfischl]

Hey Guys,

I've been using Ansible for a few weeks and have written a playbook for updating Aruba CX switches. Unfortunately, I have problems with the firmware upload on the CX6200F and 6300M.

The Problem:
fatal: [aoscx_1]: FAILED! => {"changed": false, "msg": "'GENERIC OPERATION ERROR: 401 Authorization Required: Code: 401'"}

I forgot the "-vvv" but i can post it later if required.

My test-setup:

Versions:
Ansible: ansible [core 2.14.6]
Python3: 3.10.6
Aoscx: arubanetworks.aoscx 4.2.1
Ubuntu: 22.04.2 LTS
Aruba CX 6200f: ML.10.10.1040

Inventory
all:
hosts:
aoscx_1:
ansible_host: 192.168.178.253
ansible_user:
ansible_password:
ansible_network_os: arubanetworks.aoscx.aoscx
ansible_connection: arubanetworks.aoscx.aoscx
ansible_aoscx_validate_certs: False
ansible_aoscx_use_proxy: False
ansible_acx_no_proxy: True

Playbook

• hosts: all
  collections:
  • arubanetworks.aoscx
    gather_facts: False
    tasks:
  • name: Upload
    aoscx_upload_firmware:
    partition_name: 'primary'
    firmware_file_path: '/root/ansible/firmware/ArubaOS-CX_6200_10_10_1010.swi'

Switch:
Current configuration:
!
!Version ArubaOS-CX ML.10.10.1040
!export-password: default
user admin group administrators password ciphertext
ntp server pool.ntp.org minpoll 4 maxpoll 4 iburst
ntp enable
!
ssh server vrf default
ssh server vrf mgmt
vsf member 1
type jl725a
vlan 1
spanning-tree
interface mgmt
no shutdown
ip dhcp
interface 1/1/1-1/1/28
no shutdown
no routing
vlan access 1
interface vlan 1
ip address 192.168.178.253/24
ip dhcp
! ip dhcp is ignored when static ip is configured
!
!
!
!
!
https-server vrf default
https-server vrf mgmt

https-server rest access-mode read-write \configured but not displayed in "show run"?!

---

Other tests:

aoscx_vlan - works fine
Aruba CX 6100 - Firmware upload no problems

Wireshark - in the pcap i can see that the file transfer is running

Hope you can help me.

Greetings Lukas

[tchiapuziowong]

Are you upgrading the 6200 from 10_10_1010 to 10_10_1040? Can you provide the full error? Does this error occur immediately or after some time?

Can you verify if you've installed all the required libraries outlined here?
Instructions to install requirements file installed by ansible-galaxy:
cd "$(ansible-galaxy collection list | grep -E '^#.*\.ansible' | sed 's/\# //')/arubanetworks/aoscx"
pip install -r requirements.txt --force-reinstall

[mrfischl]

1. The switch is running version 10_10_1040.
2. see below
3. After 1 min and a few seconds
4. All requirements are installed

Error:
The full traceback is:
File "/tmp/ansible_aoscx_upload_firmware_payload_x25so1fj/ansible_aoscx_upload_firmware_pa/ansible_collections/arubanetworks/aoscx/plugins/modules/aoscx_upload_firmware.py", line 167
File "/usr/local/lib/python3.10/dist-packages/pyaoscx/device.py", line 565, in upload_firm
result = self.upload_firmware_local(
File "/usr/local/lib/python3.10/dist-packages/pyaoscx/device.py", line 518, in upload_firml
success = utils.file_upload(
File "/usr/local/lib/python3.10/dist-packages/pyaoscx/utils/util.py", line 309, in file_up
raise GenericOperationError(
fatal: [aoscx_1]: FAILED! => {
"changed": false,
"invocation": {
"module_args": {
"firmware_file_path": "/root/ansible/firmware/ArubaOS-CX_6200_10_10_1010.swi",
"partition_name": "primary",
"remote_firmware_file_path": null,
"vrf": null,
"wait_firmware_upload": false
}
},
"msg": "'GENERIC OPERATION ERROR: 401 Authorization Required: Code: 401'"
}

[tchiapuziowong]

@mrfischl can you move the firmware to a different location other than the /root/ folder? perhaps the home or Desktop of the user that's running the playbook? I'll see if I can replicate this in my environment.

[mrfischl]

@tchiapuziowong In our productive system is the location /home/user/ansible/firmware/
Same error.

[tchiapuziowong]

Can you provide the versions you have installed for the following Python libraries?

paramiko
requests
ansible-pylibssh
pyaoscx
pycurl

[JSTBotanic]

Hi @mrfischl
Can you send me your code for upload firmware for Aruba AOSCX 6100 please ?
I have some errors and i don't understand.

[mrfischl]

@tchiapuziowong
paramiko 3.2.0 requests 2.31.0 ansible-pylibssh 1.1.0 pyaoscx 2.4.1 pycurl 7.45.2

[mrfischl]

@JSTBotanic Hi, check out "Getting Started with Ansible and AOS-CX" there is a example. You just need to change the module.
Module example: https://github.com/aruba/aoscx-ansible-role/blob/master/docs/aoscx_upload_firmware.md

[smirlach]

Hi @tchiapuziowong @JSTBotanic @mrfischl,
i have basically the same issue for some devices with the aoscx_facts module.

Switch Info:

Hostname           : <host>                    
System Description : FL.10.10.1050                 
System Contact     :                               
System Location    : <location>      

Vendor             : Aruba                         
Product Name       : JL659A 6300M 48SR5 CL6 PoE 4SFP56 Swch
Chassis Serial Nbr : <chassis-serial>                    
Base MAC Address   : <mac-address>                 
ArubaOS-CX Version : FL.10.10.1050

Collection version:

❯ ansible-galaxy collection list arubanetworks.aoscx

# /Users/<user>/.ansible/collections/ansible_collections
Collection          Version
------------------- -------
arubanetworks.aoscx 4.2.1  

I also tried to gather different facts which doesn't seem to work.
Ansible task snippets:

  - name: Gather facts
    arubanetworks.aoscx.aoscx_facts:
      gather_network_resources:
        - vlans
        - interfaces
      gather_subset:
        - config
    vars:
      ansible_command_timeout: 60

  - name: Gather facts
    arubanetworks.aoscx.aoscx_facts:
    vars:
      ansible_command_timeout: 60

If i don't specify ansible_command_timeout i get a timeout error.
I verified the credentials i'm using on the same devices with direct API calls and also with PYAOSCX. Both methods seem to work fine, only via Ansible i get msg: 'Network resources: ''GENERIC OPERATION ERROR: 401 Authorization Required: Code: 401'''.

The error seems to originate somewhere when calling session.api.get_module_class.

aoscx-ansible-collection/plugins/modules/aoscx_facts.py

Lines 250 to 252 in de5c759

	Interface = session.api.get_module_class(
	session, "Interface"
	)

To test that i put some debug output before and after that call. The output after that is never reached.

[tchiapuziowong]

Thank you @smirlach and @mrfischl for the information! I'm working on reproducing the issue in our setup and will bring this up internally - I may be asking for more information so I appreciate your patience and cooperation!

@smirlach can you provide the output of ansible --version?

[smirlach]

@tchiapuziowong There you go:

ansible [core 2.13.9]
  config file = None
  configured module search path = ['/Users/<user>/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/<user>/.pyenv/versions/3.9.6/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/<user>/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/<user>/.pyenv/versions/3.9.6/bin/ansible
  python version = 3.9.6 (default, May 16 2023, 16:15:39) [Clang 14.0.3 (clang-1403.0.22.14.1)]
  jinja version = 3.1.2
  libyaml = True

I think we found the issue on our side. The problem seems to be that we have a dual stack environment (IPv4/IPv6) and our firewall blocks IPv6 to the switch per default as it is in another environment as the other switches i've tested previously with. If i enter the IPv4 of the switch in the Ansible inventory directly it works without any issues.
It seems to me like aoscx_facts/pyaoscx or even the requests python package used in there is trying IPv6 and then is not falling back to IPv4 if IPv6 is not working.
Maybe you can check that and give us some feedback on that.

[tchiapuziowong]

@smirlach are defining two addresses for the ansible_host? If you're defining an IPv6 address please make sure it's in this format : '[2001::1]'

[smirlach]

@tchiapuziowong i have defined the ansible_host as a FQDN like switch1.net. The lookup for that then delivers an IPv4 and IPv6 address.

[tchiapuziowong]

@smirlach @mrfischl I'm unable to reproduce the error you're receiving - the IP address you're connecting to the switch, is that on the management VRF/interface or through the front ports?

[mrfischl]

@tchiapuziowong
The Interface-vlan 1 vrf-default is connected to the Front Ports in my test.

[mrfischl]

@tchiapuziowong
Can you maybe share a example for a 6200F Switch (ansible.cfg, playbook,, inventory)?
My playbook is still running for a 6100 Switch.

[tchiapuziowong]

@mrfischl the inventory/playbook would be the same for any model of switch - this may be an issue seen only when using a non-management VRF to connect to the switch

[mrfischl]

@tchiapuziowong
THX, I tested it! Too bad it doesn't work with the front ports.
In our case, we used hundreds of switches without mgmt-port.

[tchiapuziowong]

@mrfischl reopening - our internal team is still investigating why it's failing non-mgmt, I'll update this issue once we have more information regarding the behavior

[jonathanbaird]

i am also having this issue with a 6100 series switch. there's no option to use a mgmt vrf in my instance so i am forced to use the default vrf

[tchiapuziowong]

Hi @jonathanbaird - we're evaluating the fix for an upcoming minor release and will update the issue once it's implemented, thank you for your continued patience during this time. As a workaround you could utilize the SSH module aoscx_command to upload a firmware using a server - for a full example see our workflows repository here: https://github.com/aruba/aoscx-ansible-workflows/blob/main/copy_config_using_cli_prompt.yml

[mrfischl]

@tchiapuziowong Any updates?

[tchiapuziowong]

We've just published a fix for this issue, please install the latest version of the collection and verify if you no longer see this issue:

ansible-galaxy collection install arubanetworks.aoscx -f

[2relativ]

Hey, I had the same error message but I was uploading the firmware through http instead of using local resources.
What I did different to get around this was using the switch wait_firmware_upload: True which I found while using debug mode.
After that Ansible did indeed wait until the upload was done and confirmed instead of just triggern the update and going on. This should maybe also be used in the examples?! Both on github and on the aruba page or wait_firmware_upload should be enabled by default?

[tchiapuziowong]

@2relativ if you install the latest version of the collection are you still running into the same issue without explicitly defining wait_firmware_upload?

[2relativ]

@2relativ if you install the latest version of the collection are you still running into the same issue without explicitly defining wait_firmware_upload?

Yes. Definitely. I tried first a workaround until I saw this option in the debug log. The interesting part is, that the switch itself wouldn't let me in over webinterface also. So, I would definitely set the wait_firmware_upload as default. But please be careful since my upload is over http from a different server. So the ansible server has to explicitly request if the server is ready instead of maybe monitoring the firmware upload from himself.

[alagoutte]

Hey, I had the same error message but I was uploading the firmware through http instead of using local resources. What I did different to get around this was using the switch wait_firmware_upload: True which I found while using debug mode. After that Ansible did indeed wait until the upload was done and confirmed instead of just triggern the update and going on. This should maybe also be used in the examples?! Both on github and on the aruba page or wait_firmware_upload should be enabled by default?

i have already open a PR #98 for add missing wait_firmware_upload, I has the same issue when use http/https