Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[xcvrd] xcvrd crashes due to PR https://github.com/sonic-net/sonic-platform-common/pull/397 #18468

Closed
Junchao-Mellanox opened this issue Mar 26, 2024 · 1 comment · Fixed by sonic-net/sonic-platform-daemons#458
Assignees

Comments

@Junchao-Mellanox
Copy link
Collaborator

Description

xcvrd crashes due to PR sonic-net/sonic-platform-common#397.

In this PR, it added a new line:

self.vdm_dict = self.get_vdm(self.vdm.VDM_REAL_VALUE)

However, self.vdm might be None due to this line: https://github.com/sonic-net/sonic-platform-common/blob/2606cd5fb0a58fc956577602235a9f8f84a887c6/sonic_platform_base/sonic_xcvr/api/public/cmis.py#L31

So, if the CMIS cable is flat memory, the call to "self.vdm.VDM_REAL_VALUE" always crashes:

Mar 25 15:31:40.777905 sonic ERR pmon#xcvrd: Exception occured at DomInfoUpdateTask thread due to AttributeError("'NoneType' object has no attribute 'VDM_REAL_VALUE'")
Mar 25 15:31:40.779557 sonic ERR pmon#xcvrd: Traceback (most recent call last):
Mar 25 15:31:40.779616 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/xcvrd/xcvrd.py", line 1744, in run
Mar 25 15:31:40.779616 sonic ERR pmon#xcvrd:     self.task_worker()
Mar 25 15:31:40.779616 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/xcvrd/xcvrd.py", line 1716, in task_worker
Mar 25 15:31:40.779633 sonic ERR pmon#xcvrd:     post_port_dom_info_to_db(logical_port_name, self.port_mapping, self.xcvr_table_helper.get_dom_tbl(asic_index), self.task_stopping_event, dom_info_cache=dom_info_cache)
Mar 25 15:31:40.779644 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/xcvrd/xcvrd.py", line 627, in post_port_dom_info_to_db
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:     dom_info_dict = _wrapper_get_transceiver_dom_info(physical_port)
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/xcvrd/xcvrd.py", line 279, in _wrapper_get_transceiver_dom_info
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:     return platform_chassis.get_sfp(physical_port).get_transceiver_bulk_status()
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/sonic_platform_base/sonic_xcvr/sfp_optoe_base.py", line 32, in get_transceiver_bulk_status
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:     return api.get_transceiver_bulk_status() if api is not None else None
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/sonic_platform_base/sonic_xcvr/api/public/cmis.py", line 233, in get_transceiver_bulk_status
Mar 25 15:31:40.779816 sonic ERR pmon#xcvrd:     self.vdm_dict = self.get_vdm(self.vdm.VDM_REAL_VALUE)
Mar 25 15:31:40.779816 sonic ERR pmon#xcvrd: AttributeError: 'NoneType' object has no attribute 'VDM_REAL_VALUE'
Mar 25 15:31:40.779816 sonic ERR pmon#xcvrd: Xcvrd: exception found at child thread DomInfoUpdateTask due to AttributeError("'NoneType' object has no attribute 'VDM_REAL_VALUE'")
Mar 25 15:31:40.779827 sonic ERR pmon#xcvrd: Exiting main loop as child thread raised exception!

Steps to reproduce the issue:

  1. Insert a CMIS cable which has flat memory
  2. Start sonic
  3. Check syslog, you will see the crash in short

Describe the results you received:

Mar 25 15:31:40.777905 sonic ERR pmon#xcvrd: Exception occured at DomInfoUpdateTask thread due to AttributeError("'NoneType' object has no attribute 'VDM_REAL_VALUE'")
Mar 25 15:31:40.779557 sonic ERR pmon#xcvrd: Traceback (most recent call last):
Mar 25 15:31:40.779616 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/xcvrd/xcvrd.py", line 1744, in run
Mar 25 15:31:40.779616 sonic ERR pmon#xcvrd:     self.task_worker()
Mar 25 15:31:40.779616 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/xcvrd/xcvrd.py", line 1716, in task_worker
Mar 25 15:31:40.779633 sonic ERR pmon#xcvrd:     post_port_dom_info_to_db(logical_port_name, self.port_mapping, self.xcvr_table_helper.get_dom_tbl(asic_index), self.task_stopping_event, dom_info_cache=dom_info_cache)
Mar 25 15:31:40.779644 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/xcvrd/xcvrd.py", line 627, in post_port_dom_info_to_db
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:     dom_info_dict = _wrapper_get_transceiver_dom_info(physical_port)
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/xcvrd/xcvrd.py", line 279, in _wrapper_get_transceiver_dom_info
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:     return platform_chassis.get_sfp(physical_port).get_transceiver_bulk_status()
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/sonic_platform_base/sonic_xcvr/sfp_optoe_base.py", line 32, in get_transceiver_bulk_status
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:     return api.get_transceiver_bulk_status() if api is not None else None
Mar 25 15:31:40.779784 sonic ERR pmon#xcvrd:   File "/usr/local/lib/python3.9/dist-packages/sonic_platform_base/sonic_xcvr/api/public/cmis.py", line 233, in get_transceiver_bulk_status
Mar 25 15:31:40.779816 sonic ERR pmon#xcvrd:     self.vdm_dict = self.get_vdm(self.vdm.VDM_REAL_VALUE)
Mar 25 15:31:40.779816 sonic ERR pmon#xcvrd: AttributeError: 'NoneType' object has no attribute 'VDM_REAL_VALUE'
Mar 25 15:31:40.779816 sonic ERR pmon#xcvrd: Xcvrd: exception found at child thread DomInfoUpdateTask due to AttributeError("'NoneType' object has no attribute 'VDM_REAL_VALUE'")
Mar 25 15:31:40.779827 sonic ERR pmon#xcvrd: Exiting main loop as child thread raised exception!

Describe the results you expected:

No crashes

Output of show version:

SONiC-OS-master.505826-568149081

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants