Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test commit to run tests in upstream #556

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Test commit to run tests in upstream #556

wants to merge 2 commits into from

Conversation

06shalini
Copy link
Contributor

Signed-off-by: Shalini Khandelwal [email protected]

@06shalini 06shalini marked this pull request as draft September 16, 2020 10:06
@06shalini 06shalini force-pushed the upstream-run-process branch 5 times, most recently from 45078e1 to ed29b78 Compare September 17, 2020 13:31
Shalini Khandelwal and others added 2 commits September 21, 2020 14:09
@SilleBille
Test commmit to run tests in upstream
6515019 
Signed-off-by: Shalini Khandelwal <[email protected]>
@SilleBille
CI: Add QE pytests to upstream CI pipeline
d62e59f 
This patch executes pytests and runs CA related tests
on the VMs provisioned.

Signed-off-by: Dinesh Prasanth M K <[email protected]>
@SilleBille
Copy link
Member

I have fixed your code and refactored to run the tests as a separate step. There seems to occur assertion errors. Not sure how to fix them

2020-09-21T20:57:49.4440870Z =================================== FAILURES ===================================
2020-09-21T20:57:49.4441890Z __________________________________ test_setup __________________________________
2020-09-21T20:57:49.4442220Z 
2020-09-21T20:57:49.4443330Z ansible_module = <pytest_ansible.module_dispatcher.v28.ModuleDispatcherV28 object at 0x7f6686bfff70>
2020-09-21T20:57:49.4444360Z 
2020-09-21T20:57:49.4444840Z     @pytest.mark.setup
2020-09-21T20:57:49.4445420Z     def test_setup(ansible_module):
2020-09-21T20:57:49.4445880Z         """
2020-09-21T20:57:49.4447630Z         Prerequisites for running pytest-ansible tests
2020-09-21T20:57:49.4448280Z         """
2020-09-21T20:57:49.4448870Z         cert_setup = CertSetup(nssdb=constants.NSSDB,
2020-09-21T20:57:49.4449720Z                                db_pass=constants.CLIENT_DATABASE_PASSWORD,
2020-09-21T20:57:49.4450550Z                                host=constants.MASTER_HOSTNAME,
2020-09-21T20:57:49.4451240Z                                port=constants.CA_HTTP_PORT,
2020-09-21T20:57:49.4452630Z                                nick="'{}'".format(constants.CA_ADMIN_NICK))
2020-09-21T20:57:49.4453410Z         cert_setup.create_certdb(ansible_module)
2020-09-21T20:57:49.4454160Z         cert_setup.import_ca_cert(ansible_module)
2020-09-21T20:57:49.4455580Z         cert_setup.import_admin_p12(ansible_module, 'ca')
2020-09-21T20:57:49.4457100Z >       cert_setup.setup_role_users(ansible_module, 'ca', duration='minute')
2020-09-21T20:57:49.4457670Z 
2020-09-21T20:57:49.4459000Z /vagrant/tests/dogtag/pytest-ansible/pytest/sanity/test_role_users.py:54: 
2020-09-21T20:57:49.4459720Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2020-09-21T20:57:49.4461210Z /vagrant/tests/dogtag/pytest-ansible/pki/testlib/common/certlib.py:134: in setup_role_users
2020-09-21T20:57:49.4462250Z     self.add_expired_profile_to_ca(ansible_module, duration)
2020-09-21T20:57:49.4464030Z /vagrant/tests/dogtag/pytest-ansible/pki/testlib/common/certlib.py:226: in add_expired_profile_to_ca
2020-09-21T20:57:49.4465060Z     output_list = p_obj.create_profile(profile_params)
2020-09-21T20:57:49.4466250Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2020-09-21T20:57:49.4466440Z 
2020-09-21T20:57:49.4466920Z self = <pki.testlib.common.profile.Setup object at 0x7f668603e370>
2020-09-21T20:57:49.4468340Z user_params = {'MaxValidity': '15', 'ProfileName': 'caAgentFoobar Enrollment Profile', 'ValidFor': '1', 'notAfter': '2', ...}
2020-09-21T20:57:49.4468820Z 
2020-09-21T20:57:49.4469100Z     def create_profile(self, user_params=None):
2020-09-21T20:57:49.4469700Z         '''
2020-09-21T20:57:49.4470060Z         Create a custom profile based on user provided parameters
2020-09-21T20:57:49.4470400Z     
2020-09-21T20:57:49.4470750Z         :params dict user_params: Dictionary containing user provided
2020-09-21T20:57:49.4471190Z             values to policies, extensions
2020-09-21T20:57:49.4471770Z         '''
2020-09-21T20:57:49.4472140Z         profile_input = self.generate_profile_parameters(user_params)
2020-09-21T20:57:49.4472630Z         # Create initial profile name, description
2020-09-21T20:57:49.4473040Z         root_element = self.add_description(
2020-09-21T20:57:49.4474110Z             profile_input['ProfileId'],
2020-09-21T20:57:49.4474820Z             profile_input['name'],
2020-09-21T20:57:49.4475460Z             profile_input['Description']
2020-09-21T20:57:49.4475740Z         )
2020-09-21T20:57:49.4476020Z         # Add Key Generation input class to the xml
2020-09-21T20:57:49.4476380Z         ret = self.add_key_gen(root_element,
2020-09-21T20:57:49.4477100Z                                profile_input['Key_Generation_Class'],
2020-09-21T20:57:49.4477840Z                                profile_input['Key_Generate_InputId'])
2020-09-21T20:57:49.4478140Z     
2020-09-21T20:57:49.4478460Z         # Add subject name input and submitter information
2020-09-21T20:57:49.4479240Z >       if profile_input.has_key('subject_Name_Input_Id'):
2020-09-21T20:57:49.4480050Z E       AttributeError: 'dict' object has no attribute 'has_key'
2020-09-21T20:57:49.4480570Z 
2020-09-21T20:57:49.4481430Z /vagrant/tests/dogtag/pytest-ansible/pki/testlib/common/profile.py:1616: AttributeError
2020-09-21T20:57:49.4482310Z ------------------------------ Captured log setup ------------------------------
2020-09-21T20:57:49.4482980Z INFO     Test:__init__.py:90 ------------------------------------------------------
2020-09-21T20:57:49.4483910Z INFO     Test:__init__.py:91 Func test_setup in file: /vagrant/tests/dogtag/pytest-ansible/pytest/sanity/test_role_users.py
2020-09-21T20:57:49.4484910Z ______________ test_ca_audit_with_role_users[CA_AdminV-expected0] ______________
2020-09-21T20:57:49.4485210Z 
2020-09-21T20:57:49.4485830Z ansible_module = <pytest_ansible.module_dispatcher.v28.ModuleDispatcherV28 object at 0x7f66869f3d00>
2020-09-21T20:57:49.4486810Z certnick = 'CA_AdminV'
2020-09-21T20:57:49.4487750Z expected = ['Status: Enabled', 'Signed: true', 'Interval (seconds): 5', 'Buffer size (bytes): 512']
2020-09-21T20:57:49.4488120Z 
2020-09-21T20:57:49.4488530Z     @pytest.mark.parametrize("certnick,expected", [
2020-09-21T20:57:49.4489410Z         ("CA_AdminV", ['Status: Enabled', 'Signed: true',
2020-09-21T20:57:49.4490090Z                        'Interval (seconds): 5',
2020-09-21T20:57:49.4490720Z                        'Buffer size (bytes): 512']),
2020-09-21T20:57:49.4491640Z         (pytest.param("CA_AdminE", ['FATAL: SSL alert received: CERTIFICATE_EXPIRED'], marks=pytest.mark.xfail)),
2020-09-21T20:57:49.4492610Z         ("CA_AdminR", ['PKIException: Unauthorized'])
2020-09-21T20:57:49.4492920Z     ])
2020-09-21T20:57:49.4493280Z     def test_ca_audit_with_role_users(ansible_module, certnick, expected):
2020-09-21T20:57:49.4493640Z         """
2020-09-21T20:57:49.4494280Z         Test and verify pki ca-audit-show with CA_AdminV
2020-09-21T20:57:49.4494770Z         shows the audit configuration, with CA_AdminE and CA_AdminR
2020-09-21T20:57:49.4495260Z         verify that CLI does not show audit configuration
2020-09-21T20:57:49.4495640Z         :param ansible_module:
2020-09-21T20:57:49.4495920Z         :param certnick:
2020-09-21T20:57:49.4496340Z         :param expected:
2020-09-21T20:57:49.4496580Z         :return:
2020-09-21T20:57:49.4496770Z         """
2020-09-21T20:57:49.4497480Z         contacted = ansible_module.pki(cli='ca-audit-show',
2020-09-21T20:57:49.4497930Z                                        nssdb=constants.NSSDB,
2020-09-21T20:57:49.4498390Z                                        dbpassword=constants.CLIENT_DATABASE_PASSWORD,
2020-09-21T20:57:49.4499140Z                                        protocol='https',
2020-09-21T20:57:49.4499460Z                                        certnick=certnick)
2020-09-21T20:57:49.4499810Z         for result in contacted.values():
2020-09-21T20:57:49.4500150Z             for iter in expected:
2020-09-21T20:57:49.4500430Z                 if certnick == "CA_AdminV":
2020-09-21T20:57:49.4501080Z >                   assert iter in result['stdout']
2020-09-21T20:57:49.4501840Z E                   AssertionError: assert 'Status: Enabled' in ''
2020-09-21T20:57:49.4502140Z 
2020-09-21T20:57:49.4503250Z /vagrant/tests/dogtag/pytest-ansible/pytest/sanity/test_role_users.py:82: AssertionError
2020-09-21T20:57:49.4504190Z ------------------------------ Captured log setup ------------------------------
2020-09-21T20:57:49.4504860Z INFO     Test:__init__.py:90 ------------------------------------------------------
2020-09-21T20:57:49.4505950Z INFO     Test:__init__.py:91 Func test_ca_audit_with_role_users[CA_AdminV-expected0] in file: /vagrant/tests/dogtag/pytest-ansible/pytest/sanity/test_role_users.py
2020-09-21T20:57:49.4507230Z ______________ test_ca_audit_with_role_users[CA_AdminR-expected2] ______________
2020-09-21T20:57:49.4507690Z 
2020-09-21T20:57:49.4508350Z ansible_module = <pytest_ansible.module_dispatcher.v28.ModuleDispatcherV28 object at 0x7f66860ba2e0>
2020-09-21T20:57:49.4509550Z certnick = 'CA_AdminR', expected = ['PKIException: Unauthorized']
2020-09-21T20:57:49.4510080Z 
2020-09-21T20:57:49.4510490Z     @pytest.mark.parametrize("certnick,expected", [
2020-09-21T20:57:49.4511370Z         ("CA_AdminV", ['Status: Enabled', 'Signed: true',
2020-09-21T20:57:49.4512050Z                        'Interval (seconds): 5',
2020-09-21T20:57:49.4512680Z                        'Buffer size (bytes): 512']),
2020-09-21T20:57:49.4513600Z         (pytest.param("CA_AdminE", ['FATAL: SSL alert received: CERTIFICATE_EXPIRED'], marks=pytest.mark.xfail)),
2020-09-21T20:57:49.4514560Z         ("CA_AdminR", ['PKIException: Unauthorized'])
2020-09-21T20:57:49.4514890Z     ])
2020-09-21T20:57:49.4515250Z     def test_ca_audit_with_role_users(ansible_module, certnick, expected):
2020-09-21T20:57:49.4515600Z         """
2020-09-21T20:57:49.4516260Z         Test and verify pki ca-audit-show with CA_AdminV
2020-09-21T20:57:49.4516740Z         shows the audit configuration, with CA_AdminE and CA_AdminR
2020-09-21T20:57:49.4517230Z         verify that CLI does not show audit configuration
2020-09-21T20:57:49.4517630Z         :param ansible_module:
2020-09-21T20:57:49.4517990Z         :param certnick:
2020-09-21T20:57:49.4518260Z         :param expected:
2020-09-21T20:57:49.4518500Z         :return:
2020-09-21T20:57:49.4518700Z         """
2020-09-21T20:57:49.4519420Z         contacted = ansible_module.pki(cli='ca-audit-show',
2020-09-21T20:57:49.4519870Z                                        nssdb=constants.NSSDB,
2020-09-21T20:57:49.4520320Z                                        dbpassword=constants.CLIENT_DATABASE_PASSWORD,
2020-09-21T20:57:49.4521080Z                                        protocol='https',
2020-09-21T20:57:49.4521390Z                                        certnick=certnick)
2020-09-21T20:57:49.4521740Z         for result in contacted.values():
2020-09-21T20:57:49.4522080Z             for iter in expected:
2020-09-21T20:57:49.4522370Z                 if certnick == "CA_AdminV":
2020-09-21T20:57:49.4523030Z                     assert iter in result['stdout']
2020-09-21T20:57:49.4523930Z                     ok("Certificate: %s, Expected Output: %s , Actual Output : %s" % (certnick, iter, result['stdout']))
2020-09-21T20:57:49.4524370Z                 else:
2020-09-21T20:57:49.4524960Z >                   assert iter in result['stderr']
2020-09-21T20:57:49.4526490Z E                   AssertionError: assert 'PKIException: Unauthorized' in 'RuntimeException: org.mozilla.jss.crypto.ObjectNotFoundException: Certificate not found: CA_AdminR'
2020-09-21T20:57:49.4527490Z 
2020-09-21T20:57:49.4528300Z /vagrant/tests/dogtag/pytest-ansible/pytest/sanity/test_role_users.py:85: AssertionError

@06shalini
Copy link
Contributor Author

The error seems to be with python versions. Probably in github actions we are using latest python versions and with that the code is not compatible.

See below error message:
# Add subject name input and submitter information

  if profile_input.has_key('subject_Name_Input_Id'):

E AttributeError: 'dict' object has no attribute 'has_key'

/vagrant/tests/dogtag/pytest-ansible/pki/testlib/common/profile.py:1616: AttributeError

has_key was removed in Python 3

Possible solutions :

  1. Update the github actions to use compatible python version.
  2. Update the code and gitlab pipelines to work with latest version.

Solution 2 needs to be done. I will start working on this.
But it might take some time to get that working as we are still in process of stabilizing the tests.

@SilleBille So is it possible to try python 2.7 in github actions for now ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@06shalini @SilleBille