travis: Fix failing travis tests on main #6152

Pearl1594 · 2022-03-22T08:33:04Z

Description

This PR addresses #6115
Prior fix:

$ nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=setup/dev/advanced.cfg  -s -a tags=advanced --hypervisor=simulator test/integration/component/test_project_limits.py 

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /tmp/MarvinLogs/Mar_22_2022_12_45_24_WIEMY4. All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_maxAccountNetworks | Status : EXCEPTION ===

=== TestName: test_maxAccountNetworks | Status : EXCEPTION ===

=== TestName: test_01_project_limits | Status : SUCCESS ===

=== TestName: test_02_project_limits_normal_user | Status : SUCCESS ===

=== TestName: test_03_vm_per_project | Status : SUCCESS ===

=== TestName: test_04_publicip_per_project | Status : FAILED ===

=== TestName: test_05_snapshots_per_project | Status : SUCCESS ===

=== TestName: test_06_volumes_per_project | Status : EXCEPTION ===

=== TestName: test_07_templates_per_project | Status : SUCCESS ===

=== TestName: test_07_templates_per_project | Status : EXCEPTION ===

===final results are now copied to: /tmp/MarvinLogs/test_project_limits_JSZASI===
[12:50:20] pearl@pearl-XPS-15-7590:~/lab/shapeblue/cloudstack$ vi /tmp/MarvinLogs/test_project_limits_JSZASI/results.txt

$ nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=setup/dev/advanced.cfg  -s -a tags=advanced --hypervisor=simulator test/integration/component/test_resource_limits.py 

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /tmp/MarvinLogs/Mar_22_2022_13_09_38_XXJETC. All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_maxAccountNetworks | Status : SUCCESS ===

=== TestName: test_01_vm_per_account | Status : SUCCESS ===

=== TestName: test_02_publicip_per_account | Status : FAILED ===

=== TestName: test_03_snapshots_per_account | Status : SUCCESS ===

=== TestName: test_04_volumes_per_account | Status : SUCCESS ===

=== TestName: test_05_templates_per_account | Status : SUCCESS ===

=== TestName: test_01_publicip_per_domain | Status : FAILED ===

=== TestName: test_01_vm_per_domain | Status : SUCCESS ===

=== TestName: test_03_snapshots_per_domain | Status : SUCCESS ===

=== TestName: test_04_volumes_per_domain | Status : SUCCESS ===

=== TestName: test_05_templates_per_domain | Status : SUCCESS ===

===final results are now copied to: /tmp/MarvinLogs/test_resource_limits_4GD1ST===

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

$ nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=setup/dev/advanced.cfg  -s -a tags=advanced --hypervisor=simulator test/integration/component/test_project_limits.py 

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /tmp/MarvinLogs/Mar_22_2022_13_03_09_I9KF2Z. All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_maxAccountNetworks | Status : SUCCESS ===

=== TestName: test_01_project_limits | Status : SUCCESS ===

=== TestName: test_02_project_limits_normal_user | Status : SUCCESS ===

=== TestName: test_03_vm_per_project | Status : SUCCESS ===

=== TestName: test_04_publicip_per_project | Status : SUCCESS ===

=== TestName: test_05_snapshots_per_project | Status : SUCCESS ===

=== TestName: test_06_volumes_per_project | Status : SUCCESS ===

=== TestName: test_07_templates_per_project | Status : SUCCESS ===

===final results are now copied to: /tmp/MarvinLogs/test_project_limits_HJ63QW===

$ nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=setup/dev/advanced.cfg  -s -a tags=advanced --hypervisor=simulator test/integration/component/test_resource_limits.py 

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /tmp/MarvinLogs/Mar_22_2022_13_32_35_7B5VRA. All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_maxAccountNetworks | Status : SUCCESS ===

=== TestName: test_01_vm_per_account | Status : SUCCESS ===

=== TestName: test_02_publicip_per_account | Status : SUCCESS ===

=== TestName: test_03_snapshots_per_account | Status : SUCCESS ===

=== TestName: test_04_volumes_per_account | Status : SUCCESS ===

=== TestName: test_05_templates_per_account | Status : SUCCESS ===

=== TestName: test_01_publicip_per_domain | Status : SUCCESS ===

=== TestName: test_01_vm_per_domain | Status : SUCCESS ===

=== TestName: test_03_snapshots_per_domain | Status : SUCCESS ===

=== TestName: test_04_volumes_per_domain | Status : SUCCESS ===

=== TestName: test_05_templates_per_domain | Status : SUCCESS ===

===final results are now copied to: /tmp/MarvinLogs/test_resource_limits_X5R7SR===

Pearl1594 · 2022-03-22T11:50:05Z

@blueorangutan package

Pearl1594 · 2022-03-22T11:59:32Z

@blueorangutan package

weizhouapache · 2022-03-22T12:03:06Z

test/integration/component/test_project_limits.py

        try:
            #Clean up, terminate the created network offerings
-            cleanup_resources(self.apiclient, self.cleanup)
+            cleanup_resources(self.apiclient, reversed(self.cleanup))


You can consider super(TestProjectLimits, self).tearDown()

weizhouapache · 2022-03-22T12:03:07Z

test/integration/component/test_project_limits.py

        try:
            #Cleanup resources used
-            cleanup_resources(cls.api_client, cls._cleanup)
+            cleanup_resources(cls.api_client, reversed(cls._cleanup))


You can consider super(TestProjectLimits, cls).tearDownClass()

DaanHoogland

some remarks (and what Wei said)

DaanHoogland · 2022-03-22T12:12:16Z

test/integration/component/test_project_limits.py

-        with self.assertRaises(Exception):
-            PublicIPAddress.create(
-                                           self.apiclient,
-                                           zoneid=virtual_machine_1.zoneid,
-                                           services=self.services["server"],
-                                           networkid=network.id,
-                                           projectid=self.project.id
-                                           )


aren't those part of the test purpose; that it throws an exception when trying to create a public IP?

it constantly fails with an Assertion Error stating - no exception raised @DaanHoogland

ok, wonder why this assertion is in there than. If it is supposed to succeed, we are fine.

DaanHoogland · 2022-03-22T12:14:58Z

test/integration/component/test_project_limits.py

+        for network in networks:
+            self.cleanup.append(Network(network.__dict__))

+        self.cleanup.append(self.project_1)


it shouldn't be added to cleanup with so much posibilities of exceptions in between.

DaanHoogland · 2022-03-22T12:15:09Z

test/integration/component/test_project_limits.py

                         account=self.account.name,
                         domainid=self.account.domainid
                         )
-        self.cleanup.append(self.project_1)


why shouldn't the project be cleaned?

It is cleaned @DaanHoogland but at a later stage, as first the networks need to be cleaned up. As the tests failed with such errors - Can't delete the project yet because it has 1 Network to clean up

it shouldn't be added to the clean list with other statements inbtween. This means exceptions can occur before it is added. It must be added immediately.

the errors are because the list isn't processed in reverse. the call to super takes care of that.

Thanks @DaanHoogland , I'll update the tests with @weizhouapache 's suggestion of using super.

weizhouapache

code lgtm

travis tests results pass 💯

nvazquez · 2022-03-23T11:42:08Z

@blueorangutan package

blueorangutan · 2022-03-23T11:44:03Z

@nvazquez a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2022-03-23T12:19:12Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 2966

nvazquez

LGTM - no smoke tests needed, Travis tests are good now thanks @Pearl1594

travis: Fix failing travis tests on main

55fcdff

Pearl1594 requested review from DaanHoogland, nvazquez and weizhouapache March 22, 2022 11:50

Pearl1594 added this to the 4.17.0.0 milestone Mar 22, 2022

Pearl1594 linked an issue Mar 22, 2022 that may be closed by this pull request

travis tests failed in 4.16/main #6115

Closed

Pearl1594 marked this pull request as ready for review March 22, 2022 11:59

weizhouapache reviewed Mar 22, 2022

View reviewed changes

DaanHoogland requested changes Mar 22, 2022

View reviewed changes

Pearl1594 added 2 commits March 23, 2022 09:29

address comments

4445094

address comments

aecb021

Pearl1594 force-pushed the fix-travis-test branch from 358385d to aecb021 Compare March 23, 2022 04:00

Pearl1594 requested review from DaanHoogland and weizhouapache March 23, 2022 06:16

weizhouapache approved these changes Mar 23, 2022

View reviewed changes

nvazquez added type:bug component:travis labels Mar 23, 2022

yadvr approved these changes Mar 23, 2022

View reviewed changes

nvazquez approved these changes Mar 23, 2022

View reviewed changes

nvazquez merged commit f5e77f7 into apache:main Mar 23, 2022

travis: Fix failing travis tests on main #6152

travis: Fix failing travis tests on main #6152

Uh oh!

Conversation

Pearl1594 commented Mar 22, 2022

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

Pearl1594 commented Mar 22, 2022

Uh oh!

Pearl1594 commented Mar 22, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

nvazquez commented Mar 23, 2022

Uh oh!

blueorangutan commented Mar 23, 2022

Uh oh!

blueorangutan commented Mar 23, 2022

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants