Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry test_psu.py::TestPsuApi::test_power if power not within tolerance #14788

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

smagarwal-arista
Copy link
Contributor

@smagarwal-arista smagarwal-arista commented Sep 27, 2024

Description of PR

Summary:
If test_psu.py::TestPsuApi::test_power fails on abs(power - (voltage*current)) < power*0.1 then retry the test. The test is repeated for a maximum of three times until it passes, else it is failed. This change is introduced to account for the stringent tolerance level of 10% and the errors that might be caused due to time differences in reading each of the parameters (current, voltage and power).

Fixes # (issue)
Addresses issue: https://github.com/aristanetworks/sonic-qual.msft/issues/209

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 202012
  • 202205
  • 202305
  • 202311
  • 202405

Approach

What is the motivation for this PR?

Improve the test to account for stringent tolerance level and asymmetric reading of parameters for power calculation.

How did you do it?

If the test fails for the condition abs(power - (voltage*current)) < power*0.1, the test will collect fresh parameters (voltage, current and power) from the PSU telemetry and recheck the condition. It is limited to a maximum of 3 attempts.

How did you verify/test it?

Tested internally by varying the tolerance level and verifying the number of api calls.

Any platform specific information?

N/A

Supported testbed topology if it's a new test case?

N/A

Documentation

N/A

If the test fails on `abs(power - (voltage*current)) < power*0.1` then retry the test. Test is repeated for a maximum of three times until it passes, else it is failed.
This change is introduced to account for the stringent tolerance level of 10% and the errors that might be caused due to time differences reading each of the parameters.
@mssonicbld
Copy link
Collaborator

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing tests/platform_tests/api/test_psu.py

fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/platform_tests/api/test_psu.py:210:29: E231 missing whitespace after ','
tests/platform_tests/api/test_psu.py:238:37: E128 continuation line under-indented for visual indent
tests/platform_tests/api/test_psu.py:252:33: E128 continuation line under-indented for visual indent
tests/platform_tests/api/test_psu.py:256:121: E501 line too long (122 > 120 characters)
tests/platform_tests/api/test_psu.py:260:37: E128 continuation line under-indented for visual indent
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

If the test fails on `abs(power - (voltage*current)) < power*0.1` then retry the test. The test is repeated for a maximum of three times until it passes.
This change is introduced to account for the stringent tolerance level of 10% and the errors that might be caused due to time differences in reading each of the parameters.
@smagarwal-arista smagarwal-arista changed the title Retry test_power calculated power above 10% Retry test_psu.py::TestPsuApi::test_power if power not within tolerance Sep 27, 2024
@smagarwal-arista
Copy link
Contributor Author

/azpw run Azure.sonic-mgmt

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-mgmt

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@smagarwal-arista smagarwal-arista marked this pull request as ready for review October 1, 2024 13:40
.format(voltage, psu_id, low_threshold, high_threshold))

break

self.assert_expectations()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the self.assert_expectations() evaluate the 3 test run values? What happens for different combinations of the expected values? For example:

Test1: True
Test2: False
Test3: True

In this combination, does the test raise assert or passes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants