Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple reportportal plugin improvements and fixes #3356

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

seberm
Copy link
Collaborator

@seberm seberm commented Nov 12, 2024

This PR should fix the problem with a possible reportportal error when using result: restraint or result: custom:

Received non-ok status code 406 from ReportPortal: {"errorCode":40025,"message":"Start time of child ['Tue Nov 12 11:10:19 UTC 2024'] item should be same or later than start time ['2024-11-12T11:36:35.007'] of the parent item/launch '
11810'"}

For more info, see:

This error is probably caused by the badly sorted result start times. It can happen the datetime formats in the list are not in the same format. The Result.start_time is in ISO format but the self.time() returns str(int(time() * 1000)).

The basic reproducer

$ cat main.fmf
test: |
    tmt-report-result /test/good PASS
    tmt-report-result /test/fail FAIL
    tmt-report-result /test/weird WARN

result: restraint

duration: 5m
$ python -m tmt -r ~/tmt-learning/20 run -a report --how reportportal -v --force

Other improvements

There are also small code improvements like the possibility to ignore the SSL verification, adding the usage of Python properties instead of class methods, or adding handling of ResultOutcome.SKIP outcome which I believe is missing from the mapping.

TODOs:

  • Link the issue with the problem description as soon as it's created.

Pull Request Checklist

@seberm seberm added bug Something isn't working plugin | reportportal The reportportal report plugin labels Nov 12, 2024
@seberm seberm marked this pull request as ready for review November 12, 2024 12:05
@seberm seberm added the ci | full test Pull request is ready for the full test execution label Nov 12, 2024
@seberm
Copy link
Collaborator Author

seberm commented Nov 12, 2024

The problem with start-time/end-time timestamps should be also resolved by existing PR by @kkaarreell :

@seberm seberm force-pushed the feature/reportportal-improvements branch from b2f5688 to c1518ce Compare November 12, 2024 15:50
@seberm seberm mentioned this pull request Nov 12, 2024
8 tasks
@seberm seberm force-pushed the feature/reportportal-improvements branch from 5c4abeb to ba510e2 Compare November 12, 2024 16:17
@seberm seberm linked an issue Nov 13, 2024 that may be closed by this pull request
@seberm seberm added this to the 1.40 milestone Nov 13, 2024
@seberm
Copy link
Collaborator Author

seberm commented Nov 18, 2024

Hello @mildas, could you please test if the changes in this PR resolve your issue (#3360) and if the data are imported to ReportPortal as expected? Thanks!

@kkaarreell
Copy link
Collaborator

I have just tested a plain import of a test plan with passing and failing tests and it worked well.
I also tried to import test results that originally revealed the timestamp issue and it went well too.
I do not plan any particular testing ATM, from my POV it works well.

@seberm
Copy link
Collaborator Author

seberm commented Nov 20, 2024

@kkaarreell

I have just tested a plain import of a test plan with passing and failing tests and it worked well.
I also tried to import test results that originally revealed the timestamp issue and it went well too.
I do not plan any particular testing ATM, from my POV it works well.

Thanks for helping me with the testing! Could you please add a check to this PR that you did a review?

Also, do you know anyone else who uses the ReportPortal and could us help with the testing of these changes? What about @KwisatzHaderach?

@KwisatzHaderach
Copy link
Collaborator

Also, do you know anyone else who uses the ReportPortal and could us help with the testing of these changes? What about @KwisatzHaderach?

not actually using the rp plugin, we do all our exports through data router using polarion xml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ci | full test Pull request is ready for the full test execution plugin | reportportal The reportportal report plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Not possible to use reportportal report plugin with custom results
3 participants