Still problems with test results?

[LorenzoBettini]

On my fork, I recently saw a failure in one of the builds: https://github.com/LorenzoBettini/xtext/actions/runs/10456433264/job/28957424248

I see this:

While there are several links with raw output on that page, I still don't see, in the summary, any direct link to the failed tests. So I don't understand what's flake and what's really failed.

Since -fae has been added, the raw output from the build is not easy to navigate, because, it fails at the end, and not on the project with the failures:

Unless I miss something, I still found this new test report unusable, and in general it makes things harder

cc @cdietrich @szarnekow @HannesWell

[HannesWell]

While there are several links with raw output on that page, I still don't see, in the summary, any direct link to the failed tests. So I don't understand what's flake and what's really failed.

From your screenshot it actually looks like expected to me. For example in m2e it looks the same:
https://github.com/eclipse-m2e/m2e-core/runs/29202595998
In the comment on a PR you don't get each failure listed, you only get that from the overview. In the comment you should only get the link to the overview as described here:
#3144 (comment)

To me the only problem left seems to be that flaky tests are falsely detected as persistent errors. Maybe we are lucky and EnricoMi/publish-unit-test-result-action#539 gets more drive now and is fixed soon.

Alternatively, could the flaky tests be fixed to work on the first run?

[LorenzoBettini]

I still don't have a direct and easy access to the failed test: in the output, besides flakes, it's easy to spot the test that finally failed, while now there's no way to see the failed test immediately, at least, I don't find an easy one.

For example, in the linked URL https://github.com/LorenzoBettini/xtext/actions/runs/10456433264/job/28957424248 can you tell which test(s) finally failed?

If flaky tests could be easily fixed, it would be a better world ;)
we never found a way...

I'd tend to remove at least -fae. @szarnekow @cdietrich what do you think?