Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: v8 coverage provider has too much tradeoffs to be actually reliable #15360

Open
ericmorand opened this issue Oct 25, 2024 · 0 comments
Open

Comments

@ericmorand
Copy link

ericmorand commented Oct 25, 2024

Version

30.0.0-alpha.6-dev

Steps to reproduce

Use the v8 coverage provider.

Expected behavior

I expect the coverage report to reflect the actual coverage state of the project.

Actual behavior

The coverage report is likely to not reflect the actual coverage state of the project.

Additional context

An issue was opened years ago about the v8 coverage provider tradeoffs: #11188. It was never followed by actions, but it should have been. The points raised in this issue are valid: the v8 coverage provider is not reliable and can't be used to test the coverage of a project: it actually gives a false sense of success by ignoring a lot of true positive, or adding non relevant covered lines and non-existent covered functions that make the coverage result falsely good.

I propose that it is entirely rewritten to bring back the accuracy and correctness of istanbul while using V8 coverage data. I created a PR that adds a coverage provider fueled by One Double Zero, but I think we should go as far as making odz the official v8 coverage provider.

Environment

System:
    OS: Linux 6.8 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
    CPU: (2) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  Binaries:
    Node: 18.20.4 - ~/.nvm/versions/node/v18.20.4/bin/node
    npm: 10.7.0 - ~/.nvm/versions/node/v18.20.4/bin/npm
@ericmorand ericmorand changed the title [Bug]: v8 coverage provider hqs to much tradeoffs to be actually reliable [Bug]: v8 coverage provider has too much tradeoffs to be actually reliable Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant