Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container is reported as Passed/Successful when discovery fails #2538

Open
3 tasks done
fflaten opened this issue Jul 9, 2024 · 3 comments
Open
3 tasks done

Container is reported as Passed/Successful when discovery fails #2538

fflaten opened this issue Jul 9, 2024 · 3 comments

Comments

@fflaten
Copy link
Collaborator

fflaten commented Jul 9, 2024

Checklist

What is the issue?

When Discovery fails in a container after a valid test:

  • The blocks and tests up until the error is executed. Is this expected?
  • The block containing the discovery error is considered Passed
  • The container is reported as Passed even though it's listed under FailedContainers

The run is still reported as Failed and the error code returned from Invoke-Pester will be non-zero due to the failed container (set to 1 in repro).

Expected Behavior

Is this expected behavior?
Should Container and block be successful when only partly discovered?

Steps To Reproduce

$sb = {
    Describe "d" {
        It '1' {}

        throw 'omg'
    }
}

$container = New-PesterContainer -ScriptBlock $sb
$r = Invoke-Pester -Container $container -PassThru

Describe your environment

Pester version : 6.0.0-alpha4 /workspaces/Pester/bin/Pester.psm1
PowerShell version : 7.4.1
OS version : Unix 5.15.153.1

Possible Solution?

No response

@nohwnd
Copy link
Member

nohwnd commented Jul 10, 2024

The blocks and tests up until the error is executed. Is this expected+

Yes, originally I was failing just after discovery, but then it was very hard to migrate tests gradually because all tests had to be discoverable, instead of going test by test, file by file.

The block containing the discovery error is considered Passed

I think this should fail the block.

The container is reported as Passed even though it's listed under FailedContainers

Same here the container should fail.

@fflaten
Copy link
Collaborator Author

fflaten commented Jul 10, 2024

Thanks, that makes sense.

So fix the result values, but keep the partial container execution as-is?

@nohwnd
Copy link
Member

nohwnd commented Jul 10, 2024

I think it makes sense that way, what do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants