Test Suite : Determine mechanics and overall structure for long term

[andorsk]

WHAT: A test suite for various PE implementations
Why: To validate compliance with the specifications
Constraints: Should be able to be easy to spin up a test suite for various implementations.

@decentralgabe spoke of some test suite that can be used for PE, potentially docker based. Easy to hook in for other implementors.

[decentralgabe]

I think it would be easily to follow the pattern defined by the JWS Test Suite https://identity.foundation/JWS-Test-Suite/

We make a set of test vectors for all objects and processing. Probably a few vectors:

• Can we serialize and deserialize objects (Definitions, Submissions) across impls
• Can we verify signatures (JWTs, DIDComm, VCs?)
• Can we process presentation definintion and respond with compliant presentation submissions?
• Can we process presentation submission against a presentation definition and come up with a consistent result?

[dtmcg]

Lets try and decompose this into a couple of tasks/issues
strawman of how we could break this down

• the test vectors the suite will test
• the harness mechanism for the suite
• the client template to be uptake by the suite

[bumblefudge]

on today's call, clarified --> 5 sequential issues

1. one simple test - e2e flow, with one super-simple VC and super-simple pres_sub, etc.
2. harness (1 test)
3. normative statement audit
4. rest of tests (could be parallelized)
5. first 3 gotta be done before

[brentzundel]

not sure why this is in milestone v2.1?

[kimdhamilton]

not sure why this is in milestone v2.1?

I suppose I was feeling more optimistic about my bandwidth. Let's make it 3.0

[nklomp]

Yeah we had some discussion about it a week ago that the testsuite could also go without a version