Refactor offchain state

[45930]

Summary

The experimental offchain state API currently uses a singleton class to manage storage on behalf of a smart contract. This blocks the use case where many instances of the same smart contract need to be instantiated at the same time, because they will all use the same offchain state under the hood, leading to data mismatches.

This PR splits the offchain state API into OffchainState and OffchainStateInstance such that the OffchainState can be compiled into a circuit definition, and the OffchainStateInstance stores the internal data like which contract instance it is associated with and the merkle trees of data.

Note

I called it out in line, but please note that there is still an ongoing DevX problem where the smart contract class definition has to reference a specific offchain state instance in order to compile. That leaves us in the same problem, where unless handled by the developer, multiple contracts will point to the same storage. There is now a viable workaround, but it doesn't work as nicely as we'd like. I'd appreciate any input to overcome this issue.

Closes: #1832

[45930]

This line is the source of a lot of frustration. Here are some alternatives attempted:

• offchainState = offchainState.init()
  • Re-initialized the state every time the prover is invoked. The smart contract is not capable of staying up to date with the contract state.
  • OffchainState#memoizedInstances
    • Attempted to store previously initialized instances in a map in global state to get around the re-init problem, but ran into issues trying to read variables in provable code. E.g. if we use the contract address as a key, the prover blows up.
• offchainState: typeof offchainStateInstance
  • The contract can't compile because provable methods rely on offchainState being defined at compile time

That leaves the implementation as implemented here. The instance offchainStateInstance is tied to the class definition of the smart contract. That means 2 or more smart contract instances will all have the same offchain state instance. Even if we reset the instance state after initialization, the prover will re-run this line and set the offchain state back to the orifinally-defined value.

I found the best/only way to manage this is to turn the whole smart contract class into a closure so that the definition of each smart contract can be paired with its offchain state instance. This doesn't feel amazing, but it does unblock the use case.

[45930]

testLocal accepts a single contract instance only, so I re-implemented the behavior here for the specific case I needed.

[45930]

Contracts have the same _verificationKey.hash, but _provers need to be recompiled, or else all the contracts will point to the same instance of offchain state again.

[kadirchan]

Love it

[Trivo25]

Given that these modifications affect both the public API and developer usage patterns, we should evaluate options for maintaining backwards compatibility. If backwards compatibility isn't feasible, we may have to consider targeting V2 for these changes

[45930]

@Trivo25 this is in the experimental namespace, so do we need to be worried about compatibility?

I do think an accompanying PR in the docs is required to match this change before release.

[Trivo25]

@Trivo25 this is in the experimental namespace, so do we need to be worried about compatibility?

That's a good point, then probably not

[mitschabaude]

Attempted to store previously initialized instances in a map in global state to get around the re-init problem, but ran into issues trying to read variables in provable code. E.g. if we use the contract address as a key, the prover blows up.

@45930 sounds like that could be solved with Provable.asProver()

[45930]

sounds like that could be solved with Provable.asProver()

Ok, I think I have a working branch locally that works as expected with the memoization happening "behind the scenes". I don't love that this is hidden from the developer, but I like it better than forcing them to use a closure. WDYT?

// Closure ensures that different contract states never interfere
function ExampleContractWithState() {
  const offchainStateInstance = offchainState.init();
  class ExampleContract extends SmartContract { ]
  
  return { offchainStateInstance, ExampleContract }
}

OR

class ExampleContract extends SmartContract {
  @state(OffchainState.Commitments) offchainStateCommitments =
    offchainState.emptyCommitments();
  
  // init method handles the instance management and doesn't expose it to the user at all
  offchainState = offchainState.init(this);
}

[45930]

@mitschabaude , bf0e09e is pretty clean!

I think this API is a lot less error-prone and requires less deep knowledge to use. But it does add more secret global state, which tends to cause its own issues down the line.

[mitschabaude]

in general, this will be the wrong place for connecting the offchain state. init() is only called on initial deployment. can't we move this into offchainState.init()?

[mitschabaude]

any reason not to do instance.setContractInstance(contractInstance) here?