Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make hamt a go-ipld-prime ipld.Node #70

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

willscott
Copy link

  • The structure of the ipld.NodePrototype is a bit janky. Would be interested in suggestions on how to make that cleaner.
  • This doesn't support the fully life-cycle of NodeBuilder/NodeAssembler, so won't automatically load.
  • IPLD methods don't provide an explicit context, so context.Background is used. Ideally, a context on the node would allow for graceful cancellation of in-progress operations.

@anorth
Copy link
Member

anorth commented Nov 26, 2020

Thanks for showing what this would look like.

My first reaction here is that it's a lot of noise with little immediate value to the core use case of actors. Since the Filecoin actors code is so security-critical, and with very strict compatibility requirements, our general desire is to minimize any unnecessary code and dependencies. I didn't see a go.mod diff here, but expect you need to add go-ipld-prime which would immediately raise a flag.

This HAMT is, unfortunately, not built with go-ipld-prime to begin with. My hope for integration there is to replace it wholesale with a HAMT natively built with/for go-ipld-prime, when that framework and implementation (this one?) is ready. Along with all the other hand-rolled structures, exchanging one set of code and dependencies for another, better one (rather than including both).

I infer that you wanted this compatibility for some downstream processing. Would it work to locally extend the HAMT struct, instead of modifying it here?

@willscott
Copy link
Author

This does not need to depend on go-ipld-prime, just implement the interface defined there for an ipld.Node. This branch is being used in statediff, as a way to create a hamt.Node, and then use it as an ipld.Node in some of the generic ipld code to traverse efficiently through hamts.

@willscott
Copy link
Author

longer term, i'll transition that to the in-progress ipld-schema'd hamt, but this works as a branch for ipld tools to use in the short term. I don't think it needs to be merged, but it's useful for me to maintain it as a branch on this repo so that i can pull in the v3 changes as they finalize.

@willscott
Copy link
Author

cc @warpfork / @mvdan - I recently pushed the additional code to this for allowing HAMTs to act as reifying ADLs

@willscott willscott marked this pull request as ready for review March 23, 2022 22:36
@@ -6,11 +6,10 @@ require (
github.com/ipfs/go-cid v0.0.6
github.com/ipfs/go-ipld-cbor v0.0.4
github.com/ipfs/go-ipld-format v0.0.2 // indirect
github.com/kr/pretty v0.1.0 // indirect
github.com/ipld/go-ipld-prime v0.12.3
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you staying on an older go-ipld-prime on purpose?

}

func loadPointer(p ipld.Node) (*Pointer, error) {
if p.Kind() != ipld.Kind_Link {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you could use a switch here


func loadKV(n ipld.Node) (*KV, error) {
if n.Kind() != ipld.Kind_List {
return nil, fmt.Errorf("kv should be of kind list")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: include the actual kind

}
return &Pointer{KVs: kvs}, nil
}
return nil, fmt.Errorf("unsupported pointer kind")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto with including the actual kind

return nil, e
}
return n.LookupByString(s)
} else if key.Kind() == ipld.Kind_Bytes {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto switch and actual kind in the error

}

func (n *Node) IsNull() bool {
return n.Bitfield == nil
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beware that "is null" should only ever return true if the node is nullable, e.g. a nullable struct field, which I imagine is not the case here. It's not a generic "is empty/zero" that can be useful for any node.


// LSBlockstore creates a blockstore (get/put) interface over a link system
type LSBlockstore struct {
*ipld.LinkSystem
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: embedding this means you're also exposing its methods on your API, so perhaps you want to make it a named and unexported field instead

@BigLep
Copy link
Member

BigLep commented Apr 5, 2022

2022-04-05 conversation: the potential areas of concern that we need in review:

  1. Ensure not affecting performance for standard usecase
  2. Don't somehow break compatibility.

@willscott
Copy link
Author

This is a complementary code path and will not affect any current use cases.
I think the only/primary concern I see here is if we're okay with the extra maintenance burden of the extra code, and what level of test coverage we feel is needed to be comfortable with it staying in sync with the primary implementation over time.

@BigLep
Copy link
Member

BigLep commented Apr 19, 2022

@willscott: 2022-04-19 conversation notes:

  1. @rvagg is good to approve from a correctness standpoint once the code review comments get incorporated BUT
  2. ultimately need a repo owner (@Stebalien , @ZenGround0 ?) to approve this given the extra maintenance burden. Utlimately I think someone in Lotus needs to decide whether they're up for supporting this longer term.

This is a good change, but it does increase the amount that someone needs to understand to properly maintain this library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants