Solana Anchor Decoding #8

enoldev · 2024-10-21T12:19:04Z

No description provided.

enoldev · 2024-10-28T12:50:27Z

Some IDLs will hit this issue:

coral-xyz/anchor#3274

It has been fixed in the "master" branch, but not released yet.

ArnaudBger · 2024-11-01T18:30:53Z

@enoldev

I've tested your code for the following IDL:

ABI

   {
     "address": "DKDX8XbTnCgEk8o1RNnCUokiCmadG1Ch5HLxaz7CnhcD",
     "metadata": {
       "name": "solana_gateway",
       "version": "0.2.3",
       "spec": "0.1.0",
       "description": "Solana Secretpath Gateway"
     },
     "instructions": [
       {
         "name": "callback_test",
         "discriminator": [
           196,
           61,
           185,
           224,
           30,
           229,
           25,
           52
         ],
         "accounts": [
           {
             "name": "secretpath_gateway",
             "signer": true
           }
         ],
         "args": [
           {
             "name": "task_id",
             "type": "u64"
           },
           {
             "name": "result",
             "type": "bytes"
           }
         ]
       },
       {
         "name": "increase_task_id",
         "discriminator": [
           152,
           150,
           176,
           36,
           239,
           171,
           169,
           20
         ],
         "accounts": [
           {
             "name": "gateway_state",
             "writable": true,
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     103,
                     97,
                     116,
                     101,
                     119,
                     97,
                     121,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "owner",
             "writable": true,
             "signer": true,
             "relations": [
               "gateway_state"
             ]
           }
         ],
         "args": [
           {
             "name": "new_task_id",
             "type": "u64"
           }
         ]
       },
       {
         "name": "increase_task_state",
         "discriminator": [
           116,
           13,
           66,
           131,
           160,
           82,
           206,
           223
         ],
         "accounts": [
           {
             "name": "gateway_state",
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     103,
                     97,
                     116,
                     101,
                     119,
                     97,
                     121,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "task_state",
             "writable": true,
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     116,
                     97,
                     115,
                     107,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "owner",
             "writable": true,
             "signer": true,
             "relations": [
               "gateway_state"
             ]
           },
           {
             "name": "system_program",
             "address": "11111111111111111111111111111111"
           }
         ],
         "args": [
           {
             "name": "_len",
             "type": "u64"
           }
         ]
       },
       {
         "name": "initialize",
         "discriminator": [
           175,
           175,
           109,
           31,
           13,
           152,
           155,
           237
         ],
         "accounts": [
           {
             "name": "gateway_state",
             "writable": true,
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     103,
                     97,
                     116,
                     101,
                     119,
                     97,
                     121,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "task_state",
             "writable": true,
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     116,
                     97,
                     115,
                     107,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "owner",
             "writable": true,
             "signer": true
           },
           {
             "name": "system_program",
             "address": "11111111111111111111111111111111"
           }
         ],
         "args": []
       },
       {
         "name": "payout_balance",
         "discriminator": [
           55,
           151,
           80,
           201,
           126,
           222,
           40,
           200
         ],
         "accounts": [
           {
             "name": "gateway_state",
             "writable": true,
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     103,
                     97,
                     116,
                     101,
                     119,
                     97,
                     121,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "owner",
             "writable": true,
             "signer": true,
             "relations": [
               "gateway_state"
             ]
           },
           {
             "name": "system_program",
             "address": "11111111111111111111111111111111"
           }
         ],
         "args": []
       },
       {
         "name": "post_execution",
         "discriminator": [
           52,
           46,
           67,
           194,
           153,
           197,
           69,
           168
         ],
         "accounts": [
           {
             "name": "gateway_state",
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     103,
                     97,
                     116,
                     101,
                     119,
                     97,
                     121,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "task_state",
             "writable": true,
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     116,
                     97,
                     115,
                     107,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "signer",
             "writable": true,
             "signer": true
           },
           {
             "name": "system_program",
             "address": "11111111111111111111111111111111"
           }
         ],
         "args": [
           {
             "name": "task_id",
             "type": "u64"
           },
           {
             "name": "source_network",
             "type": "string"
           },
           {
             "name": "post_execution_info",
             "type": {
               "defined": {
                 "name": "PostExecutionInfo"
               }
             }
           }
         ]
       },
       {
         "name": "send",
         "discriminator": [
           102,
           251,
           20,
           187,
           65,
           75,
           12,
           69
         ],
         "accounts": [
           {
             "name": "gateway_state",
             "writable": true,
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     103,
                     97,
                     116,
                     101,
                     119,
                     97,
                     121,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "task_state",
             "writable": true,
             "pda": {
               "seeds": [
                 {
                   "kind": "const",
                   "value": [
                     116,
                     97,
                     115,
                     107,
                     95,
                     115,
                     116,
                     97,
                     116,
                     101
                   ]
                 }
               ]
             }
           },
           {
             "name": "user",
             "writable": true,
             "signer": true
           },
           {
             "name": "system_program",
             "address": "11111111111111111111111111111111"
           }
         ],
         "args": [
           {
             "name": "user_address",
             "type": "pubkey"
           },
           {
             "name": "routing_info",
             "type": "string"
           },
           {
             "name": "execution_info",
             "type": {
               "defined": {
                 "name": "ExecutionInfo"
               }
             }
           }
         ],
         "returns": {
           "defined": {
             "name": "SendResponse"
           }
         }
       }
     ],
     "accounts": [
       {
         "name": "GatewayState",
         "discriminator": [
           133,
           203,
           164,
           159,
           234,
           201,
           161,
           186
         ]
       },
       {
         "name": "TaskState",
         "discriminator": [
           255,
           33,
           48,
           249,
           220,
           80,
           10,
           9
         ]
       }
     ],
     "errors": [
       {
         "code": 6000,
         "name": "TaskAlreadyCompleted",
         "msg": "Task already completed"
       },
       {
         "code": 6001,
         "name": "InvalidPayloadHash",
         "msg": "Invalid payload hash"
       },
       {
         "code": 6002,
         "name": "InvalidPacketHash",
         "msg": "Invalid packet hash"
       },
       {
         "code": 6003,
         "name": "InvalidPublicKey",
         "msg": "Invalid Public key"
       },
       {
         "code": 6004,
         "name": "Secp256k1RecoverFailure",
         "msg": "Secp256k1 recovery failed"
       },
       {
         "code": 6005,
         "name": "InvalidPacketSignature",
         "msg": "Invalid packet signature"
       },
       {
         "code": 6006,
         "name": "TaskNotFound",
         "msg": "Task not found"
       },
       {
         "code": 6007,
         "name": "InsufficientFunds",
         "msg": "Insufficient funds"
       },
       {
         "code": 6008,
         "name": "InvalidIndex",
         "msg": "Invalid lookup index"
       },
       {
         "code": 6009,
         "name": "InvalidTaskId",
         "msg": "Invalid TaskID"
       },
       {
         "code": 6010,
         "name": "InvalidCallbackAddresses",
         "msg": "Callback Addresses are invalid"
       },
       {
         "code": 6011,
         "name": "BorshDataSerializationFailed",
         "msg": "Borsh Data Serialization failed"
       },
       {
         "code": 6012,
         "name": "InvalidCallbackSelector",
         "msg": "Invalid Callback Selector"
       },
       {
         "code": 6013,
         "name": "MissingRequiredSignature",
         "msg": "MissingRequiredSignature"
       }
     ],
     "types": [
       {
         "name": "ExecutionInfo",
         "type": {
           "kind": "struct",
           "fields": [
             {
               "name": "user_key",
               "type": "bytes"
             },
             {
               "name": "user_pubkey",
               "type": "bytes"
             },
             {
               "name": "routing_code_hash",
               "type": "string"
             },
             {
               "name": "task_destination_network",
               "type": "string"
             },
             {
               "name": "handle",
               "type": "string"
             },
             {
               "name": "nonce",
               "type": {
                 "array": [
                   "u8",
                   12
                 ]
               }
             },
             {
               "name": "callback_gas_limit",
               "type": "u32"
             },
             {
               "name": "payload",
               "type": "bytes"
             },
             {
               "name": "payload_signature",
               "type": {
                 "array": [
                   "u8",
                   64
                 ]
               }
             }
           ]
         }
       },
       {
         "name": "GatewayState",
         "type": {
           "kind": "struct",
           "fields": [
             {
               "name": "owner",
               "type": "pubkey"
             },
             {
               "name": "task_id",
               "type": "u64"
             },
             {
               "name": "bump",
               "type": "u8"
             }
           ]
         }
       },
       {
         "name": "PostExecutionInfo",
         "type": {
           "kind": "struct",
           "fields": [
             {
               "name": "packet_hash",
               "type": {
                 "array": [
                   "u8",
                   32
                 ]
               }
             },
             {
               "name": "callback_address",
               "type": "bytes"
             },
             {
               "name": "callback_selector",
               "type": "bytes"
             },
             {
               "name": "callback_gas_limit",
               "type": {
                 "array": [
                   "u8",
                   4
                 ]
               }
             },
             {
               "name": "packet_signature",
               "type": {
                 "array": [
                   "u8",
                   65
                 ]
               }
             },
             {
               "name": "result",
               "type": "bytes"
             }
           ]
         }
       },
       {
         "name": "SendResponse",
         "type": {
           "kind": "struct",
           "fields": [
             {
               "name": "request_id",
               "type": "u64"
             }
           ]
         }
       },
       {
         "name": "TaskState",
         "serialization": "bytemuckunsafe",
         "repr": {
           "kind": "c"
         },
         "type": {
           "kind": "struct",
           "fields": [
             {
               "name": "tasks",
               "type": {
                 "array": [
                   "u8",
                   296900
                 ]
               }
             }
           ]
         }
       }
     ]
   }

There is an error happening when unmarshalling
ERROR:

Error unmarshaling JSON: failed to unmarshal Type: {             "defined": {               "name": "PostExecutionInfo"             }           }

Can you give me an IDL for which it's working? Like that I could test it more deeply and give better feedbacks!

Important

Test are failing... Please make sure when you want to merge a PR that the test are passing !

ArnaudBger · 2024-11-01T18:46:21Z

Note

I checked the Anchor library, and it's seems that they are fetching the IDL using the method get_account from an rpc
call to a Solana node. They get the idl from the account data for whatever program_id. I think it could be a nice
feature to have this automatic IDL fetcher in the future. Instead of copy pasting the idl...
We can query our own Solana node to do so !

ArnaudBger · 2024-11-01T17:48:10Z

sol-anchor/idl.go

+	Address string `json:"address"`
+}
+
+// --- EVENTS


Suggested change

// --- EVENTS

// --- EVENTS

Instead of having everything in a file, I would rather used multiple files to improve code readability. For example, you create an events.go, instructions.go within an idl folder...

And write all the toSnakeCase kind of functions in a utils.go to share the code among all those files...

ArnaudBger · 2024-11-01T17:49:37Z

sol-anchor/idl.go

+		return fmt.Sprintf("%s: %s.%s as %s,", fieldNameSnakeCaseWithoutInitialUnderscore, variableName, fieldNameSnakeCase, cast)
+	}
+
+	// If cast is NOT needed


Suggested change

// If cast is NOT needed

// If cast is needed

You can write an example of what the function is doing above it. But those comments are not meaningful

ArnaudBger · 2024-11-01T17:53:15Z

sol-anchor/idl.go

+	return ""
+}
+
+func (f *FieldType) Print(fieldName string, variableName string, types []Type) string {


In my point of view, it's confusing to create a Print function that bundles a lot of rust code within the idl.go. I would rather write the maximum of things I can in Rust in the lib.rs.gotmpl to improve readability and to make it easier to debug/iterate...

You should be able to iterate through your IDL struct and do so

This function covers "Printing" in Rust for all the different field types possible (simple, defined, vec, array and option).

The problem with putting all of this in the template is that "defined", "vec" and "array" must be printed recursively.
For example, if the type is a user custom struct like "MyStruct", you will have to recursively print that struct, which will again contain fields with different types.

To handle the fieldType , I would rather use https://github.com/abice/go-enum. You could avoid using multiple if and use rather a switch on different case, it'll be way easier to understand and to read !

ArnaudBger · 2024-11-01T17:56:13Z

sol-anchor/idl.go

+	if f.IsVec() && !f.Vec.IsDefined {
+		return fmt.Sprintf("%s: %s.%s,", fieldNameSnakeCaseWithoutInitialUnderscore, variableName, fieldNameSnakeCase)
+	}
+


You can bundle those two if within one if f.IsVec() {} to improve readbility

ArnaudBger · 2024-11-01T17:58:27Z

sol-anchor/idl.go

+	}
+	if err := json.Unmarshal(data, &arrayType); err == nil && len(arrayType.Array) > 0 {
+		stringType, stringOk := arrayType.Array[0].(string)
+		// TODO: The length should be in position [1] of the array, but it's not decoded correctly. Why?


I can see some TODOs still pending, is it production ready?

ArnaudBger · 2024-11-01T18:01:54Z

sol-anchor/state.go

+}
+
+func (p *Project) ModuleName() string { return strings.ReplaceAll(p.Name, "-", "_") }
+func (p *Project) KebabName() string  { return strings.ReplaceAll(p.Name, "_", "-") }


There is some libraries that does this for you! + The name of the func is weird, we should on what your apply kebab transformation?

ArnaudBger · 2024-11-01T18:11:27Z

sol-anchor/templates/idls/program.json.gotmpl

@@ -0,0 +1 @@
+{{ .IdlString }}


Instead of doing this you can add your program json file in the ProjectFiles by mapping the name with the json raw bytes value. Just like it's done in evm-event-calls for example !

Example:

var rawMessage json.RawMessage if strings.HasPrefix(msg.Value, AbiFilepathPrefix) { abiPath := strings.TrimPrefix(msg.Value, AbiFilepathPrefix) fileBytes, err := os.ReadFile(abiPath) if err != nil { return loop.Seq(c.Msg().Messagef("Cannot read the ABI file %q: %s", abiPath, err).Cmd(), cmd(AskContractABI{})) } rawMessage = json.RawMessage(fileBytes) } else { rawMessage = json.RawMessage(msg.Value) } if _, err := json.Marshal(rawMessage); err != nil { return loop.Seq(c.Msg().Messagef("ABI %q isn't valid: %q", msg.Value, err).Cmd(), cmd(AskContractABI{})) } contract.RawABI = rawMessage

for _, contract := range p.Contracts { res.ProjectFiles[fmt.Sprintf("abi/%s_contract.abi.json", contract.Name)] = []byte(contract.Abi.raw) } for _, dds := range p.DynamicContracts { res.ProjectFiles[fmt.Sprintf("abi/%s_contract.abi.json", dds.Name)] = []byte(dds.Abi.raw) }

ArnaudBger · 2024-11-01T18:18:00Z

sol-anchor/idl.go

+    - vec
+	- optional (simple, defined or vec)
+*/
+func unmarshalDefined(data []byte) string {


In my point of view, it's weird to create a type within a simple func like that... I would rather create the type above it

ArnaudBger · 2024-11-01T18:21:13Z

sol-anchor/idl.go

+
+	return "", ""
+}
+


I don't understand why you created this UnmarshalJSON func, the code here is difficult to understand. Why you didn't map each json field to a struct, just like what you did with Events, Instructions, Types...

The "fields" of an Event or an Instruction could of different types: simple, defined, vec, array or optional.

For example:

Simple (e.g. string, u64...)

{ "type": "string" }

Defined (user-defined structs)

{ "type": { "defined": "MyCustomType" } }

So because the that JSON part could be of different types, we have to try which one matches (not sure if there is a better way of doing it)

ArnaudBger · 2024-11-01T18:21:56Z

sol-anchor/idl.go

+	return ""
+}
+
+func toSnakeCase(str string, initialUnderscore bool) string {


Some library implements the toSnakeCase func !

sduchesneau · 2024-11-04T18:33:14Z

sol-anchor/idl.go

+						continue
+					}
+
+					fieldsInString.WriteString(structField.Type.Print(structField.Name, fieldString, types))


make the 'print' method on 'structField' so you don't have to call structField.Name
It feels more natural to do structField.Print(...) than to print the Field from its type...
I feel that the abstractions are not all on the right objects and that they could be simplified, leading to cleaner code.
Also, methods like PrintDefined shouldn't be capitalized if they aren't used outside the package.

sduchesneau · 2024-11-04T18:34:51Z

sol-anchor/idl.go

+	return t.Option != nil
+}
+
+func (t *FieldType) Resolve() string {


not to start a flamewar here, but a switch case would reduce this function to quite a few less lines (if it fits in my code editor window, it's easier to read quickly)

…th a name param

…in state, add metadata.address and contract name if not present, fix some TypeOptions parsing

enoldev added 4 commits October 21, 2024 14:16

First version

cfd8c1a

Fixes

ccffe70

recursively get defined types

228f007

Add support for vec and option

05d9587

enoldev marked this pull request as ready for review October 28, 2024 12:16

fix

4e6f61d

ArnaudBger requested changes Nov 1, 2024

View reviewed changes

sduchesneau reviewed Nov 4, 2024

View reviewed changes

enoldev and others added 6 commits November 4, 2024 22:07

Only include types if they are used

6a147c2

add support for 'alternate' defined type marshalling, as an object wi…

889eec4

…th a name param

support address inside Metadata and publicKey aliased to 'pubkey'

3498aa9

sol-anchor: add user-defined network name, unexport unmarshalled IDL …

b3f7db9

…in state, add metadata.address and contract name if not present, fix some TypeOptions parsing

bump substreams-rs and prost dependencies on sol-anchor

1654cf1

add blockFilter on sol-anchor

0964b69

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Solana Anchor Decoding #8

Solana Anchor Decoding #8

enoldev commented Oct 21, 2024

enoldev commented Oct 28, 2024

ArnaudBger commented Nov 1, 2024 •

edited

Loading

ArnaudBger commented Nov 1, 2024 •

edited

Loading

ArnaudBger Nov 1, 2024

ArnaudBger Nov 1, 2024

ArnaudBger Nov 1, 2024

enoldev Nov 4, 2024 •

edited

Loading

ArnaudBger Nov 4, 2024

ArnaudBger Nov 1, 2024

ArnaudBger Nov 1, 2024

ArnaudBger Nov 1, 2024

ArnaudBger Nov 1, 2024

ArnaudBger Nov 1, 2024

ArnaudBger Nov 1, 2024

enoldev Nov 4, 2024 •

edited

Loading

ArnaudBger Nov 1, 2024

sduchesneau Nov 4, 2024

sduchesneau Nov 4, 2024

Solana Anchor Decoding #8

Are you sure you want to change the base?

Solana Anchor Decoding #8

Conversation

enoldev commented Oct 21, 2024

enoldev commented Oct 28, 2024

ArnaudBger commented Nov 1, 2024 • edited Loading

ArnaudBger commented Nov 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

enoldev Nov 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

enoldev Nov 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ArnaudBger commented Nov 1, 2024 •

edited

Loading

ArnaudBger commented Nov 1, 2024 •

edited

Loading

enoldev Nov 4, 2024 •

edited

Loading

enoldev Nov 4, 2024 •

edited

Loading