Introduce MPT support (XLS-33d): #5143

gregtatcam · 2024-09-19T07:41:44Z

High Level Overview of Change

New Transactions:

MPTokenIssuanceCreate
MPTokenIssuanceDestory
MPTokenIssuanceSet
MPTokenAuthorize

Modified Transactions:

Payment
Clawback

New Objects:

MPTokenIssuance
MPToken

API updates:

ledger_entry
account_objects
ledger_data

Refactor:

Add new types Asset, MPTIssue, MPTAmount to handle MPT amount.
Add SOElement constructor overload to describe the amount fields, which may have MPT amount (currently only sfAmount). The constructors are used in TxFormats to define fields supporting MPT.

Context of Change

XLS-33d specification

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Refactor (non-breaking change that only restructures code)
Performance (increase or change in throughput and/or latency)
Tests (you added tests for code that already exists, or your new feature included in this PR)
Documentation update
Chore (no impact to binary, e.g. .gitignore, formatting, dropping support for older tooling)
Release

API Impact

Public API: New feature (new methods and/or new fields)
Public API: Breaking change (in general, breaking changes should only impact the next api_version)
libxrpl change (any change that may affect libxrpl or dependents of libxrpl)
Peer protocol change (must be backward compatible or bump the peer protocol version)

Test Plan

Added test for new feature:

MPT

Future Tasks

Integrate MPT into XRPL DEX.

codecov · 2024-09-19T08:05:17Z

Codecov Report

Attention: Patch coverage is 83.18068% with 202 lines in your changes missing coverage. Please review.

Project coverage is 76.2%. Comparing base (1fbf8da) to head (8af452e).
Report is 2 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/libxrpl/protocol/STAmount.cpp	79.9%	34 Missing ⚠️
src/xrpld/rpc/handlers/LedgerEntry.cpp	5.9%	32 Missing ⚠️
src/xrpld/app/tx/detail/InvariantCheck.cpp	65.5%	30 Missing ⚠️
src/xrpld/app/tx/detail/Payment.cpp	86.5%	21 Missing ⚠️
src/libxrpl/basics/MPTAmount.cpp	12.5%	14 Missing ⚠️
src/xrpld/ledger/detail/View.cpp	91.8%	11 Missing ⚠️
src/libxrpl/protocol/Asset.cpp	70.4%	8 Missing ⚠️
src/xrpld/app/tx/detail/MPTokenAuthorize.cpp	92.5%	8 Missing ⚠️
src/libxrpl/protocol/MPTIssue.cpp	62.5%	6 Missing ⚠️
src/libxrpl/protocol/STParsedJSON.cpp	62.5%	6 Missing ⚠️
... and 13 more

Additional details and impacted files

@@            Coverage Diff            @@
##           develop   #5143     +/-   ##
=========================================
+ Coverage     76.2%   76.2%   +0.1%     
=========================================
  Files          760     775     +15     
  Lines        61568   62513    +945     
  Branches      8126    8212     +86     
=========================================
+ Hits         46909   47661    +752     
- Misses       14659   14852    +193

Files with missing lines	Coverage Δ
include/xrpl/basics/MPTAmount.h	`100.0% <100.0%> (ø)`
include/xrpl/basics/Number.h	`100.0% <100.0%> (ø)`
include/xrpl/basics/XRPAmount.h	`98.8% <ø> (ø)`
include/xrpl/basics/base_uint.h	`96.6% <ø> (ø)`
include/xrpl/protocol/AmountConversions.h	`87.5% <100.0%> (ø)`
include/xrpl/protocol/Feature.h	`100.0% <ø> (ø)`
include/xrpl/protocol/Indexes.h	`100.0% <100.0%> (ø)`
include/xrpl/protocol/Issue.h	`100.0% <100.0%> (ø)`
include/xrpl/protocol/MPTIssue.h	`100.0% <100.0%> (ø)`
include/xrpl/protocol/SField.h	`100.0% <ø> (ø)`
... and 57 more

... and 6 files with indirect coverage changes

New Transactions: - MPTokenIssuanceCreate - MPTokenIssuanceDestory - MPTokenIssuanceSet - MPTokenAuthorize Modified Transactions: - Payment - Clawback New Objects: - MPTokenIssuance - MPTokenAuthorize API updates: - ledger_entry - account_objects - ledger_data Read full spec: https://github.com/XRPLF/XRPL-Standards/tree/master/XLS-0033d-multi-purpose-tokens --------- Co-authored-by: Shawn Xie <[email protected]> Co-authored-by: Howard Hinnant <[email protected]>

HowardHinnant · 2024-09-30T19:08:56Z

include/xrpl/protocol/LedgerFormats.h

+    lsfMPTCanClawback = 0x00000040,
+
+    // ltMPTOKEN
+    lsfMPTAuthorized = 0x00000002,


lsfMPTLocked, lsfMPTCanLock and lsfMPTAuthorized do not have unique values within this enum. Is that on purpose?

The LedgerSpecificFlags enum is really just a way to define a bunch of constants. The flags are unique per ledger object type. Those are divided by the comment / label before each one. It might be worthwhile to split them into one per object type, but that's probably beyond the scope of this PR.

thejohnfreeman

Submitting the first phase of my review, a complete review of the changes to libxrpl.

include/xrpl/basics/MPTAmount.h

include/xrpl/protocol/Asset.h

include/xrpl/protocol/UintTypes.h

src/libxrpl/protocol/LedgerFormats.cpp

src/libxrpl/protocol/STTx.cpp

ximinez

Partial review:

include/xrpl/basics/MPTAmount.h

src/test/jtx/impl/mpt.cpp

include/xrpl/basics/MPTAmount.h

ximinez · 2024-09-30T21:34:55Z

include/xrpl/protocol/Asset.h

+template <ValidIssueType TIss>
+constexpr TIss const&
+Asset::get() const
+{
+    if (!std::holds_alternative<TIss>(issue_))
+        Throw<std::logic_error>("Asset is not a requested issue");
+    return std::get<TIss>(issue_);
+}
+
+template <ValidIssueType TIss>
+TIss&
+Asset::get()
+{
+    if (!std::holds_alternative<TIss>(issue_))
+        Throw<std::logic_error>("Asset is not a requested issue");
+    return std::get<TIss>(issue_);


Would it make sense to have a function like get that doesn't throw, but instead returns a std::optional<TIss>. It would return a nullopt if holds_alternative returned false.

Good idea. It'll be currently unused though. It makes sense to add when needed.

include/xrpl/protocol/LedgerFormats.h

include/xrpl/protocol/SOTemplate.h

src/libxrpl/protocol/STAmount.cpp

Co-authored-by: Ed Hennis <[email protected]>

Update MPT Payment errors to be consistent with IOU Payment.

ximinez

Partial review

ximinez · 2024-10-01T19:07:42Z

include/xrpl/basics/MPTAmount.h

+
+namespace ripple {
+
+class MPTAmount : private boost::totally_ordered<MPTAmount>,


Seeing how similar MPTAmount is to XRPAmount it's a pity that they aren't implemented with common code. I.e. through a base class, or a template or something. Rather than diving down the potential rabbit hole of implementing it, I'm just going to leave this note here for someone else or for future reference.

My opinion:

They should be called MPTNumber and XRPNumber because they represent quantities, while STAmount represents a quantity plus an asset / issue / unit (however you want to think of it). These are more like Number, and convert directly to and from it.

All of the arithmetic is moving to Number after the switchover anyway. In due time, when we retire that amendment, effectively locking it in permanently, then I think we can remove MPTAmount and XRPAmount.

Should IOUAmount change too? I think that Amount better communicates what the value is. I don't think you'd say number when talking about tokens or currencies even if the values don't have associate unit. And XRPAmount doesn't need an issue, it's implicit. Also doesn't seem like this refactoring has to be done in MPT PR. But this is just my opinion. I'll change if everyone thinks Number is better. There are over 300 instances of XRPAmount, MPTAmount, IOUAmount.

I created a ticket to refactor MPTAmount and XRPAmount to use a common code + renaming.

include/xrpl/protocol/STAmount.h

src/xrpld/app/misc/NetworkOPs.cpp

src/libxrpl/protocol/LedgerFormats.cpp

src/libxrpl/protocol/MPTIssue.cpp

ximinez · 2024-10-01T23:39:42Z

src/libxrpl/protocol/STAmount.cpp

@@ -21,6 +21,7 @@
 #include <xrpl/basics/contract.h>
 #include <xrpl/basics/safe_cast.h>
 #include <xrpl/beast/core/LexicalCast.h>
+#include <xrpl/protocol/Indexes.h>


You don't need Indexes.h here, but you do need Protocol.h.

src/xrpld/ledger/detail/View.cpp

Co-authored-by: Ed Hennis <[email protected]>

thejohnfreeman · 2024-10-02T17:42:57Z

include/xrpl/protocol/Asset.h

+/* Asset is a variant of Issue (XRP and IOU) and MPTIssue (MPT).
+ * It enables handling of different issues when either one is expected.
+ * For instance, it extends STAmount class to support either issue
+ * in a general way. It handles specifics such arithmetics and serialization
+ * depending on specific issue type held by Asset.


I find it odd that you think of this as "either one" (of two, Issue or MPTIssue) instead of "any one" (of three, XRP, IOU, or MPT). Don't we want our abstract ledger functions to think in terms of 3 issue types instead of 2 specific implementations?

I find the example (STAmount) a little hard to understand, too. The "it" at the beginning of the last sentence refers to STAmount, while the "it" in the sentence before is Asset? I don't think "extends" is the right word either. Is the example just trying to convey that STAmount uses Asset to abstract arithmetic and serialization operations over all 3 issue types?

I'm left wondering why we have this Asset instead of making Issue support 3 issue types. STAmount has a native() -> bool method. Can Asset have that too? Issue has a getIssuer() -> AccountID method. Can Asset have that too? Why does Asset exist if it's not going to offer any helper methods? With no helpers, it could just be a type alias for std::variant.

thejohnfreeman · 2024-10-02T18:32:42Z

include/xrpl/basics/MPTAmount.h

+
+namespace ripple {
+
+class MPTAmount : private boost::totally_ordered<MPTAmount>,


My opinion:

They should be called MPTNumber and XRPNumber because they represent quantities, while STAmount represents a quantity plus an asset / issue / unit (however you want to think of it). These are more like Number, and convert directly to and from it.

All of the arithmetic is moving to Number after the switchover anyway. In due time, when we retire that amendment, effectively locking it in permanently, then I think we can remove MPTAmount and XRPAmount.

thejohnfreeman · 2024-10-02T18:37:06Z

include/xrpl/basics/MPTAmount.h

+    friend std::istream&
+    operator>>(std::istream& s, MPTAmount& val);
+


I think this was left behind.

Suggested change

friend std::istream&

operator>>(std::istream& s, MPTAmount& val);

thejohnfreeman · 2024-10-02T18:40:27Z

include/xrpl/protocol/Indexes.h

+MPTID
+makeMptID(AccountID const& account, std::uint32_t sequence);
+


What do you think of putting sequence first in the parameter list? (Here and in the overload of mptIssuance(), and maybe elsewhere.)

thejohnfreeman · 2024-10-02T18:44:47Z

include/xrpl/protocol/Indexes.h

+mptoken(MPTID const& issuanceID, AccountID const& holder) noexcept;
+
+inline Keylet
+mptoken(uint256 const& issuanceKey)


This parameter should be mptokenKey right? I think I confused you with my earlier comment on the other parameters. I was just saying that the issuance parameter name should end with Key like issuanceKey and mptokenKey.

thejohnfreeman · 2024-10-02T18:51:37Z

include/xrpl/protocol/MPTIssue.h

+
+namespace ripple {
+
+class MPTIssue


What does MPTIssue give me that MPTID does not? I'm guessing it's something like "so template code can call issue.getText() regardless of whether the issue object is an Issue or an MPTIssue." Is that right? I want the description to explain why this class exists and is not just a type alias of MPTID.

thejohnfreeman · 2024-10-02T18:55:51Z

include/xrpl/protocol/jss.h

+JSS(MPTokenIssuance);     // ledger type.
+JSS(MPTokenIssuanceCreate);   // transaction type.


It's worth it so we don't get radical re-formatting every time a few strings are added. We want the change set to highlight meaningful changes. GitHub's whitespace exclusion can hide these changes, but it can mask unwanted changes too. It's fine to restore the previous formatting so that this changeset has only the added strings.

thejohnfreeman · 2024-10-02T19:03:36Z

src/libxrpl/protocol/STTx.cpp

@@ -543,6 +548,32 @@ isAccountFieldOkay(STObject const& st)
    return true;
 }

+static bool
+invalidMPTAmountInTx(STObject const& tx)


Every transactor (the base Transactor) checks that the Fee field is XRP and non-negative. No extra work is necessary to keep out MPT. If this code and its caller were removed, would any tests succeed that should fail? I'm guessing some different error codes would be returned, but that's fine.

thejohnfreeman · 2024-10-02T19:13:54Z

include/xrpl/protocol/UintTypes.h

+/** MPTID is a 192-bit value representing MPT Issuance ID,
+ * which is a concatenation of a 32-bit sequence (big endian)
+ * and a 160-bit account */
+using MPTID = base_uint<192>;


@shawnxie999 suggests they are serialized in this order because STAmount is serialized in the order of currency then account ID. Whatever the reason, can you add it to this docstring, please?

I don't think there is any relevance between MPTID and how STAmount is serialized. MPTID is serialized as one block of data, there is no distinction between sequence and account in serialization. It could have been account first and then sequence.

ximinez

Partial review

src/libxrpl/protocol/STAmount.cpp

Co-authored-by: Ed Hennis <[email protected]>

use std::min, std::max Co-authored-by: Ed Hennis <[email protected]>

* Refactor getSNValue/getMPTValue * Use std::min, std::max * Fix min/max values for MPT

thejohnfreeman · 2024-10-03T15:06:04Z

include/xrpl/protocol/TxFlags.h

+// MPTokenIssuanceCreate flags:
+constexpr std::uint32_t const tfMPTokenIssuanceCreateMask  =
+  ~(tfMPTCanLock | tfMPTRequireAuth | tfMPTCanEscrow | tfMPTCanTrade | tfMPTCanTransfer | tfMPTCanClawback | tfUniversal);
+
+// MPTokenIssuanceDestroy flags:
+constexpr std::uint32_t const tfMPTokenIssuanceDestroyMask  = ~tfUniversal;
+
+// MPTokenAuthorize flags:
+constexpr std::uint32_t const tfMPTokenAuthorizeMask  = ~(tfMPTUnauthorize | tfUniversal);
+
+// MPTokenIssuanceSet flags:
+constexpr std::uint32_t const tfMPTokenIssuanceSetMask  = ~(tfMPTLock | tfMPTUnlock | tfUniversal);


Please move the masks next to their flags, like the pattern for AMM, and please put tfUniversal first in the sequences.

gregtatcam mentioned this pull request Sep 19, 2024

Introduce MPT V1 support (XLS-33d) #5108

Closed

13 tasks

gregtatcam requested review from HowardHinnant and ximinez September 19, 2024 07:43

gregtatcam force-pushed the feature/mpt-v1-var-issues branch 2 times, most recently from 4ca85a2 to ba5f67c Compare September 19, 2024 14:22

gregtatcam force-pushed the feature/mpt-v1-var-issues branch from ba5f67c to 6d6fda2 Compare September 27, 2024 14:01

HowardHinnant reviewed Sep 30, 2024

View reviewed changes

thejohnfreeman requested changes Sep 30, 2024

View reviewed changes

Merge branch 'develop' into feature/mpt-v1-var-issues

aaf7fe5

ximinez requested changes Sep 30, 2024

View reviewed changes

gregtatcam and others added 3 commits October 1, 2024 13:25

Apply suggestions from code review

f84e9ea

Co-authored-by: Ed Hennis <[email protected]>

Address reviewer's feedback

18515dd

Allow MPT SendMax in Payment tx.

aef1426

Update MPT Payment errors to be consistent with IOU Payment.

ximinez requested changes Oct 1, 2024

View reviewed changes

gregtatcam and others added 2 commits October 2, 2024 07:47

Apply suggestions from code review

9136a89

Co-authored-by: Ed Hennis <[email protected]>

Address reviewer's feedback

16a029a

thejohnfreeman requested changes Oct 2, 2024

View reviewed changes

ximinez requested changes Oct 2, 2024

View reviewed changes

gregtatcam and others added 4 commits October 3, 2024 08:45

Manually align jss and disable clang-format

2beb2d9

Apply suggestions from code review

3cc7003

Co-authored-by: Ed Hennis <[email protected]>

Fix clang format

069273c

Apply suggestions from code review

86bcebd

use std::min, std::max Co-authored-by: Ed Hennis <[email protected]>

thejohnfreeman mentioned this pull request Oct 3, 2024

Consolidate definitions of fields, objects, transactions, and features #5122

Open

Address reviewer's feedback

b9cb9ca

* Refactor getSNValue/getMPTValue * Use std::min, std::max * Fix min/max values for MPT

thejohnfreeman reviewed Oct 3, 2024

View reviewed changes

shawnxie999 added 2 commits October 3, 2024 11:07

remove unneeded fields (#36)

8af452e

rename codes (#37)

fab0c78

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce MPT support (XLS-33d): #5143

Introduce MPT support (XLS-33d): #5143

gregtatcam commented Sep 19, 2024

codecov bot commented Sep 19, 2024 •

edited

Loading

HowardHinnant Sep 30, 2024

ximinez Sep 30, 2024

thejohnfreeman left a comment

ximinez left a comment

ximinez Sep 30, 2024

gregtatcam Oct 1, 2024

ximinez left a comment

ximinez Oct 1, 2024

thejohnfreeman Oct 2, 2024

gregtatcam Oct 3, 2024

gregtatcam Oct 3, 2024

ximinez Oct 1, 2024

thejohnfreeman Oct 2, 2024

thejohnfreeman Oct 2, 2024

thejohnfreeman Oct 2, 2024

thejohnfreeman Oct 2, 2024

thejohnfreeman Oct 2, 2024

thejohnfreeman Oct 2, 2024

thejohnfreeman Oct 2, 2024

thejohnfreeman Oct 2, 2024

thejohnfreeman Oct 2, 2024

gregtatcam Oct 3, 2024

ximinez left a comment

thejohnfreeman Oct 3, 2024


		namespace ripple {

		class MPTAmount : private boost::totally_ordered<MPTAmount>,

		friend std::istream&
		operator>>(std::istream& s, MPTAmount& val);

		MPTID
		makeMptID(AccountID const& account, std::uint32_t sequence);

		JSS(MPTokenIssuance); // ledger type.
		JSS(MPTokenIssuanceCreate); // transaction type.

Introduce MPT support (XLS-33d): #5143

Are you sure you want to change the base?

Introduce MPT support (XLS-33d): #5143

Conversation

gregtatcam commented Sep 19, 2024

High Level Overview of Change

Context of Change

Type of Change

API Impact

Test Plan

Future Tasks

codecov bot commented Sep 19, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thejohnfreeman left a comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Sep 19, 2024 •

edited

Loading