SIMD-0296: Increase Transaction Size Limit

[jacobcreech]

Today's transaction size limit is capped at 1232 bytes for the payload, significantly limiting some of the usecases that developers on Solana can do.

This proposal raises that limit to 4kb, with potential for raising higher on a new transaction format.

[cryptopapi997]

Very excited for this!

One point about this comes to mind for me upon a read-through: This SIMD explicitly leaves the fee structure for larger txs as an open question. Imo it would be better to settle this before merging this SIMD, as otherwise we run the risk of determining a fee structure that requires some extra changes to the proposed v1 tx format / lock ourselves into the v1 format prematurely.

Also, iiuc the proposed ResourceRequests field would then supercede #287 since the mask could specify CU budget & price right?

[mcintyre94]

Definitely in favour of this SIMD! Just some comments about the proposed message format.

[mcintyre94]

Are we removing durable nonce support for v1 transactions?

[steveluscher]

If not, then rename this to LifetimeSpecifier.

[KirillLykov]

As part of these discussions, it was proposed to get rid of nonce because it hits accountsdb, see https://github.com/solana-foundation/solana-improvement-documents/pull/296/files#r2129100914

[apfitzge]

We can't let noncery block this proposal. Rename it to LifetimeSpecficier seems fine.

[jacobcreech]

I'm still in favor of removal of nonce support for v1 transactions. With v0 and legacy still coexisting with v1, this shouldn't be a blocker.

[austbot]

@jacobcreech I love this idea. Thank you for putting it together and for all your hard work on it.
I have a few concerns on how it affects validators and how its rolled out without destroying backward compatibility.

Simply because QUIC doesn't specify a maximum stream size does not give license to arbitrarily increase the size. I love the idea of increasing size and want it, but im curious if this SIMD is a rubber stamp of something the core team already wants to do or if this is an actual proposal. The Solana team an community are much more thorough in how proposals are brought and I would at minimum expect some kind of analysis on how shoving 4kb through a normally 1kb pipe will affect the network. Will it require new hardware or requirements for those not on rented setups? Will the network cards need to be tweaked? One thing I really fee strongly about is to USE THE TXN VERSIONING, which you have put forth. I have seen rumblings of some new versioning thing or not doing this under a version. Breaking compatibility just kneecaps solana adoption because so many developers will just get errors and users will suddenly have daaps not working. Yes these devs should update their apps, but the reality is that some of the biggest apps with dedicated teams still dont update things(UIs, Clis, Tooling) quickly "excuse me ...cough cough ...Raydium" or they are still on legacy txns with no priority fee (I cant even tell you who this is). Another problem will be that documentation will instantly be broken, and people will be building apps with LUTs and so forth while thinking that its the correct documentation, but it wont work unless there is a commitment to backward compatibility(which sucks, its painful). Lets call it a draw then, for the pain we were all put under having to use LUTs and the PDA model in the first place, core team can relent and feel the pain of managing another version, or perhaps having a rollout of deprecating legacy txns (that should get the big boys moving).

Again thanks, and Im so excited for this.

(Not written by AI)

[ptaffet-jump]

Can we get some more details on validation/invariants? For example, the sum of the 3 values in the legacy header must be no more than NumStaticKeys, etc.

Payload Length >= 11 + 32 + NumStaticKeys*32 + 8*popcnt(ResourceRequestMask)+8*NumInstructions

Where do the offsets in the ixs field start counting from? Presumably the trailing data section, but this should be mentioned.

accounts_offset + num_accounts <=Payload Length - (11 + 32 + NumStaticKeys*32 + 8*popcnt(ResourceRequestMask)+8*NumInstructions), etc.

[ptaffet-jump]

Can we add a field for the fee that the transaction pays in lamports? I think we should take advantage of the opportunity this new format presents to switch from pricing in microlamports_per_cu to one overall price. This would be simpler for developers.
An alternative would be to just reserve bit 0 in the ResourceRequestMask for it, but why burn one of the bits?

We've also talked about the idea of a failure fee (or an in-protocol extra fee for success). This could be a use of the ResourceRequestMask, but I don't know that it fits the name exactly.

More broadly, we've talked about getting bundles in protocol. We should think about this as we develop this new format.

[steveluscher]

I presume that the accounts which are looked up in the TRAILING_DATA_SECTION are made up of (u8) indexes into the StaticKeys array above. If true, this design adds 2 bytes of data per account instruction over the existing design.

This SIMD (worst case):

• u8 num ’static keys‘
• [u8; 32] per ‘static key’
• u16 num accounts per instruction
• u16 num accounts offset per instruction
• u8 index of account in the trailing data section per address per instruction

Fixed cost: 1 byte (num ‘static keys’)
Per instruction cost: 4 bytes (num accounts + accounts offset)
Per address cost: (32 + num_instructions) bytes (key + entry in TRAILING_DATA_SECTION per instruction)

Previously (worst case):

• u8 num ‘static keys’
• [u8; 32] per ‘static key’
• compact-u16-array account indexes per address per instruction

Fixed cost: 1 byte (num ‘static keys’)
Per instruction cost: 2 bytes (compact-u16 index array length)
Per address cost: (32 + num_instructions) bytes (key + entry instruction's compact-u16 array)

[apfitzge]

It's unclear to me where you're getting 2 extra bytes per account. The design is definitely less space efficient, that's fine from our view since it's paired with a tx-size increase. It's better have constant sized fields than deal with compressed u16s inherent branchiness.

[steveluscher]

I said ‘per account’; meant ‘per instruction.’ Sorry.

[jacobcreech]

One point about this comes to mind for me upon a read-through: This SIMD explicitly leaves the fee structure for larger txs as an open question. Imo it would be better to settle this before merging this SIMD, as otherwise we run the risk of determining a fee structure that requires some extra changes to the proposed v1 tx format / lock ourselves into the v1 format prematurely.

The scheduler is not maintained via the SIMD process today and it is in validator's best interests to pack the blocks in a way that makes them the most money. This SIMD stays in line with this thought process and does not recommend a structure or tie any direct fee or CUs to it.

Also, iiuc the proposed ResourceRequests field would then supercede #287 since the mask could specify CU budget & price right?

Yes, this would supercede the proposal in 287.

[jacobcreech]

im curious if this SIMD is a rubber stamp of something the core team already wants to do or if this is an actual proposal. The Solana team an community are much more thorough in how proposals are brought and I would at minimum expect some kind of analysis on how shoving 4kb through a normally 1kb pipe will affect the network.

@austbot This SIMD is an actual proposal that has been discussed at length with multiple Solana core engineers and contributors. More testing will follow shortly with public review, just need to hack together a new cluster with these changes.

[jacobcreech]

Can we add a field for the fee that the transaction pays in lamports? I think we should take advantage of the opportunity this new format presents to switch from pricing in microlamports_per_cu to one overall price. This would be simpler for developers.

@ptaffet-jump is this to move away from the CU pricing and just have that be something the scheduler decides?

We've also talked about the idea of a failure fee (or an in-protocol extra fee for success). This could be a use of the ResourceRequestMask, but I don't know that it fits the name exactly.

Could you elaborate on this idea a bit more? I am not privy to these discussions. How would this failure fee work?

[cryptopapi997]

The scheduler is not maintained via the SIMD process today and it is in validator's best interests to pack the blocks in a way that makes them the most money. This SIMD stays in line with this thought process and does not recommend a structure or tie any direct fee or CUs to it.

I was thinking more from the developer's point of view, as I'd imagine that ceteris paribus a scheduler would always choose a tx with a smaller size over a bigger one, so having a way to be able to tell the scheduler "I care about this large size tx landing and am willing to pay extra for it" is important. But yea, I suppose in the near term the existing CU price can sort of be used for this and if a dedicated system for this is added in the future it would probably be something along the lines of "lamports per tx byte" or a fixed fee like suggested by @ptaffet-jump, both of which would fit in the new ResourceRequest API.

[KirillLykov]

One note is that if we increase the max size of the transaction v1 but not for v0, on the network layer side we cannot distinguish these two versions. Because to know the version we need to deserialize. This will create a workflow when we increase the stream size for both, but later check the version and drop those transactions v0 which are too large. Alternatively, we can introduce two streamers on two ports -- one taking transactions v0 and the other transactions v1 which would require changes in gossip to advertise both.

[apfitzge]

Because to know the version we need to deserialize. This will create a workflow when we increase the stream size for both, but later check the version and drop those transactions v0 which are too large. Alternatively, we can introduce two streamers on two ports -- one taking transactions v0 and the other transactions v1 which would require changes in gossip to advertise both.

Might be harder on network side to do that for sure, but we don't need to fully deserialize. The first byte in the transaction will tell us if it is a v1 or one of the legacy/v0 formats.

[KirillLykov]

Because to know the version we need to deserialize. This will create a workflow when we increase the stream size for both, but later check the version and drop those transactions v0 which are too large. Alternatively, we can introduce two streamers on two ports -- one taking transactions v0 and the other transactions v1 which would require changes in gossip to advertise both.

Might be harder on network side to do that for sure, but we don't need to fully deserialize. The first byte in the transaction will tell us if it is a v1 or one of the legacy/v0 formats.

Sounds like cheap solution: we can discard chunks if accumulated size is larger than expected for this type of tx. Do you know by chance if this version byte already set to 0 for transaction v0? if we need to add this flag for transaction v0 format?

[mcintyre94]

Do you know by chance if this version byte already set to 0 for transaction v0?

It's a bit more complicated because of legacy txs (before versioned)

For v0 it's set to 128
This proposal would be 129
For legacy it's the number of signatures, which is implicitly <128

[apfitzge]

It's a bit more complicated because of legacy txs (before versioned)

For v0 it's set to 128
This proposal would be 129
For legacy it's the number of signatures, which is implicitly <128

This is mostly correct.
The TxV1 in this proposal changes where the version byte is, since we have trailing signatures instead of leading signatures. In either approach the version byte is the first byte of the "message", but now the "message" begins at first byte of packet instead of after signatures.

Why this works

Previously the first byte of the packets would be the compressed-u16 for the Vec size. The size of signatures (for valid packets) is always <= 13 given the current packet limitation, which means that the first bit of the compressed-u16 is always 0.
IIRC Trent said something about offline-signed-messaages having a first byte of 128, so we cannot use that...so we start our first byte as 129.

[mcintyre94]

The TxV1 in this proposal changes where the version byte is, since we have trailing signatures instead of leading signatures. In either approach the version byte is the first byte of the "message", but now the "message" begins at first byte of packet instead of after signatures.

Ah, thanks! I hadn't spotted that change. Nice that we avoid encoding signature length twice now too.

[ripatel-fd]

Suggested change

	Historically, messages transmitted over the network must not exceed the IPc6 MTU
	Historically, messages transmitted over the network must not exceed the IPv6 MTU

Also worth noting the effective MTU is practically 1500 everywhere, not IPv6 MTU. Solana has just implemented an IPv6 recommendation incorrectly while using IPv4 only. Slightly lower (~20 ish bytes) for Double Zero on datacenters that don't support jumbo frames on the path between the user's host and the DZ gateway.