SIMD-0167: Loader-v4 #167

Lichtso · 2024-08-20T13:20:14Z

No description provided.

nickfrosty

I also noticed there are several typos on the word "throw"

proposals/0167-loader-v4.md

topointon-jump

Very nice to see this well-written, well-specified SIMD!

Would be great to get more detail on the migration path from v3 to v4.

topointon-jump · 2024-11-01T20:55:03Z

proposals/0167-loader-v4.md

+
+- loader-v3 clears the program proxy account (setting its size to zero)
+- loader-v3 transfers all funds from the programdata to the proxy account
+- loader-v3 gifts the program proxy account to loader-v4


How would the gifting work? Do we need to add a new instruction to the v3 loader? Will this upgrade mechanism be specified in a future SIMD?

I see three options here:

A migrate instruction for loader-v3 which the dApp devs trigger themselves

A migrate instruction for loader-v3 which we the protocol engineers trigger

A feature gate which scans the accounts DB and migrates all loader-v3 programs

I would prefer the second option as that allows us to keep the gathering of the loader-v3 program list and then slowly working through it off-chain and still makes sure that all loader-v3 programs get migrated.

Added a more detailed description of how this would work:
f2d4970

buffalojoec

This is looking great! I dropped a bunch of nits but overall I'm pretty happy with the architecture. Thanks for all of the back and forth!

The last piece I'm stuck on is the overloading of the withdraw instruction. I think we should bikeshed a bit about what something like an Erase instruction might look like.

proposals/0167-loader-v4.md

buffalojoec · 2025-07-22T17:02:14Z

proposals/0167-loader-v4.md

+  - Copy the chunk into the program account at the offset shifted by the
+  header size


Do we want to have any limit on chunk size that's explicit in the program?

I think we still need to come up with CU costs for all instructions. And that would then implicitly limit the amount you can copy.

I think we still need to come up with CU costs for all instructions

If we go that way, I would like to have JMP32 and more registers in SBF so that we could improve codegen a bit.

This is about the loaders instructions, not SBPF instructions.

proposals/0167-loader-v4.md

buffalojoec · 2025-07-22T17:05:52Z

proposals/0167-loader-v4.md

+  - Check there are at least three instruction accounts,
+  otherwise throw `NotEnoughAccountKeys`
+  - Check that program account and source account do not alias,
+  otherwise throw `AccountBorrowFailed`


IMO, this error sounds like a bug. I think it's better to let the user know their inputs were bad.

Suggested change

otherwise throw `AccountBorrowFailed`

otherwise throw `InvalidArgument`

This is what is generally emitted by the program runtime whenever borrows do alias. Thus it is consistent.

I still think it's misleading in this context. It's not a runtime violation, it's a program instruction specification violation, and the violation is literally within the arguments to the program. Furthermore, most devs are familiar with AccountBorrowFailed whenever they obtain a reference to input region data that may have been invalidated.

proposals/0167-loader-v4.md

allows uninitialized program accounts to be closed and rekeys the feature.

…retract in the "Motivation" section.

joncinque

Just a couple of fly-by comments, looking great overall!

joncinque · 2025-07-23T18:21:29Z

proposals/0167-loader-v4.md

+  - Check that the status stored in the program account is `NeverBeenDeployed`
+  or `Retracted`
+    otherwise throw `InvalidArgument`
+  - Charge program_length_in_bytes CUs


Accounts can be 10MB, but the max CUs for a transaction is 1.4M CUs. So all costs that use the program length will need to be reduced

Good point, should probably divide by cpi_bytes_per_unit

joncinque · 2025-07-23T18:27:49Z

proposals/0167-loader-v4.md

+    otherwise throw `InvalidArgument`
+  - Check that there are enough funds in the program account for rent
+  exemption of the new length, otherwise throw `InsufficientFunds`
+  - Set the length of the program account to the requested new size plus


Is it worth specifying special behavior in CPI to limit the new size to size_at_start_of_top_level_instruction + 10kb?

buffalojoec

Getting really close on my side. Based on our last discussion we wanted to add the following to SetAuthority:

If the program is Uninitialized will skip the existing authority check, set the new authority, and set the status to NeverBeenDeployed.
Programs that are Retracted can be finalized only if the program length is 0

buffalojoec · 2025-07-24T06:34:46Z

proposals/0167-loader-v4.md

+  - Enum variant `2u32`
+  - `u32` The new size after the operation.
+- Behavior:
+  - Charge 32 + new_size_in_bytes / cpi_bytes_per_unit CUs


Does this mean "new size of the ELF"? So, if I grow a 1,000 byte ELF to 1,005, I have to pay CUs on 1,005 bytes? Why not just charge the delta?

It is on purpose. The runtime has to memcpy all the existing data when reallocating a vector.

buffalojoec · 2025-07-24T06:38:43Z

proposals/0167-loader-v4.md

+  - Transfer lamports which are not needed for rent exemption from the
+  program account to the recipient account
+
+#### WithdrawAllLamports


I really think this instruction should have a crystal-clear name that describes what it's going to do to your program. WithdrawAllLamports is super easy to conflate with WithdrawExcessLamports, and someone could end up destroying their program.

Suggested change

#### WithdrawAllLamports

#### Erase

Or, more explicit:

Suggested change

#### WithdrawAllLamports

#### EraseAndWithdrawAllLamports

buffalojoec · 2025-07-24T06:41:56Z

proposals/0167-loader-v4.md

+- Instruction data:
+  - Enum variant `5u32`
+- Behavior:
+  - Charge 32 CUs


Very generous for ELF verification! Any thoughts on beefing this up a bit?

Another CU charge happens later, a few lines down

Lichtso force-pushed the loader-v4 branch from 8d0ec51 to ce5daeb Compare August 20, 2024 13:28

Lichtso mentioned this pull request Aug 20, 2024

Adjustments of loader-v4 anza-xyz/agave#2630

Merged

nickfrosty reviewed Aug 22, 2024

View reviewed changes

proposals/0167-loader-v4.md Outdated Show resolved Hide resolved

Lichtso force-pushed the loader-v4 branch from ce5daeb to 9fedd95 Compare August 22, 2024 18:25

github-actions bot mentioned this pull request Aug 26, 2024

Upstream Updates - Mon Aug 26 00:12:40 UTC 2024 smartcontractkit/chainlink-solana#831

Closed

Lichtso force-pushed the loader-v4 branch 12 times, most recently from fc20b4e to bd5ab89 Compare August 27, 2024 15:00

Lichtso force-pushed the loader-v4 branch from bd5ab89 to 0b3d4c5 Compare September 5, 2024 18:40

Lichtso mentioned this pull request Oct 8, 2024

SIMD-0181: Privileged Syscalls #181

Closed

topointon-jump approved these changes Nov 1, 2024

View reviewed changes

Lichtso force-pushed the loader-v4 branch from 434071b to a4f4116 Compare November 19, 2024 15:19

Lichtso mentioned this pull request Dec 30, 2024

SIMD-0164: ExtendProgramChecked loader-v3 instruction #164

Open

Lichtso force-pushed the loader-v4 branch 2 times, most recently from 1f95ac3 to a788388 Compare January 24, 2025 12:18

Lichtso mentioned this pull request Jan 24, 2025

Feature - Adds LoaderV4Instruction::Copy anza-xyz/agave#4613

Merged

joncinque mentioned this pull request Jan 27, 2025

Refactor - Makes loader-v4 the default anza-xyz/agave#2796

Merged

Lichtso mentioned this pull request Jan 28, 2025

Feature - Adds UpgradeableLoaderInstruction::Migrate anza-xyz/agave#4661

Merged

Lichtso force-pushed the loader-v4 branch 3 times, most recently from 05f0b13 to f2d4970 Compare January 29, 2025 15:09

buffalojoec reviewed Jul 22, 2025

View reviewed changes

Lichtso force-pushed the loader-v4 branch 4 times, most recently from 8450981 to 39d9a44 Compare July 22, 2025 18:34

Lichtso added 16 commits July 22, 2025 20:40

First draft

6e45206

Adds explicit finalization and next version forwarding.

a8d8a63

Clearifies delay visibility and deployment cooldown.

2c848cb

Further clarifications after implementing integration tests.

c8448d6

Changes the instruction Truncate to SetProgramLength.

5f362c3

Adds the Copy instruction.

ad40283

Adds loader-v3 instruction Migrate.

2571f83

Expands on how loader v3 to v4 migration works.

87b57e8

Review feedback

15c5636

Removes the deploy from source instruction and rekeys the feature.

3b5dd11

Migrates empty / closed loader-v3 programs to loader-v4 as well,

a97c758

allows uninitialized program accounts to be closed and rekeys the feature.

Removes signer flag from parameters of SetProgramLength.

aee9c74

Describes the removal of explicit buffer accounts and the purpose of …

03b920d

…retract in the "Motivation" section.

Factors out loader-v3 to loader-v4 migration into its own SIMD.

f359def

Removes the word "delete".

0e582c9

Adds "Adding a loader-v3 state" to "Alternatives Considered".

0f177e2

Lichtso force-pushed the loader-v4 branch 4 times, most recently from 1e1d81e to a41ad2b Compare July 23, 2025 16:58

joncinque reviewed Jul 23, 2025

View reviewed changes

Lichtso force-pushed the loader-v4 branch from a41ad2b to b777b48 Compare July 23, 2025 19:36

buffalojoec reviewed Jul 24, 2025

View reviewed changes

Suggestions from jstarry, buffalojoec and t-nelson

aa7c34f

Lichtso force-pushed the loader-v4 branch from b777b48 to aa7c34f Compare July 30, 2025 15:58

		- Copy the chunk into the program account at the offset shifted by the
		header size

	otherwise throw `AccountBorrowFailed`
	otherwise throw `InvalidArgument`

SIMD-0167: Loader-v4 #167

Are you sure you want to change the base?

SIMD-0167: Loader-v4 #167

Conversation

Lichtso commented Aug 20, 2024

Uh oh!

nickfrosty left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

topointon-jump left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Lichtso Jan 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

buffalojoec left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

joncinque left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

buffalojoec left a comment

Choose a reason for hiding this comment

Uh oh!

buffalojoec Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

topointon-jump left a comment •

edited

Loading

Lichtso Jan 16, 2025 •

edited

Loading

buffalojoec Jul 24, 2025 •

edited

Loading