Expands on how loader v3 to v4 migration works.

Author: Lichtso

proposals/0167-loader-v4.md

@@ -7,7 +7,9 @@ category: Standard
 type: Core
 status: Review
 created: 2024-08-15
-feature: TBD
+feature:
+  - 8Cb77yHjPWe9wuWUfXeh6iszFGCDGNCoFk3tprViYHNm
+  - EmhbpdVtZ2hWRGFWBDjn2i3SJD8Z36z4mpMcZJEnebnP
 ---
 
 ## Summary
@@ -45,13 +47,15 @@ None.
 
 ## Detailed Design
 
-The associated feature gate must:
+The feature gate `8Cb77yHjPWe9wuWUfXeh6iszFGCDGNCoFk3tprViYHNm` must:
 
-- add loader-v4 to the write lock demotion exceptions
 - enable loader-v4 `LoaderV411111111111111111111111111111111111` program
-management and execution
-- simultaneously disable new deployments on loader-v3
-(`BPFLoaderUpgradeab1e11111111111111111111111`),
+management and execution.
+- enable the loader-v3 `BPFLoaderUpgradeab1e11111111111111111111111`
+instruction `UpgradeableLoaderInstruction::Migrate`.
+
+An additional feature gate `EmhbpdVtZ2hWRGFWBDjn2i3SJD8Z36z4mpMcZJEnebnP`
+must disable new deployments on loader-v3,
 throwing `InvalidIstructionData` if `DeployWithMaxDataLen` is called.
 
 ### Owned Program Accounts
@@ -111,7 +115,7 @@ All program management instructions must cost 2000 CUs.
   - `[signer]` The authority of the program.
 - Instruction data:
   - Enum variant `0u32`
-  - `u32` Offset at which to write the given bytes
+  - `u32` Byte offset at which to write the given bytes
   - `[u8]` Chunk of the programs executable file
 - Behavior:
   - Check there are at least two instruction accounts,
@@ -125,14 +129,49 @@ All program management instructions must cost 2000 CUs.
   - Copy the chunk into the program account at the offset shifted by the
   header size
 
+#### Copy
+
+- Instruction accounts:
+  - `[writable]` The program account to copy to.
+  - `[signer]` The authority of the program.
+  - `[]` The program(data) account to copy from.
+- Instruction data:
+  - Enum variant `1u32`
+  - `u32` Byte offset at which to write
+  - `u32` Byte offset at which to read
+  - `u32` Length of the chunk to copy in bytes
+- Behavior:
+  - Check there are at least three instruction accounts,
+  otherwise throw `NotEnoughAccountKeys`
+  - Check that program account and source account do not alias,
+  otherwise throw `AccountBorrowFailed`
+  - Verify the program account
+  - Check the status stored in the program account is retracted,
+  otherwise throw `InvalidArgument`
+  - Check that the source account is owned by loader v1, v2, v3 or v4,
+  otherwise throw `InvalidArgument`
+  - and look-up the source header size:
+    - loader-v1: 0 bytes
+    - loader-v2: 0 bytes
+    - loader-v3: 45 bytes
+    - loader-v4: 48 bytes
+  - Check that the source end offset (sum of source offset and length) does
+  not exceed the maximum (source account length minus the source header size),
+  otherwise throw `AccountDataTooSmall`
+  - Check that the destination end offset (sum of destination offset and
+  length) does not exceed the maximum (program account length minus the loader-v4
+  header size), otherwise throw `AccountDataTooSmall`
+  - Copy the chunk between the program accounts at the offsets, each shifted by
+  the header size of their loader (account owner) respectively
+
 #### Truncate
 
 - Instruction accounts:
   - `[(signer), writable]` The program account to change the size of.
   - `[signer]` The authority of the program.
   - `[writable]` Optional, the recipient account.
 - Instruction data:
-  - Enum variant `1u32`
+  - Enum variant `2u32`
   - `u32` The new size after the operation.
 - Behavior:
   - Check there are at least two instruction accounts,
@@ -177,7 +216,7 @@ All program management instructions must cost 2000 CUs.
   - `[writable]` Optional, an undeployed source program account to take data
   and lamports from.
 - Instruction data:
-  - Enum variant `2u32`
+  - Enum variant `3u32`
 - Behavior:
   - Check there are at least two instruction accounts,
     otherwise throw `NotEnoughAccountKeys`
@@ -212,7 +251,7 @@ All program management instructions must cost 2000 CUs.
   - `[writable]` The program account to retract.
   - `[signer]` The authority of the program.
 - Instruction data:
-  - Enum variant `3u32`
+  - Enum variant `4u32`
 - Behavior:
   - Check there are at least two instruction accounts,
     otherwise throw `NotEnoughAccountKeys`
@@ -232,7 +271,7 @@ All program management instructions must cost 2000 CUs.
   - `[signer]` The current authority of the program.
   - `[signer]` The new authority of the program.
 - Instruction data:
-  - Enum variant `4u32`
+  - Enum variant `5u32`
 - Behavior:
   - Check there are at least three instruction accounts,
     otherwise throw `NotEnoughAccountKeys`
@@ -250,7 +289,7 @@ All program management instructions must cost 2000 CUs.
   - `[signer]` The current authority of the program.
   - `[]` Optional, the reserved address for the next version of the program.
 - Instruction data:
-  - Enum variant `5u32`
+  - Enum variant `6u32`
 - Behavior:
   - Check there are at least three instruction accounts,
     otherwise throw `NotEnoughAccountKeys`
@@ -271,6 +310,50 @@ All program management instructions must cost 2000 CUs.
   the previous versions program account
   - Change the status stored in the program account to finalized
 
+### Loader-v3 Migration Instruction
+
+- Instruction accounts:
+  - `[writable]` The program data account.
+  - `[writable]` The program account.
+  - `[signer]` The migration authority.
+- Instruction data:
+  - Enum variant `8u32`
+- Behavior:
+  - Check that there are at least three instruction accounts,
+  otherwise throw `NotEnoughAccountKeys`
+  - Check that the provided authority is the migration authority
+  (pubkey is `3Scf35jMNk2xXBD6areNjgMtXgp5ZspDhms8vdcbzC42`),
+  otherwise throw `IncorrectAuthority`
+  - Check that the provided authority is a signer,
+  otherwise throw `MissingRequiredSignature`
+  - Check that the program data account is writable,
+  otherwise throw `InvalidArgument`
+  - Check that the program data account is not a buffer or a program account,
+  meaning it can contain program data or be uninitialized / closed / empty,
+  otherwise throw `InvalidArgument`
+  - Check that the program data account was last modified before the current
+  slot, otherwise throw `InvalidArgument`
+  - Check that the program account is writable,
+  otherwise throw `InvalidArgument`
+  - Check that the program account is owned by loader-v3,
+  otherwise throw `IncorrectProgramId`
+  - Check that the program account points to the program data account,
+  otherwise throw `InvalidArgument`
+  - Clear the program account (setting its size to zero)
+  - Transfer all funds from the program data account to the program account
+  - Sets the owner of the program account to loader-v4
+  - CPI loader-v4 `Truncate` the program account to the size of the program
+  data account minus the loader-v3 header and use the migration authority
+  - If the program data account was not closed (size was greater 0):
+    - CPI loader-v4 `Copy` the program data account into the program account
+    - CPI loader-v4 `Deploy` the program account
+  - If the program data account was not finalized:
+    - CPI loader-v4 `TransferAuthority` to the original upgrade authority
+  - otherwise if the program data account was finalized:
+    - CPI loader-v4 `Finalize` without a next version forwarding
+  - The program data account still owned by loader-v3 is under-funded and will
+  be automatically deleted by rent collection at the end of the transaction
+
 ## Impact
 
 This proposal:
@@ -290,35 +373,29 @@ exception when shortening the length of program accounts or closing them.
 instruction, instead of having to build and redeploy an empty program.
 - properly alignes the executable file relative to the beginning of the
 account. In loader-v3 it is misaligned.
-- once all loader-v3 programs are migrated:
-  - allows transaction account loading to be simplifed, because every program
-  would load exactly one account, no need to load the proxy account to get to
-  the actual program data (which is not listed in the transaction accounts).
-  - allows the removal of the write lock demotion exception if loader-v3 is
-  present in a transaction.
-  - corrects the miscounting of the proxy account size towards the total
-  transaction account loading limit.
+
+Once loader-v4 is enabled and new programs can not be deployed on loader-v3
+anymore, the list of all loader-v3 programs becomes fixed and can be extracted
+from a snapshot. Using the added loader-v3 migration instruction and the global
+migration authority, the core protocol developers will then migrate all
+loader-v3 programs to loader-v4 programs, which once completed:
+
+- allows transaction account loading to be simplifed, because every program
+would load exactly one account, no need to load the proxy account to get to
+the actual program data (which is not listed in the transaction accounts).
+- allows the removal of the write lock demotion exception if loader-v3 is
+present in a transaction.
+- corrects the miscounting of the program data account size towards the total
+transaction account loading limit.
+- allows dApp devs to resuscitate closed loader-v3 programs if they still
+control the program authority. This allows redeployment at the same address
+or completely closing the program account in order to retrieve the locked
+funds.
 
 ## Security Considerations
 
 None.
 
 ## Backwards Compatibility
 
-This proposal does not break any existing programs. However, dapp developers
-might want to profit from the new program mangement instructions without
-influencing their users work flows. To do so they would need a way to turn the
-program accounts of loader-v3 to program accounts of loader-v4, changing the
-account owner but keeping the program address. A potential issue is that the
-programdata header of loader-v3 is only 45 bytes long while loader-v4 takes 48
-bytes. An automatic mechanism in the program runtime (triggered by feature
-activation) could then perform the following steps per program:
-
-- loader-v3 clears the program proxy account (setting its size to zero)
-- loader-v3 transfers all funds from the programdata to the proxy account
-- loader-v3 gifts the program proxy account to loader-v4
-- loader-v4 initializes it via `Truncate`
-- loader-v4 copies the data from the programdata account via `Write`
-- loader-v4 deploys it via `Deploy`
-- Optinally, loader-v4 finalizes it without a next version forwarding
-- loader-v3 closes the programdata account (setting its size to zero)
+None.