Automatic fungible token intialization

[nvdtf]

Context

Consider a case where Wallet API is used to manage thousands of custodial accounts sending and receiving multiple fungible tokens.

Problems

With current features:

1. New accounts created by Wallet API only support FLOW token, so each fungible token's vault should be added to the new account one by one by an external worker. This operation takes more time as the list of supported fungible tokens grow due to each API call resulting in a separate transaction. The total time to finalize a new user's account will be (n + 1) * x where n is the number of fungible tokens and x is Flow's average transaction seal time.

2. When new fungible tokens are introduced to a live system, an external worker has to loop through all the old accounts and add the new fungible token's vault.

Solutions

Goal

All accounts should support withdrawals/deposits for the list of fungible tokens defined in ENABLED_TOKENS. This eliminates the need for manual API calls to initialize each account for each fungible token.

• This new behaviour can be turned on using an environment variable: INIT_FUNGIBLE_VAULTS_ON_ACCOUNT_CREATION (default false)

Templated Cadence Script for New Account Creation

Currently, the new account creation Cadence script is grabbed from flow-go-sdk:

flow-wallet-api/accounts/service.go

Lines 362 to 366 in 9a47f3b

	flowTx, err := flow_templates.CreateAccount(
	publicKeys,
	nil,
	payer.Address,
	)

We can implement a custom templated Cadence script for account creation that contains the code to initialize all of the enabled fungible tokens and create the account in one batched transaction. This will require two changes to the create account function:

1. Use the new templating functions (introduced below) to generate the transaction script to create the new account.
2. On success, call InsertAccountToken to record all the vault associations in the database.

The database record prevents us from using the custom account creation script feature (SCRIPT_PATH_CREATE_ACCOUNT) to solve problem 1.

Retroactive Fungible Token Support Job

To solve problem 2, we introduce a new job that can be triggered and monitored through API endpoints. We think triggering the job automatically on boot might have unintended effects for unknowing users. We suggest:

1. GET /ops/missing-fungible-token-vaults/start Starts a job that:
   1. Fetches list of accounts that have missing vaults (compared to ENABLED_TOKENS) from the database. For each account:
   2. Generate and execute one batched transaction that adds all the missing vaults. The new templating functions (introduced below) can be used to generate the transaction script.
   3. On success, call InsertAccountToken to record all the vault associations in the database.
2. GET /ops/missing-fungible-token-vaults/stats Returns the total number of accounts with missing vaults from the database per token in ENABLED_TOKENS. This can be useful for checking how many accounts might be affected when the job is started or monitor it's progress.

• The job should be singleton. Subsequent calls to the endpoint will have no effect if it's currently running.
• Each account's batched initialization transaction can be executed asynchronously since different proposer/account keys are used for each transaction. The number of concurrent in-flight transactions is configured via OPS_WORKER_COUNT.

New Cadence Transaction Templating Functions

This is the current vault initialization transaction template:

flow-wallet-api/templates/template_strings/transactions.go

Lines 49 to 75 in 9a47f3b

	const GenericFungibleSetup = `
	import FungibleToken from "./FungibleToken.cdc"
	import TOKEN_DECLARATION_NAME from TOKEN_ADDRESS
	transaction {
	prepare(signer: AuthAccount) {
	let existingVault = signer.borrow<&TOKEN_DECLARATION_NAME.Vault>(from: /storage/TOKEN_VAULT)
	if (existingVault != nil) {
	panic("vault exists")
	}
	signer.save(<-TOKEN_DECLARATION_NAME.createEmptyVault(), to: /storage/TOKEN_VAULT)
	signer.link<&TOKEN_DECLARATION_NAME.Vault{FungibleToken.Receiver}>(
	/public/TOKEN_RECEIVER,
	target: /storage/TOKEN_VAULT
	)
	signer.link<&TOKEN_DECLARATION_NAME.Vault{FungibleToken.Balance}>(
	/public/TOKEN_BALANCE,
	target: /storage/TOKEN_VAULT
	)
	}
	}
	`

To support the needs of above we need to be able to:

1. Generate create account transactions that initialize any number of fungible tokens.
2. Generate transactions that add any number of fungible tokens to an existing account.

The above can be achieved by creating two template variables:

• {{ .Imports }} concatenated list of contract imports required for all fungible tokens.
• {{ .VaultInits }} concatenated code to check, initialize, and link all required fungible tokens.
  • {{ .VaultInitsWithCheck }} is the same as above but without the check. Checking existing vaults is not needed for new accounts since we know it doesn't exist.

Suggested batched account creation template:

import Crypto
// all the required imports
// import FungibleToken from "./FungibleToken.cdc" 
// import TOKEN_DECLARATION_NAME1 from TOKEN_ADDRESS1
// import TOKEN_DECLARATION_NAME2 from TOKEN_ADDRESS2
// ...
{{ .Imports }} 
  
transaction(publicKeys: [Crypto.KeyListEntry]) {
  prepare(signer: AuthAccount) {
    let account = AuthAccount(payer: signer)
    
    // add all the keys to the account
    for key in publicKeys {
      account.keys.add(publicKey: key.publicKey, hashAlgorithm: key.hashAlgorithm, weight: key.weight)
    }   
    
    // Contains concatenated code for every fungible token vault initialization
    //
    // account.save(<-TOKEN1_DECLARATION_NAME.createEmptyVault(), to: /storage/TOKEN1_VAULT) 
    // account.link<&TOKEN1_DECLARATION_NAME.Vault{FungibleToken.Receiver}>( 
    //   /public/TOKEN1_RECEIVER, 
    //   target: /storage/TOKEN1_VAULT 
    // )
    // account.link<&TOKEN1_DECLARATION_NAME.Vault{FungibleToken.Balance}>( 
    //   /public/TOKEN1_BALANCE, 
    //   target: /storage/TOKEN1_VAULT 
    //  )
    // ...
    {{ .VaultInits }}
}

Suggested batched vault initialization template for retroactive fungible token support job:

// all the required imports
// import FungibleToken from "./FungibleToken.cdc" 
// import TOKEN_DECLARATION_NAME1 from TOKEN_ADDRESS1
// import TOKEN_DECLARATION_NAME2 from TOKEN_ADDRESS2
// ...
{{ .Imports }} 
  
transaction { 
  prepare(account: AuthAccount) {   
    // Contains concatenated code for every fungible token vault initialization
    //
    // if account.borrow<&TOKEN1_DECLARATION_NAME.Vault>(from: /storage/TOKEN1_VAULT) == nil {
    //   account.save(<-TOKEN1_DECLARATION_NAME.createEmptyVault(), to: /storage/TOKEN1_VAULT) 
    //   account.link<&TOKEN1_DECLARATION_NAME.Vault{FungibleToken.Receiver}>( 
    //     /public/TOKEN1_RECEIVER, 
    //     target: /storage/TOKEN1_VAULT 
    //   )
    //   account.link<&TOKEN1_DECLARATION_NAME.Vault{FungibleToken.Balance}>( 
    //     /public/TOKEN1_BALANCE, 
    //     target: /storage/TOKEN1_VAULT 
    //    )
    // }
    // ...
    {{ .VaultInitsWithCheck }}
}

Testing

• All new endpoints need to be documented.
• All new code should have unit tests and integration tests as needed.
• Since this will be delivered to a large live deployment, the retroactive fungible token support job needs to be tested on at least 10k accounts with USDC.

[nvdtf]

was done #314