Ledger Entry Change Events

[JakeUrban]

CAP-67 or "Unified Asset Events" was proposed to address this problem:

Tracking the movement of Stellar assets today is complex because you need to consume both Soroban events emitted by the Stellar Asset Contract and ledger entry changes for Classic operations.

This problem was worth solving because most applications on Stellar needed to track asset movements executed using classic operations.

I'd like to propose a protocol change that tackles the generalized problem, which is:

Tracking changes to ledger entries is complex because you need to consume both Soroban events and LedgerEntryChange XDR structs for classic operations

This problem is worth solving for similar reasons to why we introduced CAP-67: many applications on Stellar need to track more than just asset movements.

CAP-67 defined events to be emitted when changes were made to AccountEntry.balance and TrustlineEntry.balance attributes. I am proposing that Stellar emit events for any change to any ledger entry, excluding changes to ledger entries for which we already emit events (ex. ContractDataEntry).

The objective of this proposal is to ensure all applications can track any change to the ledger solely through events.

[JakeUrban]

Alternatively, we can propose CAPs that define events for subsets of ledger entry changes, similar to CAP-67. For example, we can propose a CAP that emits events for changes specific to TrustlineEntry structs.

However, I would advocate for emitting events for all possible ledger entries changes, potentially with the exception of changes to ledger entries that we plan to deprecate or remove from the protocol.

[leighmcculloch]

I am proposing that Stellar emit events for any change to any ledger entry

Would the goal be to emit 1️⃣ a generic ledger-entry-changed event that then contained a diff of the fields that changed for that entry? Or 2️⃣ are you meaning emitting domain specific events for all the conceptual reasons that ledger entries change, such as events for the creation or claiming of claimable balances?

I'm 👍🏻 for adding more events to capture happenings on Stellar outside contracts. But I think we need to be specific. If we aren't specific the risk is not just about potentially emitting unnecessary events, but that we'll structure new events in ways that don't align well with their practical uses.

If the goal is to emit 2️⃣, could you share a list of events alongside their topics and data? Something like this doc we did for classic asset events.

Fyi there's a super discussion that is tracking all proposals for new events, and once this proposal becomes more clear we should add it to the list:

• New Events #1647

[dmkozh]

I am proposing that Stellar emit events for any change to any ledger entry, excluding changes to ledger entries for which we already emit events (ex. ContractDataEntry).

This is an important caveat: there is no 1:1 mapping between the contract data entries and the events emitted. Meta is still necessary to track granular changes (e.g. for block explorers). Events are needed to highlight the relevant changes.

Tracking changes to ledger entries is complex because you need to consume both Soroban events and LedgerEntryChange XDR structs for classic operations

This is not an entirely correct statement: it's sufficient to only look at the meta in order to get all the ledger entry changes. As I've pointed above, the mapping between the events and entry changes is not 1:1 in a general case. So tracking the events and tracking the ledger entry changes achieves somewhat different goals.

many applications on Stellar need to track more than just asset movements.

I think we need to specify what exactly needs to be tracked. Unified events made sense because they fit into SEP-41 data model for the most part. But in general events are an abstraction layer over the data storage, so designing them should correspond to the intended use cases and not the the internal data layout. We could of course just emit ledger entry diffs in the event format, but that wouldn't achieve the same goals as token events did.

[JakeUrban]

Would the goal be to emit 1️⃣ a generic ledger-entry-changed event that then contained a diff of the fields that changed for that entry? Or 2️⃣ are you meaning emitting domain specific events for all the conceptual reasons that ledger entries change, such as events for the creation or claiming of claimable balances?

I'm not particularly opinionated on this as long as we accomplish the following:

The objective of this proposal is to ensure all applications can track any change to the ledger solely through events.

My loosely held preference is for 2️⃣, because consuming a generic ledger-entry-changed event doesn't improve the developer experience compared to consuming the ledger-entry-change XDR structs. Clients have to parse the change struct, determine the type, then iterate over the attributes they care about to determine what has changed.

In my experience, developers parse the operations themselves or the OperationResult XDR structs instead of the LedgerEntryChange, and I suspect that is because they offer a domain-specific view.

If we emit domain-specific events, clients are told what the "type" is through the topic, and the data is limited to what has changed, not the full set of attributes for the entry struct.

I can make an attempt at defining the events, but I suspect there will be a lot of debate on these definitions so I'd like to first make sure we're in agreement of the goals / design philosophy before proceeding.

Another reason to be in favor of 2️⃣ is something @anupsdf and @MonsieurNicolas have mentioned: as TPS increases, the size of txmeta will become unsustainable. Emitting a comprehensive set of events could serve as a replacement for LedgerEntryChange structs entirely. We need to decide whether this is an explicit goal for this proposal or not (someone from core/data/platform should weigh in on this).

This is an important caveat: there is no 1:1 mapping between the contract data entries and the events emitted. Meta is still necessary to track granular changes (e.g. for block explorers). Events are needed to highlight the relevant changes.

Agreed, no need to emit events for contract data changes since events are already emitted by the contract to describe the relevant changes to clients.

the mapping between the events and entry changes is not 1:1 in a general case

Also agreed. Lets use "adding a signer to an account" as an example. I think this should result in something like:

contract: ?, topics: ["account", "G...", "signers", "added"], data: {public_key: G..., weight: 1}

[leighmcculloch]

I'm in favour of 2️⃣ as well, for the same reasons @dmkozh shared above:

events are an abstraction layer over the data storage, so designing them should correspond to the intended use cases and not the the internal data layout.

[dmkozh]

contract: ?, topics: ["account", "G...", "signers", "added"], data: {public_key: G..., weight: 1}

FWIW account and contract can be used interchangeably; G-accounts can be considered a built-in custom account in the context of Soroban. So your example would look like

contract: G.., topics: ["signer_added"], data: {public_key: G..., weight: 1}

Emitting a comprehensive set of events could serve as a replacement for LedgerEntryChange structs entirely. We need to decide whether this is an explicit goal for this proposal or not

I don't think this should be a goal. Events aren't necessarily the most space-efficient format, and block explorers would probably still want the actual entry changes both for the sake of backwards compatibility and for the sake of tracking entries not covered by the events (such as Soroban entries). I think that compressing the meta is a separate goal; but for non-block-explorer users I agree that it would make sense to strive for making meta not necessary.

[JakeUrban]

Ok, I'll take a stab at defining the events then.

@dmkozh on your suggested change to the event I provided:

• are there any cases where the state change isn't associated with a particular stellar account or contract? I can't think of any but want to double check.
• are you in favor of reducing the topics to a size of 1? I defined topics array as ["account", "signers", "added"] so that applications could match on all "account" events for example, rather than the more specific "signers_added". I'm not opinionated on this but am curious on why you suggested the change.

[dmkozh]

are there any cases where the state change isn't associated with a particular stellar account or contract? I can't think of any but want to double check.

I guess it depends on which operations/entries we go for and on the semantics. I would prefer to exclude some entries like data, but if we were to go for these, we'd need to get creative with the data schema.

are you in favor of reducing the topics to a size of 1?

I don't have a particular preference, but my frame of reference is the existing events (like SEP-41 or NFT events), and these never have 'hierarchical' event names as the one you've suggested. But again, in the end this largely depends on the use case, like whether it is useful to index/query these topics separately.

[JakeUrban]

Ok, @leighmcculloch @dmkozh PTAL. I'm not attached to this approach but this was what made the most sense to me on the first pass. I'm sure there are several aspects of this that need correction / adjustment!

Rationale

• The contract value will be the public key of the Stellar account associated with the ledger entry that was created, updated, or deleted
• The first topic will be the type of ledger entry being created, updated, or deleted
• The second topic will be the type of change (created, updated, or deleted)
• The data value will be a ScVal representing a map, where each map will have two top-level keys:
  • one of created, updated, or deleted: the values for each key will be a map, where each entry in the map is an attriute of the ledger entry
  • meta: a set of key-value pairs for additional attributes relevant to the ledger entry change
• Attributes of ledger entries that were not created, updated, or deleted will not be present in the data value
• Events for the following are out of scope:
  • asset events: covered in CAP-67
  • contract-related entries: events are already emitted to represent relevant changes for contract data, code, & ttl entries
  • config-related entries: events are alreadya emitted to represent relevant changes for config entires

Events

Account Created

contract: G...
topics: ["account", "created"]
data: {
    "meta": {
        "funder": "G..."
    }, 
    "created": {
        "account_id": "G...", 
        "balance": "1", 
        "seq_num": ""
        ... # remaining attributes for AccountEntry XDR structs
    }
}

Account Deleted (merged)

contract: G...
topics: ["account", "deleted"]
data: {
    "meta": {
        "to": "G..."
    },
    "deleted": {
        "account_id": "G..."
        ... # remaining attributes for AccountEntry XDR structs
    }
}

Account Updated

Example: Set Options to Add Signer and Updated Medium Threshold

contract: G...
topics: ["account", "updated"]
data: {
    "meta": {},
    "updated": {
        "seq_num": "", # only present if the account was also the source of the transaction
        "num_subentries": "3", # the updated number of subentries, because the signer below was added
        "signers": [{"signer": "G...", "weight": "1"}] # only the signer that was added, updated, or removed is present
        "thresholds": {"med": "2"} # only the threshold that was changed, if any, is present
    }
}

Trustline Created

contract: G...
topics: ["trustline", "created"]
data: {
    "meta": {
        "asset": "",
        "sponsor": "G..." # only if the base reserve for the trustline was sponsored
    },
    "created": {
        "account_id": "G...",
        "asset": "",
        "flags": {
            "authorized": true,
            "authorized_to_maintain_liabilities": true,
            "clawback_enabled": false
        },
        ... # remaining attributes for TrustlineEntry XDR structs
    }
}

Trustline Updated

Example: SetTrustlineFlags to Deauthorize Trustline

contract: G...
topics: ["trustline", "updated"]
data: {
    "meta": {
        "asset": "",
    },
    "updated": {
        "flags": {
            "authorized": "false"
        }
    }
}

Offer Entry Created

contract: G...
topics: ["offer", "created"]
data: {
    "meta": {
        "offer_id": "123",
    },
    "created": {
        "seller_id": "G...",
        "offer_id": "123",
        "selling": {
            "asset_type": "credit_alphanum4",
            "asset_code": "USD",
            "asset_issuer": "G...",
            "amount": "1000000000"
        },
        "buying": {
            "asset_type": "native",
            "amount": "500000000"
        },
        "price": {
            "n": "1",
            "d": "2"
        },
        ... # remaining OfferEntry attributes
    }
}

Offer Entry Updated

Example: Update Offer Price and Amount

contract: G...
topics: ["offer", "updated"]
data: {
    "meta": {
        "offer_id": "123",
    },
    "updated": {
        "selling": {
            "amount": "2000000000" # only the changed amount
        },
        "buying": {
            "amount": "1000000000" # only the changed amount
        },
        "price": {
            "n": "2",
            "d": "3" # only the changed price
        }
    }
}

Offer Entry Deleted (pulled or crossed)

contract: G...
topics: ["offer", "deleted"]
data: {
    "meta": {
        "offer_id": "123",
        "reason": "pulled" # or "crossed" if the offer was crossed
    },
    "deleted": {
        "seller_id": "G..."
    }
}

Claimable Balance Created

contract: G...
topics: ["claimable_balance", "created"]
data: {
    "meta": {
        "sponsor": "G..." # only if sponsored
        "balance_id": "00000000...",
    },
    "created": {
        "balance_id": "00000000...",
        "claimants": [
            {
                "destination": "G...",
                "predicate": {
                    "type": "and",
                    "and": [
                        {
                            "type": "not",
                            "not": {
                                "type": "abs_before",
                                "abs_before": "2025-12-31T23:59:59Z"
                            }
                        },
                        {
                            "type": "rel_before",
                            "rel_before": "2592000"
                        }
                    ]
                }
            }
        ],
        "asset": {
            "asset_type": "credit_alphanum4",
            "asset_code": "USD",
            "asset_issuer": "G...",
        },
        "amount": "1000000000"
        "ext": {
            "v": 1,
            "v1": {
                "flags": {
                    "transferable": true,
                    "clawback_enabled": false
                }
            }
        }
    }
}

Liquidity Pool Created

contract: G...
topics: ["liquidity_pool", "created"]
data: {
    "meta": {
        "liquidity_pool_id": "00000000...",
    },
    "created": {
        "liquidity_pool_id": "00000000...",
        "type": "constant_product",
        "constant_product": {
            "asset_a": {
                "asset_type": "credit_alphanum4",
                "asset_code": "USD",
                "asset_issuer": "G...",
                "amount": "1000000000"
            },
            "asset_b": {
                "asset_type": "native",
                "amount": "500000000"
            },
            "fee": {
                "n": "30",
                "d": "10000"
            }
        },
        "total_trustlines": "2",
        ... # remaining attributes for LiquidityPoolEntry
    }
}

Liquidity Pool Updated

Example: Update Pool Reserves and Shares After Trade

contract: G...
topics: ["liquidity_pool", "updated"]
data: {
    "meta": {
        "liquidity_pool_id": "00000000...",
    },
    "updated": {
        "reserve_a": "900000000", # only the changed reserve
        "reserve_b": "600000000", # only the changed reserve
        "total_shares": "1100000000", # only the changed total shares
    }
}

Data Entry Created

contract: G...
topics: ["data", "created"]
data: {
    "meta": {
        "sponsor": "G..." # only if sponsored
    },
    "created": {
        "data_name": "profile",
        "data_value": "eyJ1c2VybmFtZSI6ICJqb2huX2RvZSIsICJlbWFpbCI6ICJqb2huQGV4YW1wbGUuY29tIn0=",
    }
}

Data Entry Updated

Example: Update Data Value

contract: G...
topics: ["data", "updated"]
data: {
    "meta": {},
    "updated": {
        "data_name": "profile",
        "data_value": "eyJ1c2VybmFtZSI6ICJqb2huX2RvZSIsICJlbWFpbCI6ICJqb2huQGV4YW1wbGUuY29tIiwgInBob25lIjogIisxMjM0NTY3ODkwIn0="
    }
}

Data Entry Deleted

contract: G...
topics: ["data", "deleted"]
data: {
    "meta": {},
    "deleted": {
        "data_name": "profile"
    }
}

[JakeUrban]

I don't think this would happen. Or, rather, this can already happen today if you try to track balance movement via both ledger entry diffs and the unified events. The events and entry diffs are meant to be exclusive and serve different purposes.

I mean, we're working with a client that needs to monitor for accounts and trustlines being created, and they also use events to monitor value movements, so if ledger entry diffs and unified events are meant to be exclusive, we're not seeing clients treat them as such. I'd argue that this indicates a change should be made such that each can be used exclusively for the purposes they were meant to serve.

What are the different purposes?

My understanding is that events were designed to be domain-specific, and their purpose is to be consumed by clients interested in relevant changes to the associated state.

However, its not clear to me what the purpose of ledger entry changes are, if its purpose is not the same as events. I recognize they describe changes to Stellar's underlying data storage, but who is that useful to if its not the same audience as consumers of events?

[dmkozh]

we're working with a client that needs to monitor for accounts and trustlines being created, and they also use events to monitor value movements

That seems kind of weird to me, given that value movements are reflected in the respective account and trustline entries. But weirdness aside, they still track the value separately from the entry changes, so I don't see an issue here. The concern you had there was that a client would ingest the balance change twice: once from events, once from the entry diffs. That concern already exists, and it doesn't seem too hard to work around.

What are the different purposes?

Token events track value movements and have narrowly defined semantics. Ledger diffs just track the internal storage state and can be interpreted in an application-specific fashion.

My understanding is that events were designed to be domain-specific, and their purpose is to be consumed by clients interested in relevant changes to the associated state.

Event purpose is to abstract away the state implementation (or sometimes to not store the state on-chain at all). Entry diffs represent the state changes without any semantics attached to them (like it's not possible to tell from a balance change what exactly has caused a balance change; one needs to analyze the full transaction to understand that). It's sometimes possible to infer one from another, but that's not a hard requirement.

Basically the end goals might be the same (learn something about the on-chain state), but the approach and the user expectations are quite different (as I've mentioned above, for the events I expect that if I index the topics, I should be able to efficiently answer most, if not all the questions I have; that's not the case for the ledger diffs). That's why I'm not a fan of the idea of just converting the ledger diffs into the event-compatible format and would rather prefer the events to follow the existing philosophy (like the sequence number example I've provided above).

[leighmcculloch]

events were designed to be domain-specific, and their purpose is to be consumed by clients interested in relevant changes to the associated state.

Events describe something happened, but they don't actually need to have anything to do with state. It's reasonable for some contracts to emit events to inform off-chain systems that something happened, without everything in that event being captured in state on chain.

---

For account creation and an account trusting an asset, I think it could be reasonable to add domain events for these. The fact that storage is also being created is just happenstance. I don't think we need to come up with a story for how to emit events for all storage changes, but rather add the domain events that the use cases are indicating are important. Some of those events may overlap with storage being created or removed, but the storage isn't what's driving the need for the event.

@JakeUrban So that we can better understand the "account creation" and "account trusts an asset" domain events, what will wallets do when they see these events?

[JakeUrban]

The partner we're working with needs the has the following flow:

• client requests that the partner provide an account to facilitate a deposit into their platform
• client constructs a transaction with the following schema:
  • fee bump transaction
  • sponsoring reserves
  • create account
  • add trustline for the asset the client wants to deposit
• provide deposit address in response to client

The partner needs to track the accounts and trustlines created, as well as the fees paid and reserves sponsored for each client so they can retroactively bill them. They're using events as the primary mechanism for tracking onchain activity (transfers, deauthorizations, clawbacks, all through the SAC), so it would be ideal for them if account creation, trustline creation, and sponsorship data was provided through events emitted as well.

[dmkozh]

FWIW the fact of account and trustline creation could be tracked within the scope of the token events, if trustline creation emitted e.g. a 0-value self transfer of the respective asset. Then the logic could be 'the first incoming transfer to the address creates an account/trustline'. Not saying we should do that, just brainstorming. But the benefit is that we don't introduce any new entities.
We could also emit just the 'account created' and 'trustline created' events and it's not clear if they need to contain any data payload at all.
Not sure about what kind of information is necessary for the sponsorship data though.

[mollykarcher]

This has been suggested in the above thread ^^ but it's getting long so pulling this conversation out into another thread. I too, think we should tackle this in a very specific, use-case driven way, rather than a generic/sweeping implementation that will likely mirror what we have today (a la, horizon /effects). IMO, this means we should have separate, targeted SEPs (+ corresponding CAPs) that define the structures of these events.

For example, the first couple event structures defined above are around account create/update/delete. We currently don't have any existing SEPs defined relating to contract accounts, so perhaps it is best to start there. What events would be generally useful for accounts to emit and downstreams to process, regardless of whether they are contract accounts or classic accounts? Those events should probably be codified in a SEP, and then we'd have a corresponding CAP which effectively implements the SEP for classic/built-in accounts, similar to the relationship between SEP-41 and CAP-67. I would imagine we would continue to follow this pattern. The community agrees on a SEP for a specific use-case, and then "classic" is kind of like the protocol built-in contract that we then update to implement the standard.

Another are could be that defi discussion referenced above. We had actually internally taken a pass at defining part of that data model (see here - cc @karthikiyer56 ) at one point.

So I guess I'm saying, maybe it's best to break up this discussion into multiple, smaller, targeted use-case based SEPs/discussions.

[leighmcculloch]

targeted SEPs (+ corresponding CAPs)

What events would be generally useful for accounts to emit and downstreams to process, regardless of whether they are contract accounts or classic accounts?

These are key points. Events that can be omitted by both classic operations and contracts create greater possibility of innovation, and allow the ecosystem to create new implementations with new behaviours. Events will be most useful when they work irrespective of whether they are implemented by a classic operation or a contract.