Skip to content

Commit 31c358e

Browse files
committed
Merge branch 'smart-restart-proposal' of github.com:wen-coding/solana-improvement-documents into smart-restart-proposal
2 parents f65c6aa + bafaac5 commit 31c358e

33 files changed

+4850
-15
lines changed

.github/linter/customRules.ts

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ export const enforceHeaderStructure = {
3434
let token = filtered[index]
3535
tempHeadings = tempHeadings.filter(item => item !== token.line)
3636

37-
if (index + 1 >= filtered.length) {
37+
if (index >= filtered.length) {
3838
onError({
3939
lineNumber: 1,
4040
detail: `Expected heading \`${tempHeadings[0]}\` and none exists. Please follow the structure outlined in the Proposal Template.`,
@@ -99,7 +99,7 @@ export const enforceMetadataStructure = {
9999
})
100100

101101
Object.keys(frontMatter).forEach((key) => {
102-
if (!(requiredMetadata as any)[key] && !(optionalMetadata as any)[key]) {
102+
if (!(requiredMetadata as any)[key] && !optionalMetadata.includes(key)) {
103103
onError({
104104
lineNumber: 1,
105105
detail: `Front matter contains invalid metadata \`${key}\``,
@@ -119,9 +119,12 @@ const requiredMetadata = {
119119
created: {},
120120
}
121121

122-
const optionalMetadata = {
123-
feature: {},
124-
}
122+
const optionalMetadata = [
123+
"feature",
124+
"supersedes",
125+
"superseded-by",
126+
"extends",
127+
]
125128

126129
export const metadataSimdIsValid = {
127130
names: ["front-matter-has-simd"],
@@ -278,12 +281,13 @@ export const metadataStatusIsValid = {
278281

279282
const validStatus = [
280283
"Idea",
281-
"Draft",
284+
"Draft",
282285
"Review",
283286
"Accepted",
284287
"Stagnant",
285288
"Withdrawn",
286289
"Implemented",
290+
"Activated",
287291
]
288292

289293
if (!validStatus.includes(status)) {
@@ -295,4 +299,4 @@ export const metadataStatusIsValid = {
295299
})
296300
}
297301
},
298-
}
302+
}

README.md

Lines changed: 50 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,50 @@
1-
# Solana Improvement Documents
2-
Solana IMprovement Documents (SIMD) describe proposed and accepted changes to the Solana protocol.
1+
# Solana Improvement Documents (SIMDs)
2+
3+
The goal of the SIMD project is to standardize and provide high-quality
4+
documentation for Solana and its ecosystem. This repository tracks past and
5+
ongoing improvements to Solana in the form of Solana Improvement Documents
6+
(SIMDs).
7+
[SIMD-0001](https://github.com/solana-foundation/solana-improvement-documents/blob/main/proposals/0001-simd-process.md)
8+
governs the SIMD process.
9+
10+
## SIMD Types
11+
12+
SIMDs can be divided into the following categories:
13+
14+
- **Standard SIMDs**:
15+
Describe changes that affect most or all Solana implementations, such as:
16+
- **Core**:
17+
Changes affecting consensus or substantial changes to the validator.
18+
- **Networking**:
19+
Changes or substantial improvements to network protocol specifications.
20+
- **Interfaces**:
21+
Breaking changes around the client JSON RPC API specifications and standards.
22+
- **Meta SIMDs**:
23+
Describe a process surrounding Solana or propose a change to (or an event in)
24+
a process.
25+
26+
## Before You Begin
27+
28+
Before you write a SIMD, ideas MUST be thoroughly discussed and vetted on the
29+
[ideas section](https://github.com/solana-foundation/solana-improvement-documents/discussions/categories/ideas)
30+
within this
31+
[repo's disucssion page](https://github.com/solana-foundation/solana-improvement-documents/discussions).
32+
Read and review [SIMD-0001](https://github.com/solana-foundation/solana-improvement-documents/blob/main/proposals/0001-simd-process.md),
33+
which describes the SIMD process in detail.
34+
35+
This repository is for documenting standards and not for implementation help.
36+
For specific questions and concerns regarding SIMDs, it's best to discuss them
37+
in the [questions section](https://github.com/solana-foundation/solana-improvement-documents/discussions/categories/questions)
38+
of this [repo's disucssion page](https://github.com/solana-foundation/solana-improvement-documents/discussions).
39+
40+
## Access Policy
41+
42+
The SIMD repository has three levels of access, as detailed in
43+
[SIMD-0007](https://github.com/solana-foundation/solana-improvement-documents/blob/main/proposals/0007-access-policy.md):
44+
45+
1. Triage
46+
2. Write
47+
3. Maintain
48+
49+
To request access or report misuse, please follow the procedures outlined in
50+
SIMD-0007.

XXXX-template.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,11 @@ type: Core/Networking/Interface/Meta
88
status: Draft
99
created: (fill me in with today's date, YYYY-MM-DD)
1010
feature: (fill in with feature tracking issues once accepted)
11+
supersedes: (optional - fill this in if the SIMD supersedes a previous SIMD)
12+
superseded-by: (optional - fill this in if the SIMD is superseded by a subsequent
13+
SIMD)
14+
extends: (optional - fill this in if the SIMD extends the design of a previous
15+
SIMD)
1116
---
1217

1318
## Summary
@@ -39,6 +44,12 @@ to another Solana core contributor. The generally means:
3944
- Interaction with other features
4045
- Edge cases
4146

47+
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
48+
"SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this
49+
document are to be interpreted as described in [RFC
50+
2119](https://www.ietf.org/rfc/rfc2119.txt) and [RFC
51+
8174](https://www.ietf.org/rfc/rfc8174.txt).
52+
4253
## Impact
4354

4455
How will the implemented proposal impacts dapp developers, validators, and core contributors?

proposals/0001-simd-process.md

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,10 @@ simd: '0001'
33
title: Solana Proposal Process
44
authors:
55
- Jacob Creech (Solana Foundation)
6+
- Ben Hawkins (Solana Foundation)
67
category: Meta
78
type: Meta
8-
status: Draft
9+
status: Living
910
created: 2022-10-18
1011
---
1112

@@ -68,7 +69,7 @@ Proposals but apply to areas other than the Solana protocol itself. They may
6869
propose an implementation, but not to Solana's codebase; they often require
6970
community consensus and users are typically not free to ignore them. Examples
7071
include procedures, guidelines, changes to the decision-making process, and
71-
changes to the tools or environment used in Solana development. Any meta-EIP is
72+
changes to the tools or environment used in Solana development. Any meta-SIMD is
7273
also considered a Process Proposals.
7374

7475
## Proposal Lifecycle
@@ -79,9 +80,11 @@ The stages in a lifecycle of a proposal are as follows:
7980
- Draft
8081
- Review
8182
- Accepted
83+
- Living
8284
- Stagnant
8385
- Withdrawn
8486
- Implemented
87+
- Activated
8588

8689
```mermaid
8790
flowchart LR
@@ -101,6 +104,10 @@ flowchart LR
101104
Idea ---> Draft;
102105
Draft ---> Review;
103106
Review ---> Accepted;
107+
Accepted ---> Implemented;
108+
Implemented ---> Activated;
109+
Review ---> Living;
110+
Accepted ---> Withdrawn;
104111
105112
Draft ---> Stagnant;
106113
Review ---> Stagnant;
@@ -114,9 +121,10 @@ proposal author -- the reviewers, and the Solana Core Contributors.
114121

115122
Before you begin writing a formal proposal, you should vet your idea. Ask the
116123
Solana core community first if an idea is original to avoid wasting time on
117-
something that will be rejected based on prior research. It is thus recommended
118-
to discuss the proposal on the Solana Tech Discord under the #core-technology
119-
channel.
124+
something that will be rejected based on prior research. Be sure to post your
125+
ideas to the
126+
[SIMD ideas discussion page](https://github.com/solana-foundation/solana-improvement-documents/discussions/categories/ideas)
127+
and gather feedback before making your formal Proposal
120128

121129
### Draft
122130

@@ -128,7 +136,8 @@ is descriptive)
128136
- Fill in the proposal. Put care into the details: proposals that do not
129137
present convincing motivation, demonstrate lack of understanding of the
130138
design's impact, or are disingenuous about the drawbacks or alternatives tend
131-
to be poorly received.
139+
to be poorly received. Low quality proposals with limited engagement will be
140+
closed by SIMD repository maintainers.
132141
- Submit a pull request.
133142
- Now that your proposal has an open pull request, use the issue number of the
134143
PR to update the `XXXX-` prefix to the number.
@@ -158,6 +167,23 @@ far the most effective way to see a proposal through to completion: authors
158167
should not expect that other project developers will take on responsibility for
159168
implementing their accepted feature.
160169

170+
### Implemented
171+
172+
When all relevant teams have completed development of the SIMD's feature, the
173+
SIMD is "Implemented".
174+
175+
### Activated
176+
177+
A proposal will have the status Activated once it has been implemented,
178+
tested, and finally activated on mainnet beta.
179+
180+
### Living
181+
182+
A special status for SIMDs that are designed to be continually updated and not
183+
reach a state of finality. This includes most notably SIMD-1. This status must
184+
undergo extra scrutiny and review when updating the status from review to
185+
living.
186+
161187
### Stagnant
162188

163189
If a proposal reaches 6 months without activity, the proposal will be
Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
---
2+
simd: '0009'
3+
title: Lockout Violation Detection
4+
authors:
5+
- carllin
6+
- ashwinsekar
7+
- wencoding
8+
category: Standard
9+
type: Core
10+
status: Draft
11+
created: 2022-12-12
12+
feature: (fill in with feature tracking issues once accepted)
13+
---
14+
15+
## Summary
16+
17+
An algorithm designed to catch validators that violate lockout rules when
18+
voting.
19+
20+
## Motivation
21+
22+
Validators that violate lockout rules unfairly earn rewards and put cluster.
23+
consensus at risk
24+
25+
## Alternatives Considered
26+
27+
None
28+
29+
## New Terminology
30+
31+
None
32+
33+
## Detailed Design
34+
35+
Assume:
36+
37+
1. A database that we will store vote information ingested from
38+
gossip/turbine, which in turn will be consulted to detect lockout violations
39+
as follows.
40+
41+
2. Tracking of all SlotHashes on the canonical/rooted fork in the database
42+
for the last epoch.
43+
44+
The following sections will go over the possible lockout violations that will
45+
be detected and punished.
46+
47+
### Rooting a different fork
48+
49+
If there are any votes made that have a root `R` missing from the rooted
50+
SlotHashes list, slash them on the rooted fork since they have committed
51+
to a different fork
52+
53+
### Removing lockouts
54+
55+
Attempting to illegally *remove* a lockout in a newer vote that that should
56+
have still existed based on an older vote is a lockout violation. We detect
57+
this as follows:
58+
59+
1. For all non root slots `S` in each vote, track a range `(S, S + 2^n)` where
60+
`n` is the confirmation count. For each slot `S` we only have to track the
61+
greatest such lockout range in the database.
62+
63+
2. For each new vote `V`, for each slot `S` in the vote, lookup in the database
64+
to see if there's some range `(X, X + 2^n)` where `S` is in this range, but
65+
`X` is missing from the vote `V`. This is a lockout violation because this
66+
implies that the validator made a vote where `S` popped off `X`, but the
67+
lockout on `X` from an earlier vote should have prevented that from happening.
68+
69+
Note for each interval `(S, S + 2^n)` we also need to be able to lookup the
70+
vote. This is important for being able to pull up the votes later as part of a
71+
proof.
72+
73+
### Reducing lockout
74+
75+
Attempting to illegally *reduce* a lockout from an older vote in a newer vote
76+
is a lockout violation.
77+
78+
The tricky bit of this is determining that one vote `V` is *older* than
79+
another vote `V'`. To this end, we track a flag `is_vote_earlier` whose usage
80+
we will highlight in the protocol below.
81+
82+
1. We set `is_vote_earlier = false`.
83+
84+
2. For each newly seen vote `V` made by a validator `X`, for each slot `S` in
85+
`V`:
86+
- If `S` exists in the database:
87+
- Compare the lockout `L_V` in the vote `V` on `S` against the greatest
88+
lockout `L_D` in the database for that slot `S` made by that validator
89+
`X`.
90+
- If `L_V` < `L_D`, set `is_vote_earlier=true`.
91+
- If `L_V` == `L_D`, continue.
92+
- If `L_V > L_D`, check if the flag `is_vote_earlier=true`. If so,
93+
this implies this vote `V` was *older* than some vote `V'` that
94+
comitted the greater `S'` lockout to the database, yet `V` has a
95+
lesser lockout on `S`, which means the validator reduced lockout on
96+
`S` in a later vote. This is a lockout violation.
97+
- If `S` does not exist in the database, the above `Removing lockouts`
98+
section describes the protocol that will catch violations.
99+
100+
### Reducing roots
101+
102+
Reducing the root from earlier to later votes is a lockout violation. We detect
103+
this as follows:
104+
105+
1. For each validator we track a rooted set in the database. We can remove an
106+
interval `(S, S + 2^n)` from the database once the slot becomes a root add it
107+
to a rooted set for this validator, and any new votes < root also get added to
108+
rooted set.
109+
110+
2. When we see a vote with root N on the main fork, then we remove all
111+
intervals `(M, P)` where `M < N && P >= N` and add `M` to the rooted set.
112+
113+
So for example if we see:
114+
(Assume `{slot: confirmation count}` format)
115+
116+
- `{root: 0, 1: 4, 3: 3, 5: 2, 7: 1}`
117+
118+
- `{root: 5, 7: 1}`
119+
120+
- Then we add `{1, 3}` to rooted set and remove their intervals from the
121+
interval tree because both of those are `< 5`, but have lockouts that extend
122+
past `5`
123+
124+
Note here also that that artificially increasing your lockout is not a
125+
slashable offense (here we root 5 however 7 still has a conf count of 1),
126+
because adopting stricter lockout does not weaken commitment on any previously
127+
committed fork.
128+
129+
Thus, if we then later saw a vote:
130+
131+
- `{1: 2, 2: 1}` on a different fork we would say it's slashable because the
132+
lockout on 2 extended past a rooted slot 3 in the rooted fork, so 2 should have
133+
prevented the vote for 3 from being made, evicted
134+
- `{1: 2, 4: 1}` on a different fork, then because 3 was rooted and 3 does not
135+
exist in this vote, it's implied 3 was popped off. However, 3 was rooted so it
136+
couldn't have been popped off by 4, so that's slashable
137+
138+
Ordering here is tricky though, for instance what if we saw vote 2 then vote 1?
139+
We would retroactively have to add `{1,3}` to the rooted set
140+
141+
Also note here that evicted slots will not be added to the rooted set. For
142+
example, imagine:
143+
144+
- `{root: 0, 1: 3, 3: 2, 4: 1}`
145+
146+
- `{root: 0, 1: 4, 3: 3, 7: 2, 9: 1}`
147+
148+
- `{root: 7, 9: 2, 10: 1}`
149+
150+
Here we add `{1, 3}` to the rooted set, but 4 doesn't get added because
151+
`4 + 2^1 < 7`, so it does not overlap the root of `7`. This means the interval
152+
`(4, 4 + 2^1)` remains in the database. This is important because:
153+
154+
- If we see a vote `{root: 0, 1: 3, 3: 2, 5: 1}` on another fork, this is only
155+
known to be slashable by seeing this interval `(4, 4 + 2^1)` (because it
156+
doesn't include `4` in the vote, but `4's` lockout should have prevented it
157+
from being popped off)
158+
- We don't want to add `4` to the rooted set to prevent slashing a valid vote
159+
on a different fork like `{root: 0, 1, 3, 10}`. If `4` was present in the
160+
rooted set, we would report an error because `10` should not have popped off `4`
161+
162+
## Impact
163+
164+
Validators snitching on voting misbehavior will be more effective.
165+
166+
## Security Considerations
167+
168+
None
169+
170+
## Backwards Compatibility
171+
172+
Not applicable.

0 commit comments

Comments
 (0)