Skip to content

Commit eb3a0eb

Browse files
committed
Add DistributedReplicatedLog example.
The property `IsSync` in `DistributedReplicatedLog.tla` does not hold; the node with the longest log may forever append a value before the other nodes can catch up. TLC finds the following (valid) counterexample (for three nodes): ```tla Error: Temporal properties were violated. Error: The following behavior constitutes a counter-example: State 1: <Initial predicate> cLogs = (n1 :> <<>> @@ n2 :> <<>> @@ n3 :> <<>>) State 2: <Extend(n2) line 35, col 5 to line 38, col 49 of module DistributedReplicatedLog> cLogs = (n1 :> <<>> @@ n2 :> <<v>> @@ n3 :> <<>>) State 3: <Extend(n2) line 35, col 5 to line 38, col 49 of module DistributedReplicatedLog> cLogs = (n1 :> <<>> @@ n2 :> <<v, v>> @@ n3 :> <<>>) State 4: <Copy(n3) line 26, col 9 to line 32, col 96 of module DistributedReplicatedLog> cLogs = (n1 :> <<>> @@ n2 :> <<v, v>> @@ n3 :> <<v>>) Back to state 2: <Copy(n1) line 26, col 9 to line 32, col 96 of module DistributedReplicatedLog> ``` Signed-off-by: Markus Alexander Kuppe <[email protected]>
1 parent f7d436b commit eb3a0eb

File tree

4 files changed

+120
-1
lines changed

4 files changed

+120
-1
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
This spec was inspired by https://github.com/microsoft/CCF/blob/main/tla/consensus/abs.tla.
2+
3+
This spec has a machine-closed fairness constraint, which differs from the the CRDT and
4+
ReplicatedLog examples. However, this spec assumes that a server can consistently read the
5+
state of all other servers, which is clearly not a realistic assumption for a real
6+
distributed system. A real system would rely on some messaging protocol to determine the
7+
lag between servers (compare Raft).
8+
9+
---- MODULE DistributedReplicatedLog ----
10+
EXTENDS Sequences, SequencesExt, Integers, FiniteSets, FiniteSetsExt
11+
12+
CONSTANT Lag, Servers, Values
13+
ASSUME Lag \in Nat /\ IsFiniteSet(Servers)
14+
15+
VARIABLE cLogs
16+
vars == <<cLogs>>
17+
18+
TypeOK ==
19+
/\ cLogs \in [Servers -> Seq(Values)]
20+
21+
Init ==
22+
/\ cLogs \in [Servers -> {<< >>}]
23+
24+
Copy(i) ==
25+
\E j \in Servers:
26+
/\ Len(cLogs[j]) > Len(cLogs[i])
27+
/\ \* Sync some prefix up to prefix = suffix of the unsynced suffix.
28+
LET L == (Len(cLogs[j]) - Len(cLogs[i]))
29+
\* Force to proportionally to the lag L copy more.
30+
\* Lag: 1 -> 0..L, 2 -> 1..L, 3 -> 2..L
31+
IN \E l \in L-1 .. L:
32+
cLogs' = [cLogs EXCEPT ![i] = @ \o SubSeq(cLogs[j], Len(@) + 1, Len(@) + l)]
33+
34+
Extend(i) ==
35+
/\ \A j \in Servers:
36+
Len(cLogs[j]) \leq Len(cLogs[i])
37+
/\ \E s \in BoundedSeq(Values, Lag - Max({Len(cLogs[i]) - Len(cLogs[j]) : j \in Servers})):
38+
cLogs' = [cLogs EXCEPT ![i] = @ \o s]
39+
40+
Next ==
41+
\E i \in Servers:
42+
\/ Copy(i)
43+
\/ Extend(i)
44+
45+
Spec ==
46+
/\ Init
47+
/\ [][Next]_vars
48+
/\ \A s \in Servers: WF_vars(Extend(s)) /\ WF_vars(Copy(s))
49+
50+
----
51+
\* Invariants
52+
53+
Abs(n) ==
54+
IF n < 0 THEN -n ELSE n
55+
56+
BoundedLag ==
57+
\A i, j \in Servers: Abs(Len(cLogs[i]) - Len(cLogs[j])) <= Lag
58+
59+
THEOREM Spec => []BoundedLag
60+
61+
----
62+
\* Liveness
63+
64+
AllExtending ==
65+
\A s \in Servers: []<><<IsStrictPrefix(cLogs[s], cLogs'[s])>>_cLogs
66+
67+
THEOREM Spec => AllExtending
68+
69+
InSync ==
70+
\* TLC correctly verifies that InSync is not a property of the system because
71+
\* followers are permitted to copy only a prefix of the missing suffix.
72+
\A i, j \in Servers : []<>(cLogs[i] = cLogs[j])
73+
74+
====
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
SPECIFICATION
2+
Spec
3+
4+
CONSTANTS
5+
Servers = {n1, n2, n3}
6+
Values = {v}
7+
Lag = 3
8+
9+
INVARIANTS
10+
TypeOK
11+
BoundedLag
12+
13+
PROPERTIES
14+
AllExtending
15+
InSync
16+
17+
VIEW
18+
DropCommonPrefix
19+
20+
CHECK_DEADLOCK
21+
TRUE
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---- MODULE MCDistributedReplicatedLog ----
2+
EXTENDS DistributedReplicatedLog, FiniteSetsExt
3+
4+
ASSUME
5+
\* LongestCommonPrefix in View for a single server would always shorten the
6+
\* log to <<>>, reducing the state-space to a single state.
7+
Cardinality(Servers) > 1
8+
9+
----
10+
11+
\* Combining the following conditions makes the state space finite:
12+
\* 1) The divergence of any two logs is bounded (See Extend action)
13+
\*
14+
\* 2) Terms is a *finite* set.
15+
ASSUME IsFiniteSet(Values)
16+
\*
17+
\* 3) The longest common prefix of all logs is discarded.
18+
DropCommonPrefix ==
19+
LET commonPrefixBound == Len(LongestCommonPrefix(Range(cLogs)))
20+
Drop(seq, idx) == SubSeq(seq, idx + 1, Len(seq))
21+
IN [ s \in Servers |-> Drop(cLogs[s], commonPrefixBound) ]
22+
23+
====

specifications/FiniteMonotonic/README.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,5 @@ Neither this approach nor the older approach have a proof of soundness and compl
44

55
Specs & models include:
66
- `CRDT.tla`: the spec of a basic grow-only counter CRDT
7-
- `ReplicatedLog.tla`: the "spec of a basic append-only replicated log
7+
- `ReplicatedLog.tla`: the spec of a basic append-only replicated log
8+
- `DistributedReplicatedLog.tla`: a spec of a distributed replicated log that demonstrates how the technique can be used to find violations of a liveness property (`Insync`).

0 commit comments

Comments
 (0)