Skip to content

Commit 0b2fd52

Browse files
ashcherbakovashcherbakov
authored
INDY-670, INDY-790: multi-signature and state proofs (#385)
* bls signatures charm-based implementation * bls infrustructure and hierarchy * sign one msg * implement bls_key_registry; support seed for BLS keys * fix tests * expect all data in str * bls-bft implementation * bls-bft implementation * fix tests * call BLS validate COMMIT in replica * add BLS_KEY to Node txn * add init_bls_keys script * fix init script * support calculation of multi-sig; add gc * rename test_send_txns_no_bls to test_send_txns_partial_bls to make it match file name and purpose * fix conditions and getting of public key by node name in BlsBftPlenum * modify tests to avoid 3pc batching * make replica add bls signature to COMMIT message * change signature of calculate_multi_sig to let it return optional str * add test_send_txns_full_bls test * make replica register own commit in bls registry * move docstrings from BlsBftPlenum to BlsBft * add base class for BLS validation errors * raise exception if validation of bls signature for commit failed * add validate_multi_sig method to BlsBft * make BlsBftPlenum raise exceptions in validation methods and do not store commits * comment raising of exception if there is no signature since it is allowed as of now * update test_validate_pre_prepare_incorrect_multi_sig * in replica handle exceptions from bls * update tests in test_bls_bft.py to make them test multisig using prepares instead of commits * move signatures from commit to prepares as it is described in design doc * remove debug logging and unused import * make replica add latest multi-sig to preprepare * create BlsMultiSignatureField validator and test for it * remove redundant test case for BlsMultiSignature message * update test of validation of commit and prepare preprepare messages * use BlsMultiSignatureField instead of BlsMultiSignature in preprepare * use multi-sig as a tuple instead of a dict * propagate primaries multisig to other nodes * return bls signature back to commit * add log for saving multi sig * use multisig for domain ledger only * fix tests in test_primary_selector * fix test for commit message * fix name of test_send_txns_full_bls test * add signarue for current batch to preprepare * refactor verification methods of BlsBftPlenum * update test of PrePrepare message * Add key value bls store * Save on master instance only * check if multisig is none before handling it * add missing empty line * pass bls store to domain request handler * add bls_Store to TestNode * fix broken import of PREPARE * add constants for state proof fields * exclude state proof from checking of rreply equality * Integrate indy_crypto lib * fix excluding of state_proof fromr replies * Add indy-crypto factory * New indy-crypto api * Final indy_crypto doc * add correct root hash to multi singature in preprepare * create class for multi signature * use MultiSignature in bls_bft * add todo about updating BlsMultiSignature message * add checks of multi sig fields * init bls keys in tests * send BLS sigs in COMMITs only * use MultiSignature in BlsStore * add as_dict and define __eq__ in MultiSignature * use MultiSignature in BlsBftPlenum * Add new dep python3-indy-crypto to install req * blskey support * remove redundant args * update signatures, remove validation of prepare, use MultiSignature * Make generator type str * remove sig from (pre)prepare, add state to commit * Add blsfactoryindycrypto; add dedicated seed for bls * support 48-bit seed * disable charm-based tests * re-factored bls_bft move all bls-related logic from replica to bls_bft * re-factored bls_bft move all bls-related logic from replica to bls_bft * use pool_state for multi-sig * fixes * fixed tests * Edit docker file to install indy-crypto (#384) * Edit docker file to install indy-crypto (#384) * pass pool_state to bls_bft * fix static validation errors * fix static validation errors * fix static validation errors * fix static validation errors * fix static validation errors * add txn time to nym * Bls storage (#387) * Edit docker file to install indy-crypto * Change libindy-crypto source to sovrin repo instead of wget it * remove bls multisig message test * use pool_state_root; get rid of charm-code * use correct pool_state_root for multi-sig * update tests * fix tests * support getting a value for a specified root * create bls keys in initLocalKeys * return bls key in initLocalKeys * update usages of initLocalKeys * generate bls keys not only for local nodes * fix static code validation errors * get BLS keys for the given pool root state * extend tests * add tests for BLS sig consensus * enable BLS for domain ledger only * fix tests to include BLS keys into NODE txn for new nodes * use list instead of tuple when creating PrePrepare to have correct equal * more bls_bft tests; improve logging * fix usage of initLocalKeys gen_steward_key * fix code review findings * add docs * build and install python3-indy-crypto with libindy-crypto dependency (needs to be re-factored)
1 parent 869fc7b commit 0b2fd52

File tree

87 files changed

+3114
-563
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

87 files changed

+3114
-563
lines changed

‎build-scripts/ubuntu-1604/build-3rd-parties.sh

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,39 @@ function build_from_pypi {
3838
rm ${PREREM_TMP}
3939
}
4040

41+
function build_indy_crypto {
42+
PACKAGE_NAME=$1
43+
44+
if [ -z $2 ]; then
45+
PACKAGE_VERSION=""
46+
else
47+
PACKAGE_VERSION="==$2"
48+
fi
49+
POSTINST_TMP=postinst-${PACKAGE_NAME}
50+
PREREM_TMP=prerm-${PACKAGE_NAME}
51+
cp postinst ${POSTINST_TMP}
52+
cp prerm ${PREREM_TMP}
53+
sed -i 's/{package_name}/'${PACKAGE_NAME}'/' ${POSTINST_TMP}
54+
sed -i 's/{package_name}/'${PACKAGE_NAME}'/' ${PREREM_TMP}
55+
56+
fpm --input-type "python" \
57+
--output-type "deb" \
58+
--architecture "amd64" \
59+
--verbose \
60+
--python-bin "/usr/bin/python3" \
61+
--exclude "*.pyc" \
62+
--exclude "*.pyo" \
63+
--maintainer "Hyperledger <[email protected]>" \
64+
--after-install ${POSTINST_TMP} \
65+
--before-remove ${PREREM_TMP} \
66+
--package ${OUTPUT_PATH} \
67+
--depends libindy-crypto \
68+
--name ${PACKAGE_NAME} \
69+
${PACKAGE_NAME}${PACKAGE_VERSION}
70+
71+
rm ${POSTINST_TMP}
72+
rm ${PREREM_TMP}
73+
}
4174

4275
build_from_pypi ioflo 1.5.4
4376
build_from_pypi orderedset 2.0
@@ -50,3 +83,4 @@ build_from_pypi pyzmq 16.0.2
5083
build_from_pypi intervaltree 2.1.0
5184
build_from_pypi portalocker 0.5.7
5285
build_from_pypi sortedcontainers 1.5.7
86+
build_indy_crypto python3-indy-crypto 0.1.2

‎ci/ubuntu.dockerfile

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,17 +11,24 @@ RUN apt-get update -y && apt-get install -y \
1111
python3.5 \
1212
python3-pip \
1313
python-setuptools \
14+
apt-transport-https \
15+
ca-certificates \
1416
python3-nacl
1517
RUN pip3 install -U \
1618
pip \
1719
setuptools \
1820
virtualenv
21+
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 68DB5E88
22+
RUN echo "deb https://repo.sovrin.org/deb xenial master" >> /etc/apt/sources.list
23+
RUN apt-get update -y && apt-get install -y libindy-crypto
1924
RUN useradd -ms /bin/bash -u $uid $user
2025
USER $user
2126
RUN virtualenv -p python3.5 /home/$user/test
2227
USER root
2328
RUN ln -sf /home/$user/test/bin/python /usr/local/bin/python
2429
RUN ln -sf /home/$user/test/bin/pip /usr/local/bin/pip
30+
RUN wget https://repo.evernym.com/libindy_crypto/ubuntu/stable/0.1.2/libindy-crypto_0.1.2_amd64.deb
31+
RUN dpkg -i ./libindy-crypto_0.1.2_amd64.deb
2532
USER $user
2633
# TODO: Automate dependency collection
2734
RUN pip install jsonpickle \
@@ -45,5 +52,6 @@ RUN pip install jsonpickle \
4552
ioflo==1.5.4 \
4653
psutil \
4754
intervaltree \
48-
pytest-xdist
55+
pytest-xdist \
56+
python3-indy-crypto==0.1.2
4957
WORKDIR /home/$user

‎common/serializers/base58_serializer.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
import base58
2+
from common.serializers.mapping_serializer import MappingSerializer
3+
4+
5+
class Base58Serializer(MappingSerializer):
6+
def serialize(self, data, fields=None, toBytes=False):
7+
return base58.b58encode(data)
8+
9+
def deserialize(self, data, fields=None):
10+
return base58.b58decode(data)

‎common/serializers/serialization.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
from typing import Mapping
22

3+
from common.serializers.base58_serializer import Base58Serializer
34
from common.serializers.json_serializer import JsonSerializer
45
from common.serializers.msgpack_serializer import MsgPackSerializer
56
from common.serializers.signing_serializer import SigningSerializer
@@ -10,6 +11,8 @@
1011
domain_state_serializer = JsonSerializer()
1112
pool_state_serializer = JsonSerializer()
1213
client_req_rep_store_serializer = JsonSerializer()
14+
multi_sig_store_serializer = JsonSerializer()
15+
state_roots_serializer = Base58Serializer()
1316

1417

1518
def serialize_msg_for_signing(msg: Mapping, topLevelKeysToIgnore=None):

‎crypto/__init__.py

Whitespace-only changes.

‎crypto/bls/__init__.py

Whitespace-only changes.

‎crypto/bls/bls_bft.py

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
from abc import ABCMeta, abstractmethod
2+
3+
from common.serializers.serialization import state_roots_serializer
4+
from crypto.bls.bls_crypto import BlsCrypto
5+
from crypto.bls.bls_key_register import BlsKeyRegister
6+
from plenum.common.messages.node_messages import PrePrepare, Prepare, Commit
7+
8+
9+
class BlsBft(metaclass=ABCMeta):
10+
def __init__(self,
11+
bls_crypto: BlsCrypto,
12+
bls_key_register: BlsKeyRegister,
13+
node_id,
14+
is_master,
15+
bls_store):
16+
self.bls_key_register = bls_key_register
17+
self.node_id = node_id
18+
19+
self._bls_crypto = bls_crypto
20+
self._is_master = is_master
21+
self._bls_store = bls_store
22+
self._state_root_serializer = state_roots_serializer
23+
24+
@abstractmethod
25+
def validate_pre_prepare(self, pre_prepare: PrePrepare, sender):
26+
'''
27+
Validates PrePrepare for correct BLS signatures.
28+
Raises SuspiciousNode exception if there are errors
29+
:param pre_prepare: pre-prepare to be validated
30+
:param sender: sender's Node name
31+
:return:
32+
'''
33+
pass
34+
35+
@abstractmethod
36+
def validate_prepare(self, prepare: Prepare, sender):
37+
'''
38+
Validates Prepare for correct BLS signatures.
39+
Raises SuspiciousNode exception if there are errors
40+
:param prepare: prepare to be validated
41+
:param sender: sender's Node name
42+
:return:
43+
'''
44+
pass
45+
46+
@abstractmethod
47+
def validate_commit(self, commit: Commit, sender, state_root_hash):
48+
'''
49+
Validates Commit for correct BLS signatures.
50+
Raises SuspiciousNode exception if there are errors
51+
:param commit: commit to be validated
52+
:param sender: sender's Node name
53+
:param state_root_hash: domain state root hash to validate BLS against
54+
:return:
55+
'''
56+
pass
57+
58+
@abstractmethod
59+
def process_pre_prepare(self, pre_prepare: PrePrepare, sender):
60+
'''
61+
Performs BLS-related logic for a given PrePrepare (for example,
62+
saving multi-signature calculated by Pre-Prepare for last batches).
63+
:param pre_prepare: pre-prepare to be processed
64+
:param sender: sender's Node name
65+
:return:
66+
'''
67+
pass
68+
69+
@abstractmethod
70+
def process_prepare(self, prepare: Prepare, sender):
71+
'''
72+
Performs BLS-related logic for a given Prepare
73+
:param prepare: prepare to be processed
74+
:param sender: sender's Node name
75+
:return:
76+
'''
77+
pass
78+
79+
@abstractmethod
80+
def process_commit(self, commit: Commit, sender):
81+
'''
82+
Performs BLS-related logic for a given Commit (for example, saving BLS signatures from this Commit)
83+
:param commit: commit to be processed
84+
:param sender: sender's Node name
85+
:return:
86+
'''
87+
pass
88+
89+
@abstractmethod
90+
def process_order(self, key, state_root_hash, quorums, ledger_id):
91+
'''
92+
Performs BLS-related logic when Ordering (for example, calculate a temporarily multi-sig by a current Node
93+
which will be replaced by Primary's multi-sig in process_prepare).
94+
:param key: 3PC-key re;ated to the Ordered message
95+
:param state_root: domain state root hash to validate BLS against
96+
:param quorums: quorums
97+
:param ledger_id: ledger's ID
98+
:return:
99+
'''
100+
pass
101+
102+
@abstractmethod
103+
def update_pre_prepare(self, pre_prepare_params, ledger_id):
104+
'''
105+
Adds BLS-related parameters to be used for creation of a new PrePrepare
106+
:param pre_prepare_params: a list of existing parameters
107+
:param ledger_id: ledger's ID
108+
:return: pre_prepare_params updated with BLS ones
109+
'''
110+
pass
111+
112+
@abstractmethod
113+
def update_prepare(self, prepare_params, ledger_id):
114+
'''
115+
Adds BLS-related parameters to be used for creation of a new Prepare
116+
:param prepare_params: a list of existing parameters
117+
:param ledger_id: ledger's ID
118+
:return: pre_prepare_params updated with BLS ones
119+
'''
120+
pass
121+
122+
@abstractmethod
123+
def update_commit(self, commit_params, state_root_hash, ledger_id):
124+
'''
125+
Adds BLS-related parameters to be used for creation of a new Commit
126+
:param commit_params: a list of existing parameters
127+
:param ledger_id: ledger's ID
128+
:return: pre_prepare_params updated with BLS ones
129+
'''
130+
pass
131+
132+
@abstractmethod
133+
def gc(self, key_3PC):
134+
"""
135+
Do some cleaning if needed
136+
137+
:param key_3PC: 3PC-key
138+
"""
139+
pass
140+
141+
142+
class BlsValidationError(Exception):
143+
"""
144+
BLS signature validation error
145+
"""

‎crypto/bls/bls_crypto.py

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
from abc import ABCMeta, abstractmethod
2+
from collections import namedtuple
3+
from typing import Sequence
4+
5+
GroupParams = namedtuple('GroupParams',
6+
'group_name, g')
7+
8+
9+
class BlsGroupParamsLoader(metaclass=ABCMeta):
10+
@abstractmethod
11+
def load_group_params(self) -> GroupParams:
12+
pass
13+
14+
15+
class BlsCrypto(metaclass=ABCMeta):
16+
def __init__(self, sk: str, pk: str, params: GroupParams):
17+
assert sk
18+
assert pk
19+
self._sk = sk
20+
self.pk = pk
21+
self._group_params = params
22+
23+
@staticmethod
24+
@abstractmethod
25+
def generate_keys(params: GroupParams, seed=None) -> (str, str):
26+
pass
27+
28+
@abstractmethod
29+
def sign(self, message: str) -> str:
30+
pass
31+
32+
@abstractmethod
33+
def create_multi_sig(self, signatures: Sequence[str]) -> str:
34+
pass
35+
36+
@abstractmethod
37+
def verify_sig(self, signature: str, message: str, pk: str) -> bool:
38+
pass
39+
40+
@abstractmethod
41+
def verify_multi_sig(self, signature: str, message: str, pks: Sequence[str]) -> bool:
42+
pass

‎crypto/bls/bls_factory.py

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
from abc import ABCMeta, abstractmethod
2+
3+
from crypto.bls.bls_bft import BlsBft
4+
from crypto.bls.bls_crypto import BlsCrypto, BlsGroupParamsLoader
5+
from crypto.bls.bls_key_manager import BlsKeyManager
6+
from crypto.bls.bls_key_register import BlsKeyRegister
7+
from plenum.bls.bls_store import BlsStore
8+
9+
10+
class BlsFactoryCrypto(metaclass=ABCMeta):
11+
def generate_bls_keys(self, seed=None) -> str:
12+
return self._get_bls_crypto_class().generate_keys(
13+
self._load_group_params(),
14+
seed)
15+
16+
def generate_and_store_bls_keys(self, seed=None) -> str:
17+
bls_key_manager = self._create_key_manager(
18+
self._load_group_params())
19+
20+
sk, pk = self.generate_bls_keys(seed)
21+
stored_sk, stored_pk = bls_key_manager.save_keys(sk, pk)
22+
23+
return stored_pk
24+
25+
def create_bls_crypto_from_saved_keys(self) -> BlsCrypto:
26+
group_params = self._load_group_params()
27+
bls_key_manager = self._create_key_manager(group_params)
28+
sk, pk = bls_key_manager.load_keys()
29+
return self._create_bls_crypto(sk, pk, group_params)
30+
31+
def _load_group_params(self):
32+
return self._create_group_params_loader().load_group_params()
33+
34+
@abstractmethod
35+
def _create_group_params_loader(self) -> BlsGroupParamsLoader:
36+
pass
37+
38+
@abstractmethod
39+
def _create_key_manager(self, group_params) -> BlsKeyManager:
40+
pass
41+
42+
@abstractmethod
43+
def _get_bls_crypto_class(self):
44+
pass
45+
46+
@abstractmethod
47+
def _create_bls_crypto(self, sk, pk, group_params):
48+
pass
49+
50+
51+
class BlsFactoryBft(metaclass=ABCMeta):
52+
53+
def __init__(self, bls_factory_crypto: BlsFactoryCrypto):
54+
self._bls_factory_crypto = bls_factory_crypto
55+
56+
def create_bls_bft(self, is_master) -> BlsBft:
57+
bls_crypto = self._bls_factory_crypto.create_bls_crypto_from_saved_keys()
58+
bls_key_register = self._create_bls_key_register()
59+
return self._create_bls_bft(bls_crypto, bls_key_register, is_master)
60+
61+
@abstractmethod
62+
def create_bls_store(self) -> BlsStore:
63+
pass
64+
65+
@abstractmethod
66+
def _create_bls_key_register(self) -> BlsKeyRegister:
67+
pass
68+
69+
@abstractmethod
70+
def _create_bls_bft(self, bls_crypto, bls_key_register, is_master) -> BlsBft:
71+
pass

0 commit comments

Comments
 (0)