Skip to content

Reduce Preprepare size #1924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 34 commits into from
Jul 14, 2022
Merged

Reduce Preprepare size #1924

merged 34 commits into from
Jul 14, 2022

Conversation

mcortesi
Copy link
Contributor

@mcortesi mcortesi commented Jul 14, 2022

Description

This PR modifies RoundChangeCertificate encoding so that we don't serialize N times the proposal, instead we do that only once.

Backwards compatibility

Change is not backward compatible, requires 2/3 of validators to use the new version at that same time

if err != nil {
return err
}
if proposal, ok := propIndex[im.ProposalHash]; ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could im.ProposalHash be nil here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only if someone is sending a malicious message, I think. But in this case, as the propIndex is generated with valid proposals, it shouldn't find anything and in that case

mess := make([]Message, len(iMess))
for i, im := range iMess {
mess[i] = Message{
Code: im.Message.Code,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could im.Message be nil here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm. As much as before, if an element of the list had been nil. Not sure if it's great not to check, but it's not worse than before

}

// Iterate values. RLP does not support maps
proposals := make([]*types.Block, len(proposalsMap))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be helpful to log the number of proposals we're seeing here (e.g. with trace logging)?

If this does not fix the issue, it would be nice to get some fresh logging data to see what the composition of the round change certificate looks like.


if proposal != nil {
// we don't use the height since we know they MUST be the same
proposalsMap[proposal.Hash()] = proposal
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we can't RLP a map, is there a set data structure we can use here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using that map to build the proposal list. As the same proposal will be set in the same hash, that result array will be already a set

@hbandura hbandura marked this pull request as ready for review July 14, 2022 22:03
@hbandura hbandura requested a review from a team as a code owner July 14, 2022 22:03
@hbandura hbandura requested review from gastonponti and nategraf and removed request for a team July 14, 2022 22:03
@mcortesi mcortesi merged commit 152b6dc into release/1.5.x Jul 14, 2022
@mcortesi mcortesi deleted the hbandura/roundchange_boost branch July 14, 2022 22:05
hbandura added a commit that referenced this pull request Jul 20, 2022
* Log line from Warn to Debug

* Bump version to 1.5.8

* New encoding for RoundChangeCertificate

* clean up NewIndexedRoundChangeMessage

* Use pointers whenever possible to reduce call by value costs

* Merge

* Fix compile error

* Added comment

* Added docs

* Add error checking

* Add error

* Clean up encoding logic

* more cleanup

* Fix compile errors

* Added doc

* Fix dummy object in test

* Fix

* Fix tests

* Remove nil

* fix proposal not rlp serializable

* fix test

* Add tests

* Fix lint

* Fix test

* fix test

* Fix test

* Fix

* Add log

* Revert change

* cleanup (#1925)

* cleanup

* revert

* Fix debug log

Co-authored-by: Pasto <[email protected]>
Co-authored-by: Gaston Ponti <[email protected]>
hbandura added a commit that referenced this pull request Jul 21, 2022
Cherry pick reduce preprepare size from release/1.5.x

* Log line from Warn to Debug

* Bump version to 1.5.8

* New encoding for RoundChangeCertificate

* clean up NewIndexedRoundChangeMessage

* Use pointers whenever possible to reduce call by value costs

* Merge

* Fix compile error

* Added comment

* Added docs

* Add error checking

* Add error

* Clean up encoding logic

* more cleanup

* Fix compile errors

* Added doc

* Fix dummy object in test

* Fix

* Fix tests

* Remove nil

* fix proposal not rlp serializable

* fix test

* Add tests

* Fix lint

* Fix test

* fix test

* Fix test

* Fix

* Add log

* Revert change

* cleanup (#1925)

* cleanup

* revert

* Fix debug log

Co-authored-by: Pasto <[email protected]>
Co-authored-by: Gaston Ponti <[email protected]>

Co-authored-by: Mariano Cortesi <[email protected]>
Co-authored-by: Gaston Ponti <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants