Skip to content

feat: Add maglev consistent hashing #2831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: develop
Choose a base branch
from
Open

feat: Add maglev consistent hashing #2831

wants to merge 3 commits into from

Conversation

dongjiang1989
Copy link
Contributor

@dongjiang1989
Copy link
Contributor Author 

goos: darwin
goarch: amd64
pkg: dubbo.apache.org/dubbo-go/v3/cluster/loadbalance
cpu: VirtualApple @ 2.50GHz
BenchmarkRoudrobinLoadbalance-8                       	    7713	    157304 ns/op	   88116 B/op	    2555 allocs/op
BenchmarkLeastativeLoadbalance-8                      	    7767	    148902 ns/op	   81886 B/op	    2043 allocs/op
BenchmarkConsistenthashingLoadbalance-8               	   59444	     20656 ns/op	    3440 B/op	     267 allocs/op
BenchmarkP2CLoadbalance-8                             	 2774712	       441.4 ns/op	     472 B/op	      10 allocs/op
BenchmarkInterleavedWeightedRoundRobinLoadbalance-8   	   22604	     52958 ns/op	   45312 B/op	    1027 allocs/op
BenchmarkRandomLoadbalance-8                          	   24267	     47699 ns/op	   38992 B/op	     767 allocs/op
BenchmarkAliasMethodLoadbalance-8                     	   24961	     47120 ns/op	   49232 B/op	     772 allocs/op
BenchmarkMaglevConsistentHashingLoadBalance-8         	    8415	    127673 ns/op	  598406 B/op	     262 allocs/op
PASS
ok  	dubbo.apache.org/dubbo-go/v3/cluster/loadbalance	14.628s

@marsevilspirit
Copy link
Member

From the benchmark results alone, the performance of maglev does not seem very good.
Could you explain the reason for adding maglev?

@dongjiang1989 dongjiang1989 force-pushed the maglev-consistent-hash branch from c8cc99b to 1a125e1 Compare April 14, 2025 07:27
@dongjiang1989
Copy link
Contributor Author

Thanks @marsevilspirit
The internal multi-language service loadbalance uniformly uses Maglev consistent hashing.
It is hoped that the unified Maglev consistent hashing will be used after the Dubbo service is upgraded.

BTW, I will optimize this algorithm implementation

@dongjiang1989 dongjiang1989 changed the base branch from main to develop April 14, 2025 10:10
@dongjiang1989 dongjiang1989 force-pushed the maglev-consistent-hash branch from 1a125e1 to 23ecde6 Compare April 14, 2025 10:10
@dongjiang1989
Copy link
Contributor Author

dongjiang1989 commented Apr 15, 2025 •
edited
Loading 

goos: darwin
goarch: amd64
pkg: dubbo.apache.org/dubbo-go/v3/cluster/loadbalance
cpu: VirtualApple @ 2.50GHz
BenchmarkRoudrobinLoadbalance-8                       	    5324	    393337 ns/op	   88116 B/op	    2555 allocs/op
BenchmarkLeastativeLoadbalance-8                      	    5378	    249848 ns/op	   81885 B/op	    2043 allocs/op
BenchmarkConsistenthashingLoadbalance-8               	   46246	     26854 ns/op	    3440 B/op	     267 allocs/op
BenchmarkP2CLoadbalance-8                             	 2654329	       428.2 ns/op	     472 B/op	      10 allocs/op
BenchmarkInterleavedWeightedRoundRobinLoadbalance-8   	   22348	     54638 ns/op	   45312 B/op	    1027 allocs/op
BenchmarkRandomLoadbalance-8                          	   25276	     45845 ns/op	   38992 B/op	     767 allocs/op
BenchmarkAliasMethodLoadbalance-8                     	   22459	     49926 ns/op	   49232 B/op	     772 allocs/op
BenchmarkMaglevConsistentHashingLoadBalance-8         	    9170	    132202 ns/op	  598406 B/op	     262 allocs/op
PASS
ok  	dubbo.apache.org/dubbo-go/v3/cluster/loadbalance	17.807s

@dongjiang1989
add maglev consistent hashing
6dcecd1 
Signed-off-by: dongjiang <[email protected]>
@dongjiang1989
optimize algorithm
c6c089a 
Signed-off-by: dongjiang <[email protected]>
@dongjiang1989
update maglev
27e29ac 
Signed-off-by: dongjiang <[email protected]>
@dongjiang1989 dongjiang1989 force-pushed the maglev-consistent-hash branch from 4aa4ee9 to 27e29ac Compare April 22, 2025 10:04
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
12.3% Duplication on New Code

See analysis details on SonarQube Cloud

xuzifu666
xuzifu666 reviewed Jun 12, 2025
View reviewed changes
Copy link
Member

@xuzifu666 xuzifu666 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some minor comments.

cluster/loadbalance/maglevconsistenthashing/maglev.go
return m
}

// buildLookupTable 构建 Maglev 查找表
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why are the annotations in Chinese used here, and other places contain English annotations? Can they all be unified into English annotations?

cluster/loadbalance/maglevconsistenthashing/maglev.go
}
}

// generatePermutations 生成每个节点的排列
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

cluster/loadbalance/maglevconsistenthashing/maglev.go
return permutations
}

// calculatePermutation 计算单个节点的排列
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

cluster/loadbalance/maglevconsistenthashing/maglev.go
return xxhash.Sum64String(s)
}

// isPrime 函数用于判断一个数是否为素数
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

cluster/loadbalance/maglevconsistenthashing/maglev.go
return true
}

// NextPrime 函数用于找出比 n 大的第一个素数
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xuzifu666 xuzifu666 xuzifu666 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dongjiang1989 @marsevilspirit @xuzifu666