Skip to content

Commit 269e396

Browse files
roystchiangroystchiang
committed
add memcached auto-discovery support from thanos (#4412)
Signed-off-by: Roy Chiang <[email protected]>
1 parent 76d5b3c commit 269e396

File tree

15 files changed

+319
-27
lines changed

15 files changed

+319
-27
lines changed

‎CHANGELOG.md

+1
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@
4141
* [ENHANCEMENT] Exemplars are now emitted for all gRPC calls and many operations tracked by histograms. #4462
4242
* [ENHANCEMENT] New options `-server.http-listen-network` and `-server.grpc-listen-network` allow binding as 'tcp4' or 'tcp6'. #4462
4343
* [ENHANCEMENT] Rulers: Using shuffle sharding subring on GetRules API. #4466
44+
* [ENHANCEMENT] Support memcached auto-discovery via `auto-discovery` flag, introduced by thanos in https://github.com/thanos-io/thanos/pull/4487. Both AWS and Google Cloud memcached service support auto-discovery, which returns a list of nodes of the memcached cluster. #4412
4445
* [BUGFIX] Fixes a panic in the query-tee when comparing result. #4465
4546
* [BUGFIX] Frontend: Fixes @ modifier functions (start/end) when splitting queries by time. #4464
4647
* [BUGFIX] Compactor: compactor will no longer try to compact blocks that are already marked for deletion. Previously compactor would consider blocks marked for deletion within `-compactor.deletion-delay / 2` period as eligible for compaction. #4328

‎docs/blocks-storage/querier.md

+12
Original file line numberDiff line numberDiff line change
@@ -512,6 +512,10 @@ blocks_storage:
512512
# CLI flag: -blocks-storage.bucket-store.index-cache.memcached.max-item-size
513513
[max_item_size: <int> | default = 1048576]
514514
515+
# Use memcached auto-discovery instead of DNS resolution
516+
# CLI flag: -blocks-storage.bucket-store.index-cache.memcached.auto-discovery
517+
[auto_discovery: <boolean> | default = false]
518+
515519
chunks_cache:
516520
# Backend for chunks cache, if not empty. Supported values: memcached.
517521
# CLI flag: -blocks-storage.bucket-store.chunks-cache.backend
@@ -559,6 +563,10 @@ blocks_storage:
559563
# CLI flag: -blocks-storage.bucket-store.chunks-cache.memcached.max-item-size
560564
[max_item_size: <int> | default = 1048576]
561565
566+
# Use memcached auto-discovery instead of DNS resolution
567+
# CLI flag: -blocks-storage.bucket-store.chunks-cache.memcached.auto-discovery
568+
[auto_discovery: <boolean> | default = false]
569+
562570
# Size of each subrange that bucket object is split into for better
563571
# caching.
564572
# CLI flag: -blocks-storage.bucket-store.chunks-cache.subrange-size
@@ -625,6 +633,10 @@ blocks_storage:
625633
# CLI flag: -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size
626634
[max_item_size: <int> | default = 1048576]
627635
636+
# Use memcached auto-discovery instead of DNS resolution
637+
# CLI flag: -blocks-storage.bucket-store.metadata-cache.memcached.auto-discovery
638+
[auto_discovery: <boolean> | default = false]
639+
628640
# How long to cache list of tenants in the bucket.
629641
# CLI flag: -blocks-storage.bucket-store.metadata-cache.tenants-list-ttl
630642
[tenants_list_ttl: <duration> | default = 15m]

‎docs/blocks-storage/store-gateway.md

+12
Original file line numberDiff line numberDiff line change
@@ -576,6 +576,10 @@ blocks_storage:
576576
# CLI flag: -blocks-storage.bucket-store.index-cache.memcached.max-item-size
577577
[max_item_size: <int> | default = 1048576]
578578
579+
# Use memcached auto-discovery instead of DNS resolution
580+
# CLI flag: -blocks-storage.bucket-store.index-cache.memcached.auto-discovery
581+
[auto_discovery: <boolean> | default = false]
582+
579583
chunks_cache:
580584
# Backend for chunks cache, if not empty. Supported values: memcached.
581585
# CLI flag: -blocks-storage.bucket-store.chunks-cache.backend
@@ -623,6 +627,10 @@ blocks_storage:
623627
# CLI flag: -blocks-storage.bucket-store.chunks-cache.memcached.max-item-size
624628
[max_item_size: <int> | default = 1048576]
625629
630+
# Use memcached auto-discovery instead of DNS resolution
631+
# CLI flag: -blocks-storage.bucket-store.chunks-cache.memcached.auto-discovery
632+
[auto_discovery: <boolean> | default = false]
633+
626634
# Size of each subrange that bucket object is split into for better
627635
# caching.
628636
# CLI flag: -blocks-storage.bucket-store.chunks-cache.subrange-size
@@ -689,6 +697,10 @@ blocks_storage:
689697
# CLI flag: -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size
690698
[max_item_size: <int> | default = 1048576]
691699
700+
# Use memcached auto-discovery instead of DNS resolution
701+
# CLI flag: -blocks-storage.bucket-store.metadata-cache.memcached.auto-discovery
702+
[auto_discovery: <boolean> | default = false]
703+
692704
# How long to cache list of tenants in the bucket.
693705
# CLI flag: -blocks-storage.bucket-store.metadata-cache.tenants-list-ttl
694706
[tenants_list_ttl: <duration> | default = 15m]

‎docs/configuration/arguments.md

+2
Original file line numberDiff line numberDiff line change
@@ -533,6 +533,8 @@ The DNS service discovery, inspired from Thanos DNS SD, supports different disco
533533

534534
If **no prefix** is provided, the provided IP or hostname will be used straightaway without pre-resolving it.
535535

536+
If you are using a managed memcached service from [Google Cloud](https://cloud.google.com/memorystore/docs/memcached/auto-discovery-overview), or [AWS](https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/AutoDiscovery.HowAutoDiscoveryWorks.html), use the [auto-discovery](./config-file-reference.md#memcached-client-config) flag instead of DNS discovery.
537+
536538
## Logging of IP of reverse proxy
537539

538540
If a reverse proxy is used in front of Cortex it might be diffult to troubleshoot errors. The following 3 settings can be used to log the IP address passed along by the reverse proxy in headers like X-Forwarded-For.

‎docs/configuration/config-file-reference.md

+12
Original file line numberDiff line numberDiff line change
@@ -4765,6 +4765,10 @@ bucket_store:
47654765
# CLI flag: -blocks-storage.bucket-store.index-cache.memcached.max-item-size
47664766
[max_item_size: <int> | default = 1048576]
47674767
4768+
# Use memcached auto-discovery instead of DNS resolution
4769+
# CLI flag: -blocks-storage.bucket-store.index-cache.memcached.auto-discovery
4770+
[auto_discovery: <boolean> | default = false]
4771+
47684772
chunks_cache:
47694773
# Backend for chunks cache, if not empty. Supported values: memcached.
47704774
# CLI flag: -blocks-storage.bucket-store.chunks-cache.backend
@@ -4812,6 +4816,10 @@ bucket_store:
48124816
# CLI flag: -blocks-storage.bucket-store.chunks-cache.memcached.max-item-size
48134817
[max_item_size: <int> | default = 1048576]
48144818
4819+
# Use memcached auto-discovery instead of DNS resolution
4820+
# CLI flag: -blocks-storage.bucket-store.chunks-cache.memcached.auto-discovery
4821+
[auto_discovery: <boolean> | default = false]
4822+
48154823
# Size of each subrange that bucket object is split into for better caching.
48164824
# CLI flag: -blocks-storage.bucket-store.chunks-cache.subrange-size
48174825
[subrange_size: <int> | default = 16000]
@@ -4877,6 +4885,10 @@ bucket_store:
48774885
# CLI flag: -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size
48784886
[max_item_size: <int> | default = 1048576]
48794887
4888+
# Use memcached auto-discovery instead of DNS resolution
4889+
# CLI flag: -blocks-storage.bucket-store.metadata-cache.memcached.auto-discovery
4890+
[auto_discovery: <boolean> | default = false]
4891+
48804892
# How long to cache list of tenants in the bucket.
48814893
# CLI flag: -blocks-storage.bucket-store.metadata-cache.tenants-list-ttl
48824894
[tenants_list_ttl: <duration> | default = 15m]

‎go.mod

+1-1
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ require (
5050
github.com/sony/gobreaker v0.4.1
5151
github.com/spf13/afero v1.2.2
5252
github.com/stretchr/testify v1.7.0
53-
github.com/thanos-io/thanos v0.22.0
53+
github.com/thanos-io/thanos v0.19.1-0.20210803192524-baea4ce9ef52
5454
github.com/uber/jaeger-client-go v2.29.1+incompatible
5555
github.com/weaveworks/common v0.0.0-20210901124008-1fa3f9fa874c
5656
go.etcd.io/bbolt v1.3.6

‎go.sum

+3-2
Original file line numberDiff line numberDiff line change
@@ -1715,8 +1715,8 @@ github.com/thanos-io/thanos v0.13.1-0.20210204123931-82545cdd16fe/go.mod h1:ZLDG
17151715
github.com/thanos-io/thanos v0.13.1-0.20210224074000-659446cab117/go.mod h1:kdqFpzdkveIKpNNECVJd75RPvgsAifQgJymwCdfev1w=
17161716
github.com/thanos-io/thanos v0.13.1-0.20210226164558-03dace0a1aa1/go.mod h1:gMCy4oCteKTT7VuXVvXLTPGzzjovX1VPE5p+HgL1hyU=
17171717
github.com/thanos-io/thanos v0.13.1-0.20210401085038-d7dff0c84d17/go.mod h1:zU8KqE+6A+HksK4wiep8e/3UvCZLm+Wrw9AqZGaAm9k=
1718-
github.com/thanos-io/thanos v0.22.0 h1:bHTzC0ZaP5rBJ2pOeJ73+hO/p3U4tSshJoR3pumM6sA=
1719-
github.com/thanos-io/thanos v0.22.0/go.mod h1:SZDWz3phcUcBr4MYFoPFRvl+Z9Nbi45HlwQlwSZSt+Q=
1718+
github.com/thanos-io/thanos v0.19.1-0.20210803192524-baea4ce9ef52 h1:aRxV+ebdkd5XqY1+vkwmdOpc66OpWtcvSF1e0qiRKsU=
1719+
github.com/thanos-io/thanos v0.19.1-0.20210803192524-baea4ce9ef52/go.mod h1:Xskx78e0CYL6w0yDNOZHGdvwQMlsuzPsePmPtbp9Xuk=
17201720
github.com/themihai/gomemcache v0.0.0-20180902122335-24332e2d58ab h1:7ZR3hmisBWw77ZpO1/o86g+JV3VKlk3d48jopJxzTjU=
17211721
github.com/themihai/gomemcache v0.0.0-20180902122335-24332e2d58ab/go.mod h1:eheTFp954zcWZXCU8d0AT76ftsQOTo4DTqkN/h3k1MY=
17221722
github.com/tidwall/pretty v0.0.0-20180105212114-65a9db5fad51/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
@@ -1900,6 +1900,7 @@ go.uber.org/atomic v1.8.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
19001900
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
19011901
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
19021902
go.uber.org/automaxprocs v1.2.0/go.mod h1:YfO3fm683kQpzETxlTGZhGIVmXAhaw3gxeBADbpZtnU=
1903+
go.uber.org/automaxprocs v1.4.0/go.mod h1:/mTEdr7LvHhs0v7mjdxDreTz1OG5zdZGqgOnhWiR/+Q=
19031904
go.uber.org/goleak v1.0.0/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A=
19041905
go.uber.org/goleak v1.1.10 h1:z+mqJhf6ss6BSfSM671tgKyZBFPTTJM+HLxnhPC3wu0=
19051906
go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A=

‎pkg/storage/tsdb/memcache_client_config.go

+3
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ type MemcachedClientConfig struct {
1818
MaxGetMultiConcurrency int `yaml:"max_get_multi_concurrency"`
1919
MaxGetMultiBatchSize int `yaml:"max_get_multi_batch_size"`
2020
MaxItemSize int `yaml:"max_item_size"`
21+
AutoDiscovery bool `yaml:"auto_discovery"`
2122
}
2223

2324
func (cfg *MemcachedClientConfig) RegisterFlagsWithPrefix(f *flag.FlagSet, prefix string) {
@@ -29,6 +30,7 @@ func (cfg *MemcachedClientConfig) RegisterFlagsWithPrefix(f *flag.FlagSet, prefi
2930
f.IntVar(&cfg.MaxGetMultiConcurrency, prefix+"max-get-multi-concurrency", 100, "The maximum number of concurrent connections running get operations. If set to 0, concurrency is unlimited.")
3031
f.IntVar(&cfg.MaxGetMultiBatchSize, prefix+"max-get-multi-batch-size", 0, "The maximum number of keys a single underlying get operation should run. If more keys are specified, internally keys are split into multiple batches and fetched concurrently, honoring the max concurrency. If set to 0, the max batch size is unlimited.")
3132
f.IntVar(&cfg.MaxItemSize, prefix+"max-item-size", 1024*1024, "The maximum size of an item stored in memcached. Bigger items are not stored. If set to 0, no maximum size is enforced.")
33+
f.BoolVar(&cfg.AutoDiscovery, prefix+"auto-discovery", false, "Use memcached auto-discovery instead of DNS resolution")
3234
}
3335

3436
func (cfg *MemcachedClientConfig) GetAddresses() []string {
@@ -59,5 +61,6 @@ func (cfg MemcachedClientConfig) ToMemcachedClientConfig() cacheutil.MemcachedCl
5961
MaxGetMultiBatchSize: cfg.MaxGetMultiBatchSize,
6062
MaxItemSize: model.Bytes(cfg.MaxItemSize),
6163
DNSProviderUpdateInterval: 30 * time.Second,
64+
AutoDiscovery: cfg.AutoDiscovery,
6265
}
6366
}

‎vendor/github.com/thanos-io/thanos/pkg/block/metadata/meta.go

+1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎vendor/github.com/thanos-io/thanos/pkg/cacheutil/memcached_client.go

+41-16
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)