Deprecated Cortex chunks storage

[pracucci]

What this PR does:
During the last Cortex community meeting we discussed about deprecating the Cortex chunks storage and putting it into maintenance mode. This PR proposes to do the first step and announce it (CHANGELOG + doc).

No config change has been done to switch the default storage from chunks to blocks, in order to not introduce any breaking change.

I'm also proposing to delete k8s/ manifests (as discussed during the last community meeting as well) because they're out of date, unmaintained and some config is also broken. We're seen some users getting confused by it so, unless we'll start actively maintaining and continuously testing them, I think it's better to get rid of them.

If this PR will be accepted & merged, I will also take care of gently closing all open PRs proposing non-bug fix changes to the chunks storage.

Which issue(s) this PR fixes:
N/A

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[bboreham]

Good job. I think the engine will still default to chunks, so perhaps explain that, with reference to our compatibility guarantees.

[pracucci]

I think the engine will still default to chunks, so perhaps explain that, with reference to our compatibility guarantees.

@bboreham Good point. Where do you suggest to mention it, other than the CHANGELOG?

[bboreham]

Perhaps at the top of docs/configuration/arguments.md ?

I think it could be on the flag too:

cortex/pkg/chunk/storage/factory.go

Line 114 in d382e1d

f.StringVar(&cfg.Engine, "store.engine", "chunks", "The storage engine to use: chunks or blocks.")

E.g. "The storage engine to use: chunks (deprecated) or blocks."

[bboreham]

I guess we would also close all issues relating to the chunks storage: storage/chunks Chunks storage engine

[jtlisi]

LGTM

@pracucci Any thoughts on potential reducing the usage of chunks storage in the e2e tests on the repo? We are starting to hit close to 30 minutes per run and shaving a few minutes off would be nice.

[pstibrany]

Deprecating chunks sounds good to me, but having deprecated default value for non-deprecated option looks weird. Is it time to make Cortex 2 release to fix this, and do some options cleanup?

[bboreham]

Agreed on changing the default and on reducing e2e testing, later, but getting the "deprecated" message in front of people is a priority.

[pracucci]

I've created a meta issue to track the next steps about deprecating the chunks storage:
#4295