Fix race condition when opening and closing tsdb concurrently

[alanprot]

What this PR does:
Fix race condition when opening and closing tsdb concurrently.

Before this change, we had a potential race condition where we released the userState lock before actually deleting the local directory, while ingester received a new request and is trying to create a new TSDB.

We could see on our log the following errors/warns:

level=error ts=2021-03-13T17:35:58.129738177Z caller=ingester_v2.go:1759 msg="failed to delete local TSDB" user=tenant_id err="unlinkat /data/tsdb/tenant_id/wal: directory not empty"
...
...
level=warn ts=2021-03-13T17:35:58.424349159Z caller=grpc_logging.go:38 method=/cortex.Ingester/Push duration=303.330741ms err="user=tenant_id: failed to open TSDB: /data/tsdb/tenant_id: repair corrupted WAL: cannot handle error: open WAL segment: 412: open /data/tsdb/tenant_id/wal/00000412: no such file or directory" msg="gRPC\n"

The PR includes a test that reproduce the problem.

Which issue(s) this PR fixes:
Fixes #

Checklist

• Tests updated
• [NA] Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[pracucci]

LGTM

[alanprot]

Thanks!! :D