db: manually managed memTable arena memory

[petermattis]

No description provided.

[petermattis]

This change is 

[petermattis]

I extracted the memTable memory management commit from #523. All but the last commit here are the commits in #523.

[petermattis]

This is ready for a look.

Reviewable status: 0 of 6 files reviewed, 1 unresolved discussion (waiting on @itsbilal and @sumeerbhola)

---

open.go, line 64 at r1 (raw file):

		// allocated memory resources. Note that rather than look for an error, we
		// look for the return of a nil DB pointer.
		if r := recover(); db == nil {

This was probably the trickiest part of the change: properly performing cleanup during the various error conditions that can occur during Open. Something similar showed up in #529.

[petermattis]

The first commit is #529 which is needed for all of the test changes to properly close DBs.

[petermattis]

This has been rebased now that #529 has been merged.

[sumeerbhola]

Reviewed 32 of 41 files at r2, 6 of 6 files at r3.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @itsbilal and @petermattis)

---

mem_table.go, line 42 at r3 (raw file):

// A memTable is implemented on top of a lock-free arena-backed skiplist. An
// arena is a fixed size contiguous chunk of memory (see
// Options.MemTableSize). A memTable's memory consumtion is thus fixed at

nit: consumption

---

mem_table.go, line 49 at r3 (raw file):

// A batch is "applied" to a memTable in a two step process: prepare(batch) ->
// apply(batch). memTable.prepare() is not thread-safe and must be called with
// external sychronization. Preparation reserves space in the memTable for the

nit: synchronization

---

internal/arenaskl/arena.go, line 43 at r3 (raw file):

)

// NewArena allocates a new arena of the specified size and returns it.

comment is stale

[petermattis]

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @itsbilal and @sumeerbhola)

---

mem_table.go, line 42 at r3 (raw file):

Previously, sumeerbhola wrote…

nit: consumption

Done. Apparently, I need to use spellcheck on my comments.

---

internal/arenaskl/arena.go, line 43 at r3 (raw file):

Previously, sumeerbhola wrote…

comment is stale

Done.

[petermattis]

TFTR!

[petermattis]

I had to add a few test-only fixes to missed Cache.Unref() calls. Not sure why these popped up now and not in #529. Oh well, they were easy to find. go test -count 100 was helpful in sussing these out.