Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to support strict JSON unmarshal for limits #4298

Merged
merged 3 commits into from
Jun 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## master / unreleased

* [CHANGE] Enable strict JSON unmarshal for `pkg/util/validation.Limits` struct. The custom `UnmarshalJSON()` will now fail if the input has unknown fields. #4298
* [CHANGE] Cortex chunks storage has been deprecated and it's now in maintenance mode: all Cortex users are encouraged to migrate to the blocks storage. No new features will be added to the chunks storage. The default Cortex configuration still runs the chunks engine; please check out the [blocks storage doc](https://cortexmetrics.io/docs/blocks-storage/) on how to configure Cortex to run with the blocks storage. #4268
* [CHANGE] The example Kubernetes manifests (stored at `k8s/`) have been removed due to a lack of proper support and maintenance. #4268
* [CHANGE] Querier / ruler: deprecated `-store.query-chunk-limit` CLI flag (and its respective YAML config option `max_chunks_per_query`) in favour of `-querier.max-fetched-chunks-per-query` (and its respective YAML config option `max_fetched_chunks_per_query`). The new limit specifies the maximum number of chunks that can be fetched in a single query from ingesters and long-term storage: the total number of actual fetched chunks could be 2x the limit, being independently applied when querying ingesters and long-term storage. #4125
Expand Down
6 changes: 5 additions & 1 deletion pkg/util/validation/limits.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package validation

import (
"bytes"
"encoding/json"
"errors"
"flag"
Expand Down Expand Up @@ -229,7 +230,10 @@ func (l *Limits) UnmarshalJSON(data []byte) error {
}

type plain Limits
return json.Unmarshal(data, (*plain)(l))
dec := json.NewDecoder(bytes.NewReader(data))
dec.DisallowUnknownFields()

return dec.Decode((*plain)(l))
}

func (l *Limits) copyNotificationIntegrationLimits(defaults NotificationRateLimitMap) {
Expand Down
10 changes: 10 additions & 0 deletions pkg/util/validation/limits_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package validation
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"

Expand Down Expand Up @@ -165,6 +166,15 @@ func TestLimitsLoadingFromJson(t *testing.T) {

assert.Equal(t, 0.5, l.IngestionRate, "from json")
assert.Equal(t, 100, l.MaxLabelNameLength, "from defaults")

// Unmarshal should fail if input contains unknown struct fields and
// the decoder flag `json.Decoder.DisallowUnknownFields()` is set
inp = `{"unknown_fields": 100}`
l = Limits{}
dec := json.NewDecoder(strings.NewReader(inp))
dec.DisallowUnknownFields()
err = dec.Decode(&l)
assert.Error(t, err)
}

func TestLimitsTagsYamlMatchJson(t *testing.T) {
Expand Down