Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to support strict JSON unmarshal for limits #4298

Merged
merged 3 commits into from
Jun 22, 2021

Conversation

kavirajk
Copy link
Contributor

@kavirajk kavirajk commented Jun 18, 2021

What this PR does:
Main problem is currently we cannot strict unmarshal JSON for Limits struct using json.DisallowUnknownFields(). Because custom UnmarshalJSON uses simple json.Unmarshal() function. (is it intentional?)

This PR Adds ability to support strict JSON unmarshal for limits struct

This is the default behaviour for YAML now.

	in := `{"unknown_fields": 100}`
	l := validation.Limits{}

	fmt.Println(yaml.UnmarshalStrict([]byte(in), &l))

	// yaml: unmarshal errors:
	// line 1: field unknown_fields not found in type validation.plain

This PR adds same behaviour if unmarshalled from JSON input as well.

Which issue(s) this PR fixes:
Fixes NA

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@kavirajk kavirajk force-pushed the limits-strict-unmarshal-json branch from 41628b6 to 7d5919a Compare June 18, 2021 14:44
kavirajk added 2 commits June 18, 2021 16:45
This is the default behaviour for YAML now.
```
	in := `{"unknown_fields": 100}`
	l := cortex.Limits{}

	fmt.Println(yaml.UnmarshalStrict([]byte(in), &l))

	// yaml: unmarshal errors:
	// line 1: field unknown_fields not found in type validation.plain
```

This PR adds same behaviour if unmarshalled from JSON input as well.

Signed-off-by: Kaviraj <[email protected]>
Signed-off-by: Kaviraj <[email protected]>
@kavirajk kavirajk force-pushed the limits-strict-unmarshal-json branch from 7d5919a to e74e600 Compare June 18, 2021 14:47
Copy link
Contributor

@stevesg stevesg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine assuming this is the behavior we want, but some points to consider:

  • Does mean that we lose forward compatibility? i.e. a "newer" configuration file (with unknown fields) cannot be parsed by an older version of Cortex. Whatever is producing the limits/overrides files, must be perfectly in sync with the Cortex version. This may be an acceptable trade-off.
  • Correct me if I'm wrong, but does this now mean that JSON parsing does not accept unknown fields, whilst the YAML parsing does? This perhaps is a little odd, I would expect both to have the same behavior.
  • Also [unrelated curiosity], given runtime_config.go:65, I wonder if it was actually intentional for the YAML parsing to accept unknown fields...

@pstibrany
Copy link
Contributor

  • Correct me if I'm wrong, but does this now mean that JSON parsing does not accept unknown fields, whilst the YAML parsing does? This perhaps is a little odd, I would expect both to have the same behavior.

As you point out in your next point, we already use strict parsing for YAML runtime config (which includes overrides), so this PR actually unifies both YAML and JSON parsing to strict.

Cortex itself doesn't use JSON parsing for limits, as far as I know.

@stevesg
Copy link
Contributor

stevesg commented Jun 21, 2021

Ah my mistake, I misread the UnmarshalYAML function. OK then it makes sense that both are strict.

@kavirajk
Copy link
Contributor Author

Thanks @stevesg @pstibrany!. Yes that's right this PR unifies both JSON and YAML strict with unknown fields!

Co-authored-by: Steve Simpson <[email protected]>
Signed-off-by: Kaviraj <[email protected]>
@kavirajk kavirajk force-pushed the limits-strict-unmarshal-json branch from c0bd56f to 53b51fb Compare June 22, 2021 09:33
Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@pracucci pracucci enabled auto-merge (squash) June 22, 2021 09:41
Copy link
Contributor

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!

@pracucci pracucci merged commit ceb4646 into cortexproject:master Jun 22, 2021
@kavirajk kavirajk deleted the limits-strict-unmarshal-json branch June 22, 2021 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants