Skip to content

Request Provided Currency Rates #1753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Jun 10, 2021
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
2989689
First draft
guscarreon Feb 23, 2021
4d3196c
update with latest master
guscarreon Feb 23, 2021
f50db9f
Last test case working
guscarreon Mar 15, 2021
1e69876
Merge branch 'PBS-698' of https://github.com/guscarreon/prebid-server…
Mar 15, 2021
40afa09
Added Json test, corrected descriptions
Mar 16, 2021
5605a98
Mansi's review and resolve merge conflict
guscarreon Mar 23, 2021
57dde30
resolved openrtb2 merge conflicts
Mar 26, 2021
8b17152
Added inverse custom conversion rates
Apr 12, 2021
86b4350
Scott's eedback
Apr 13, 2021
95c82d3
Corrected error assertion
Apr 13, 2021
177e1e1
errortypes import in exchange.go
Apr 13, 2021
c1f136a
Merge branch 'master' into PBS-698
Apr 13, 2021
c0c81d7
Corrected test to not have a nil e.bidIDGenerator
Apr 13, 2021
5376694
update with latest master
Apr 16, 2021
348d83b
Created rateEngine Rates implementation and reingeneered getAuctionCu…
May 5, 2021
ed48b9c
remove .swp file
May 5, 2021
dc36e68
Corrected test cases
May 5, 2021
8f7df0a
Merge branch 'master' into PBS-698
May 5, 2021
3731241
Corrected native json test
May 5, 2021
00dda98
Scott's review. Renaming objects, getting rid of []Conversions
May 7, 2021
72aa90c
Merge branch 'master' into PBS-698
May 7, 2021
a7bba04
Corrected expected error message
May 7, 2021
9adf8b2
Renamed GroupedConversions to AggregateConversions and rate_engines.g…
May 11, 2021
11fa5d2
Removed logically dead code in getAuctionCurrencyRates
May 17, 2021
6ff91ed
Merge branch 'master' into PBS-698
May 21, 2021
e454459
Implemented NoConversionRate error type and Scott's review
May 24, 2021
b60b68f
Conversions disabled logic, warning and aggregateConversions GetRate …
May 25, 2021
561dff8
Mansi's review June 1st
Jun 2, 2021
b3db4a2
Mansi's review Jun 8th
Jun 8, 2021
663dde0
Scott's review June 8th
Jun 9, 2021
32a3a22
Scott's review June 9th
Jun 9, 2021
c9ee8e1
Mansi's feedback in regards of file and directory naming and addition…
Jun 9, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions currency/rates.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,10 @@ func (r *Rates) GetRate(from string, to string) (float64, error) {
if r.Conversions != nil {
if conversion, present := r.Conversions[fromUnit.String()][toUnit.String()]; present {
// In case we have an entry FROM -> TO
return conversion, err
return conversion, nil
} else if conversion, present := r.Conversions[toUnit.String()][fromUnit.String()]; present {
// In case we have an entry TO -> FROM
return 1 / conversion, err
return 1 / conversion, nil
}
return 0, fmt.Errorf("Currency conversion rate not found: '%s' => '%s'", fromUnit.String(), toUnit.String())
}
Expand Down
50 changes: 50 additions & 0 deletions endpoints/openrtb2/auction.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"net/url"
"regexp"
"strconv"
"strings"
"time"

"github.com/buger/jsonparser"
Expand All @@ -37,6 +38,7 @@ import (
"github.com/prebid/prebid-server/util/httputil"
"github.com/prebid/prebid-server/util/iputil"
"golang.org/x/net/publicsuffix"
"golang.org/x/text/currency"
)

const storedRequestTimeoutMillis = 50
Expand Down Expand Up @@ -340,6 +342,10 @@ func (deps *endpointDeps) validateRequest(req *openrtb2.BidRequest) []error {
if err := deps.validateEidPermissions(bidExt, aliases); err != nil {
return []error{err}
}

if err := validateCustomRates(bidExt.Prebid.BidReqConversions); err != nil {
return []error{err}
}
}

if (req.Site == nil && req.App == nil) || (req.Site != nil && req.App != nil) {
Expand Down Expand Up @@ -434,6 +440,50 @@ func validateSChains(req *openrtb_ext.ExtRequest) error {
return err
}

// validateCustomRates discards invalid 3-digit currency codes found in the bidRequest.ext.prebid.currency
// field. If all of them were discarded and bidRequest.ext.prebid.currency was set to false, it means
// we have no currencies to work with and this function returns a fatal BadInput error
func validateCustomRates(bidReqCurrencyRates *openrtb_ext.ExtRequestCurrency) error {
if bidReqCurrencyRates == nil {
return nil
}

var badCurrencyCodes []string

for fromCurrency, rates := range bidReqCurrencyRates.ConversionRates {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Extra line break.

// Check if fromCurrency is a valid 3-letter currency code
if _, err := currency.ParseISO(fromCurrency); err != nil {
badCurrencyCodes = append(badCurrencyCodes, fromCurrency)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we throw errors on invalid currency codes instead of silently removing them? .. or perhaps throw a warning?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is throwing a fatal error only if all of the custom currency codes were found to be wrong. The reason this approach made more sense to me is that we ignore malformed or inaccurate 3-digit currency codes inside requestBid() anyways.

197                 var conversionRate float64
198                 var err error
199                 for _, bidReqCur := range request.Cur {
200                     if conversionRate, err = conversions.GetRate(bidResponse.Currency, bidReqCur); err == nil {
201                         seatBid.currency = bidReqCur
202                         break
203                     }
204                 }
exchange/bidder.go

But, I'm open to ideas here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's seek out @hhhjort @mansinahar @bsardo for additional thoughts here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I agree with @SyntaxNode here that we should at least append an error/warning about the bad currency codes so that the user knows what exactly the problem was because right now we just remove them which might eventually lead to a currency not found error and that doesn't indicate the exact issue.

Consider for example, someone wanted to provide and override for EUR to USD and by mistake they spelled USD as UST in their request, we'd remove that entry from the currency rates and then if we do get a bid in USD we won't be able to find the currency rate and return a currency not found error. This is kinda misleading, so I'd suggest we either return an error right here in the validating request step and mark the request invalid or we at least add a warning message in the response that points out the invalid currency code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modified this function to throw fatal bad input error at the first invalid or malformed currency code found in the map

delete(bidReqCurrencyRates.ConversionRates, fromCurrency)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once we've recognized the current fromCurrency as an invalid currency and removed from the list, is there any reason to still go through the toCurrencies for it? I believe we should just continue to the next entry in the map after deleting this entry.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Added a continue to skip to next entry

}

// Check if currencies mapped to fromCurrency are valid 3-letter currency codes
for toCurrency := range rates {
if _, err := currency.ParseISO(toCurrency); err != nil {
badCurrencyCodes = append(badCurrencyCodes, toCurrency)
delete(rates, toCurrency)
}
}

// If all of fromCurrency's mapped currencies were invalid, remove fromCurrency map entry
if len(rates) == 0 {
delete(bidReqCurrencyRates.ConversionRates, fromCurrency)
}
}

// If we are retricted to only use custom currency rates but the currency rate map is empty, return error
customRatesOnly := bidReqCurrencyRates.UsePBSRates != nil && !(*bidReqCurrencyRates.UsePBSRates)
if customRatesOnly && len(bidReqCurrencyRates.ConversionRates) == 0 {
if len(badCurrencyCodes) > 0 {
return &errortypes.BadInput{Message: fmt.Sprintf("Required custom currency rates are all invalid. %s", strings.Join(badCurrencyCodes, ","))}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might I suggest a different wording, to key on that this is a required field only because UsePBSRates is set to true in their request and that possibly not all currency names are invalid. Something along the lines of:

ext.prebid.currency.usePbsRates is set to false and all ext.prebid,currency.rates are invalid. invalid ISO-4217 code: %s 

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But to reach this branch bidReqCurrencyRates.ConversionRates must be empty which means either it was empty from the beginning, which leads to returning the error in line 484, or it got emptied after finding all of the 3-letter currency codes to be wrong that resolves in returning error in 482. Do you agree with this approach? We could still reword the error message or even refactor this part if needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's necessarily true. As the code is currently, a ConversionRates entry will be removed if the base currency is wrong even if the mapped currencies are correct. For example:

ConversionRates: map[string]map[string]float64{
    "FOO": {
        "GBP": 1.2,
        "MXN": 0.05,
        "JPY": 0.95,
    },
}

If I'm reading the code correctly, this will cause the error message to be reached when UsePBSRates is true. While the base currency of FOO is invalid, the mapped currencies of GBP, MXN, and JPY are valid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part of the code has been removed because we are now immediately returning a bad input error when the first non-valid code is found.

} else {
return &errortypes.BadInput{Message: "Required custom currency rates field is empty"}
}
}
return nil
}

func (deps *endpointDeps) validateEidPermissions(req *openrtb_ext.ExtRequest, aliases map[string]string) error {
if req == nil || req.Prebid.Data == nil {
return nil
Expand Down
Loading