Skip to content

GDPR consentManagement module #2213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 52 commits into from
May 1, 2018
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
bcf27d8
initial commit
jsnellbaker Feb 14, 2018
cda55fe
wip update 2
jsnellbaker Feb 15, 2018
77de2e5
wip update 3
jsnellbaker Feb 15, 2018
ebe1ee8
example
mkendall07 Feb 15, 2018
3cda142
clean up
mkendall07 Feb 15, 2018
6337a2e
wip update 3
jsnellbaker Feb 16, 2018
78f0e9c
hook setup for callBids
jsnellbaker Feb 16, 2018
2412b4f
wip update 4
jsnellbaker Feb 20, 2018
b8c1da6
changed gdpr code to be async-like
jsnellbaker Feb 20, 2018
173d945
cleaned up the callback chain
mkendall07 Feb 20, 2018
6f57a89
added iab cmp detection logic
jsnellbaker Feb 20, 2018
83a4ed9
Merge branch 'master' into gdpr
jsnellbaker Feb 21, 2018
2a587eb
moved hook, reverted unit test changes, and restructed gdpr module
jsnellbaker Feb 23, 2018
2018378
renaming module from gdpr to consentManagement
jsnellbaker Feb 26, 2018
0081610
prebidserver adatper update, additional changes in module
jsnellbaker Feb 28, 2018
a46ca14
updated unit tests for all areas, updates to module logic and structu…
jsnellbaker Mar 1, 2018
6739fd0
adding missing default value
jsnellbaker Mar 6, 2018
63d0d21
removing accidentally committed load time testing code
jsnellbaker Mar 6, 2018
effc19c
changes to layout of consentManagement code and other items based on …
jsnellbaker Mar 9, 2018
e6d8068
moved unit test to different location
jsnellbaker Mar 9, 2018
326e712
finished incomplete unit test in appnexusBidAdapter_spec file
jsnellbaker Mar 9, 2018
112a61b
altered CMP function call logic
jsnellbaker Mar 13, 2018
91b6d83
refactored consentManagement AN lookup function and added gdprDataHan…
jsnellbaker Mar 19, 2018
a425228
some minor cleanup from previous commit
jsnellbaker Mar 19, 2018
f273018
change spacing to try to fix travis issue
jsnellbaker Mar 19, 2018
2e465fd
added scenario to support consentTimeout=0 skip setTimeout
jsnellbaker Mar 22, 2018
540b4b5
updated some comments
jsnellbaker Mar 23, 2018
4de3df0
refactored exit logic for module
jsnellbaker Mar 23, 2018
7f78734
added support for consentRequired field in config
jsnellbaker Mar 27, 2018
d6a4807
merge master into gdpr; fixed conflicts
jsnellbaker Mar 28, 2018
8d23307
remove internal consentRequired default
jsnellbaker Mar 29, 2018
2ccfedf
minor comment fixes
jsnellbaker Apr 2, 2018
a96b129
comment fixes that should be have part of last commit
jsnellbaker Apr 2, 2018
b7811f8
Merge branch 'master' into gdpr
jsnellbaker Apr 12, 2018
9a1f09b
fix includes issue and added gdprConsent to getUserSyncs function
jsnellbaker Apr 12, 2018
73d02f0
Merge branch 'master' into gdpr
jsnellbaker Apr 12, 2018
5ae5eff
renamed default CMP and config field to cmpApi
jsnellbaker Apr 12, 2018
78fcc64
wip - using postmessage to call cmp
jsnellbaker Apr 13, 2018
c39b12d
postMessage workflow added, removed CMP eventlistener check
jsnellbaker Apr 13, 2018
213f2fa
removed if statement
jsnellbaker Apr 13, 2018
405a6f2
cleanup; removed variable and unneeded comments
jsnellbaker Apr 16, 2018
fc41a5a
add gdpr tests pages
jsnellbaker Apr 18, 2018
296e9ca
merge 'master' into branch 'gdpr' + resolve conflict
jsnellbaker Apr 18, 2018
750797a
updates for 1.1 CMP spec
jsnellbaker Apr 24, 2018
effa5b5
remove rogue debugger in unit test
jsnellbaker Apr 24, 2018
a3ca63f
restructured 1.1 CMP iframe code, renamed utils function, cleaned up …
jsnellbaker Apr 26, 2018
4bc0f9b
GDPR support in adform adapter (#2396)
Pupis Apr 27, 2018
4a6e273
merge branch 'master' into branch 'gdpr' + resolve conflicts
jsnellbaker Apr 27, 2018
2f32574
Add gdpr support for PubMaticBidAdapter (#2469)
PubMatic-OpenWrap Apr 30, 2018
f042d7c
GDPR support for AOL adapter (#2443)
vzhukovsky Apr 30, 2018
56f6df4
removing iframe example pages
jsnellbaker Apr 30, 2018
1a35235
comment updates
jsnellbaker Apr 30, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions modules/appnexusBidAdapter.js
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,13 @@ export const spec = {
if (member > 0) {
payload.member_id = member;
}
if (bidderRequest && bidderRequest.gdprConsent) {
payload.gdprConsent = {
gdprConsentString: bidderRequest.gdprConsent.consentString,
gdprConsentRequired: bidderRequest.gdprConsent.consentRequired
};
}

const payloadString = JSON.stringify(payload);
return {
method: 'POST',
Expand Down
142 changes: 142 additions & 0 deletions modules/consentManagement.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
import * as utils from 'src/utils';
import { config } from 'src/config';
import { setTimeout } from 'core-js/library/web/timers';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of this? All the browsers we support seem to support setTimeout in the way you are using it w/o this polyfill.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed it, it got added automatically.


// add new CMPs here
const availCMPs = ['iab'];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably call this appnexus and not iab

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated references of iab to appnexus


export let userCMP;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these vars need to be exported?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm exporting these vars so I can read them in the consentManagement_spec.js test file for the setConfig unit tests.

export let consentId = '';
export let consentTimeout;
export let lookUpFailureChoice;

export function requestBidsHook(config, fn) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would like to see JSdocs notation on exported functions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some more descriptive information to the exported functions.

let adUnits = config.adUnits || $$PREBID_GLOBAL$$.adUnits;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be using $$PREBID_GLOBAL$$ here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the $$PREBID_GLOBAL$$ fall back. I found this type of setup from another module that was hooked on the requestBids function and wanted to keep consistency.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update I need to have this value in place to actually grab the adUnits list object.

The config attribute that's included in the requestBids has a place for adUnits, but it's dependent on it being passed specifically in the prebid config when the requestBids is invoked. Currently only the bidsBackHandler is passed along with requestBids in most of our example docs, so I expect the adUnits is going to be undefined at this stage of the auction process.

@mkendall07 Please let me know if there are any strong concerns about relying on $$PREBID_GLOBAL$$.adUnits.

let context = this;
let args = arguments;
let nextFn = fn;
let cmpActive = true;
let _timer;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use underscore consistently please. All or none

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the underscore for this var to make it consistent with the others.


// in case we already have consent (eg during bid refresh)
if (consentId) {
adUnits = applyConsent(consentId);
return fn.apply(context, args);
}

// ensure user requested supported cmp, otherwise abort and return to hooked function
if (!availCMPs.includes(userCMP)) {
utils.logWarn(`CMP framework ${userCMP} is not a supported framework. Aborting consentManagement module and resuming auction.`);
return fn.apply(context, args);
}

// start of CMP specific logic
if (userCMP === 'iab') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move this logic to a separate function and define at the top of the file. That will make it easier / cleaner for other CMPs to integrate.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reorganized the functions, please take a look on the next commit and let me know what you think.

if (!window.__cmp) {
utils.logWarn('IAB CMP framework is not detected. Aborting consentManagement module and resuming auction.');
return fn.apply(context, args);
}

lookupIabId();
}

function lookupIabId () {
// lookup times and user interaction with CMP prompts can greatly vary, so enforcing a timeout on the CMP process
_timer = setTimeout(cmpTimedOut, consentTimeout);

// first lookup - to determine if new or existing user
// if new user, then wait for user to make a choice and then run postLookup method
// if existing user, then skip to postLookup method
window.__cmp('getConsentData', 'vendorConsents', function(consentString) {
if (cmpActive) {
if (consentString === null || !consentString) {
window.__cmp('addEventListener', 'onSubmit', function() {
if (cmpActive) {
// redo lookup to find new string based on user's choices
window.__cmp('getConsentData', 'vendorConsents', postLookup);
}
});
} else {
postLookup(consentString);
}
}
});
}

// after we have grabbed ideal ID from CMP, apply the data to adUnits object and finish up the module
function postLookup(consentString) {
if (cmpActive) {
if (typeof consentString != 'string' || consentString === '') {
exitFailedCMP(`CMP returned unexpected value during lookup process; returned value was (${consentString}).`);
} else {
clearTimeout(_timer);
consentId = consentString;
adUnits = applyConsent(consentString);
fn.apply(context, args);
}
}
}

// assuming we have valid consent ID, apply to adUnits object
function applyConsent(consentString) {
adUnits.forEach(adUnit => {
adUnit['gdprConsent'] = {
consentString: consentString,
consentRequired: true
};
});
return adUnits;
}

function cmpTimedOut () {
if (cmpActive) {
exitFailedCMP('CMP workflow exceeded timeout threshold.');
}
}

function exitFailedCMP(message) {
cmpActive = false;
if (lookUpFailureChoice === 'proceed') {
utils.logWarn(message + ' Resuming auction without consent data as per consentManagement config.');
adUnits = applyConsent(undefined);
nextFn.apply(context, args);
} else {
utils.logError(message + ' Canceling auction as per consentManagement config.');
}
}
}

export function resetConsentId() {
consentId = '';
}

export function setConfig(config) {
if (typeof config.cmp === 'string') {
userCMP = config.cmp;
} else {
userCMP = 'iab';
utils.logInfo(`consentManagement config did not specify cmp. Using system default setting (${userCMP}).`);
}

if (typeof config.waitForConsentTimeout === 'number') {
consentTimeout = config.waitForConsentTimeout;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should just be called timeout. It's already under the consentManagement property, so we know it relates to that and the way it's worded here makes the value expected ambiguous (I would guess a bool w/o looking at the code)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I changed the name to timeout.

} else {
consentTimeout = 5000;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defaults like these should be made constant values and defined at the top of the file

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created default variables and utilized them in the setConfig part of the module code in place of the actual values.

utils.logInfo(`consentManagement config did not specify waitForConsentTimeout. Using system default setting (${consentTimeout}).`);
}

if (typeof config.lookUpFailureResolution === 'string') {
if (config.lookUpFailureResolution === 'proceed' || config.lookUpFailureResolution === 'cancel') {
lookUpFailureChoice = config.lookUpFailureResolution;
} else {
lookUpFailureChoice = 'proceed';
utils.logWarn(`Invalid choice was set for consentManagement lookUpFailureResolution property. Using system default (${lookUpFailureChoice}).`);
}
} else {
lookUpFailureChoice = 'proceed';
utils.logInfo(`consentManagement config did not specify lookUpFailureResolution. Using system default setting (${lookUpFailureChoice}).`);
}

$$PREBID_GLOBAL$$.requestBids.addHook(requestBidsHook, 50);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always forget what the addHook signature looks like. What's the 50 value signify?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above for the change in the pre1api file.

}
config.getConfig('consentManagement', config => setConfig(config.consentManagement));
2 changes: 1 addition & 1 deletion modules/pre1api.js
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ pbjs.requestBids.addHook((config, next = config) => {
} else {
logWarn(`${MODULE_NAME} module: concurrency has been disabled and "$$PREBID_GLOBAL$$.requestBids" call was queued`);
}
}, 100);
}, 5);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the impact of this change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The number (also relates to the other comment below) acts as a priority level for the hooked functions (when there are multiple hooked functions on the same hook).

The pbjs.requestBids function has 3 hooked functions from 3 different pbjs modules: pre1api, PubCommonId and consentManagement. So lowering this priority level ensures that the pre1api module's hooked function will (generally) go last in the sequence if all these modules were enabled together.

Having the pre1api module go last (specifically after consentManagement) is related to the need for the pre1api to execute right before the pbjs.requestBids so it knows which auction is currently active. The consentManagement module uses/waits on callbacks (to retrieve information from the CMP), and this buffering of the auction process could cause issues for the pre1api knowing which auction is active if there was a time gap because it ran first. So the lower priority helps avoid this scenario.

The 5 specifically is lower than the default priority that's set for any hooked function (which is 10), but still higher than the base function pbjs.requestBids (which is 0). So it should generally always the be last hook to run before the pbjs.requestBids would execute.


Object.keys(auctionPropMap).forEach(prop => {
if (prop === 'allBidsAvailable') {
Expand Down
11 changes: 8 additions & 3 deletions modules/prebidServerBidAdapter.js
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ function _getDigiTrustQueryParams() {
*/
const LEGACY_PROTOCOL = {

buildRequest(s2sBidRequest, adUnits) {
buildRequest(s2sBidRequest, bidRequests, adUnits) {
// pbs expects an ad_unit.video attribute if the imp is video
adUnits.forEach(adUnit => {
const videoMediaType = utils.deepAccess(adUnit, 'mediaTypes.video');
Expand Down Expand Up @@ -381,7 +381,7 @@ const OPEN_RTB_PROTOCOL = {

bidMap: {},

buildRequest(s2sBidRequest, adUnits) {
buildRequest(s2sBidRequest, bidRequests, adUnits) {
let imps = [];

// transform ad unit into array of OpenRTB impression objects
Expand Down Expand Up @@ -456,6 +456,11 @@ const OPEN_RTB_PROTOCOL = {
request.user = { ext: { digitrust: digiTrust } };
}

if (bidRequests && bidRequests[0].gdprConsent) {
request.regs = { ext: { gdpr: bidRequests[0].gdprConsent.consentRequired ? 1 : 0 } };
request.user = { ext: { consent: bidRequests[0].gdprConsent.consentString } };
}

return request;
},

Expand Down Expand Up @@ -556,7 +561,7 @@ export function PrebidServer() {
.reduce(utils.flatten)
.filter(utils.uniques);

const request = protocolAdapter().buildRequest(s2sBidRequest, adUnitsWithSizes);
const request = protocolAdapter().buildRequest(s2sBidRequest, bidRequests, adUnitsWithSizes);
const requestJson = JSON.stringify(request);

ajax(
Expand Down
5 changes: 5 additions & 0 deletions src/adaptermanager.js
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,11 @@ exports.makeBidRequests = function(adUnits, auctionStart, auctionId, cbTimeout,
bidRequests.push(bidderRequest);
}
});
if (adUnits[0].gdprConsent) {
bidRequests.forEach(bidRequest => {
bidRequest['gdprConsent'] = adUnits[0].gdprConsent;
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there's a better way to do this....

}
return bidRequests;
};

Expand Down
20 changes: 18 additions & 2 deletions test/spec/modules/appnexusBidAdapter_spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ describe('AppNexusAdapter', () => {
});
});

it('should attache native params to the request', () => {
it('should attach native params to the request', () => {
let bidRequest = Object.assign({},
bidRequests[0],
{
Expand Down Expand Up @@ -290,7 +290,7 @@ describe('AppNexusAdapter', () => {
}]);
});

it('should should add payment rules to the request', () => {
it('should add payment rules to the request', () => {
let bidRequest = Object.assign({},
bidRequests[0],
{
Expand All @@ -306,6 +306,22 @@ describe('AppNexusAdapter', () => {

expect(payload.tags[0].use_pmt_rule).to.equal(true);
});

it('should gdpr consent information to the request', () => {
let bidderRequest = {
'bidderCode': 'appnexus',
'auctionId': '1d1a030790a475',
'bidderRequestId': '22edbae2733bf6',
'timeout': 3000,
'consentId': 'BOJ8RZsOJ8RZsABAB8AAAAAZ+A=='
};
bidderRequest.bids = bidRequests;

const request = spec.buildRequests(bidRequests, bidderRequest);
const payload = JSON.parse(request.data);

// add expect checks - new spec will likely have the consent data in the request.data field
});
})

describe('interpretResponse', () => {
Expand Down
Loading