Skip to content

feat: Add new logger module - Pubstack Analytics Module #1331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Aug 3, 2020
Merged

feat: Add new logger module - Pubstack Analytics Module #1331

merged 8 commits into from
Aug 3, 2020

Conversation

gpolaert
Copy link
Contributor

@gpolaert gpolaert commented Jun 3, 2020

Add a new logger module: Pubstack
Ability for users to send bid events directly to Pubstack.

@gpolaert
Copy link
Contributor Author

gpolaert commented Jun 4, 2020

Let me know what I have to do this that PR ;)

@SyntaxNode
Copy link
Contributor

Sweet! The first analytics module! :)

We'll review the PR tomorrow.

@gpolaert
Copy link
Contributor Author

gpolaert commented Jun 5, 2020

🙏 Let me know how we can improve this first version/draft ;)

Copy link
Contributor

@SyntaxNode SyntaxNode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's my initial pass. I'd also request that you add metrics for both influx and prometheus to give hosts visibiltiy into how this module is doing. Things like write counters, buffer sizes, and a best effort of total memory impact would be appreciated.

@SyntaxNode
Copy link
Contributor

Thank you for the first pass update. Heading in a good direction. The build passed on Go 1.12 and 1.13. It failed on 1.14 with the error:

[pubstack] Fail to initialize pubstack module, fail to acquire configuration

.. which looks like a failed http call. You'll want to stub out the http client to use canned responses in place of actual calls during the tests. My recommendation to allow configuration of the http client to avoid using the default client would help here.

@gpolaert gpolaert requested review from SyntaxNode and hhhjort June 29, 2020 07:39
gpolaert and others added 2 commits July 1, 2020 16:26
* V1 Pubstack (#7)

* feat: Add Pubstack Logger (#6)

* first version of pubstack analytics

* bypass viperconfig

* commit #1

* gofmt

* update configuration and make the tests pass

* add readme on how to configure the adapter and update the network calls

* update logging and fix intake url definition

* feat: Pubstack Analytics Connector

* fixing go mod

* fix: bad behaviour on appending path to auction url

* add buffering

* support bootstyrap like configuration

* implement route for all the objects

* supports termination signal handling for goroutines

* move readme to the correct location

* wording

* enable configuration reload + add tests

* fix logs messages

* fix tests

* fix log line

* conclude merge

* merge

* update go mod

Co-authored-by: Amaury Ravanel <[email protected]>

* fix duplicated channel keys

Co-authored-by: Amaury Ravanel <[email protected]>

* first pass - PR reviews

* rename channel* -> eventChannel

* dead code

* Review (#10)

* use json.Decoder

* update documentation

* use nil instead []byte("")

* clean code

* do not use http.DefaultClient

* fix race condition (need validation)

* separate the sender and buffer logics

* refactor the default configuration

* remove error counter

* Review GP + AR

* updating default config

* add more logs

* remove alias fields in json

* fix json serializer

* close event channels

Co-authored-by: Amaury Ravanel <[email protected]>
@gpolaert
Copy link
Contributor Author

gpolaert commented Jul 3, 2020

@SyntaxNode How can I re-enable travis?

@SyntaxNode
Copy link
Contributor

@SyntaxNode How can I re-enable travis?

It's enabled, but it's been a bit flaky with reporting back to GitHub. I'll kick it off again.

hhhjort
hhhjort previously approved these changes Jul 6, 2020
@gpolaert
Copy link
Contributor Author

Hello,

What's the next step here :)?

@SyntaxNode
Copy link
Contributor

Hello,

What's the next step here :)?

Every code PR needs two approvals before being merged. I'm assigned as the second reviewer, but I've had a meeting heavy week. Sorry. I'm unintentionally holding you up here. I'll give it another pass today or worst case over the weekend. I'd really like to get this included in the next release.

I greatly appreciate your responsiveness and support to the feedback. Building a module has a lot more complexity than an adapter.

Copy link
Contributor

@SyntaxNode SyntaxNode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great! I'm very happy with how you addressed naming and potential thread race issues. I have a few comments for explicitly defining with the mux's are protecting, passing around the http client via the struct so it can be subbed out for tests and for a new pool in the near future (I'll do that part after this merges in), and missing test coverage. We strive for 90% test coverage and the packages in this PR are currently closer to 50%.

@stale
Copy link

stale bot commented Jul 20, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jul 20, 2020
This reverts commit f57d9d6.

# Conflicts:
#	analytics/config/config_test.go
@gpolaert
Copy link
Contributor Author

gpolaert commented Aug 2, 2020

Thanks gyus! (@hhhjort and @SyntaxNode)
Your reviews help me a lot to better understand the specific of this language 😅

@gpolaert gpolaert requested review from SyntaxNode and hhhjort August 2, 2020 08:02
@hhhjort hhhjort merged commit deb19c3 into prebid:master Aug 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants