Skip to content

[receiver/sqlqueryreceiver]: break up datasource parameter into separate fields #39762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

justinianvoss22
Copy link
Contributor

@justinianvoss22 justinianvoss22 commented Apr 29, 2025

Description

  • Breaks up values used in datasource parameter. Sends values to receiver separately in order to process them separately.
  • Datasource is still required (as before). However, if datasource is not specified, the individual fields are used.
  • Allows for encryption, character escaping, and driver-specific processing of these fields.

Link to tracking issue

Fixes #39760

Testing

  • Unit tests:
    • scraper_test.go
    • config_test.go
    • receiver_test.go
  • Testing locally:
    • Set up agent with sqlreceiver running
    • Use this configuration:
            sqlquery:
              driver: postgres
              host: localhost
              port: 5432
              username: justinvoss
              database: bindplane18
              additional_params:
                sslmode: disable
              storage: file_storage
              queries:
                - sql: "select * from resources"
                  tracking_start_value: "10000"
                  logs:
                    - body_column: name
    
  • Using the old configuration should still work too:
           sqlquery:
               datasource: postgresql://justinvoss@localhost:5432/bindplane18?sslmode=disable
               driver: postgres
               queries:
                   - logs:
                       - attribute_columns:
                           - name
                           - kind
                       - body_column: data
                     sql: select * from resources```
                    
    
    

Documentation

  • Update README.md

@justinianvoss22 justinianvoss22 requested review from dmitryax, crobert-1 and a team as code owners April 29, 2025 20:05
Copy link

linux-foundation-easycla bot commented Apr 29, 2025

@justinianvoss22 justinianvoss22 marked this pull request as draft April 29, 2025 20:05
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch 4 times, most recently from 451b0c0 to 67d4161 Compare April 30, 2025 17:45
@djaglowski djaglowski changed the title [receiver/sqlreceiver]: break up datasource parameter into struct [receiver/sqlqueryreceiver]: break up datasource parameter into struct Apr 30, 2025
@justinianvoss22
Copy link
Contributor Author

justinianvoss22 commented Apr 30, 2025

@dmitryax @crobert-1 Originally, I converted the datasource parameter from a string into a struct. I realized it might be better to have a separate parameter, datasource_config, in order to not break existing functionality. Let me know your thoughts on this.

@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch from 3cbfc3c to a9cad77 Compare April 30, 2025 18:17
@justinianvoss22 justinianvoss22 changed the title [receiver/sqlqueryreceiver]: break up datasource parameter into struct [receiver/sqlqueryreceiver]: adddatasource_config parameter Apr 30, 2025
@justinianvoss22 justinianvoss22 changed the title [receiver/sqlqueryreceiver]: adddatasource_config parameter [receiver/sqlqueryreceiver]: add datasource_config parameter Apr 30, 2025
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch 2 times, most recently from 47b940e to fcf6f8f Compare May 1, 2025 15:07
@justinianvoss22
Copy link
Contributor Author

justinianvoss22 commented May 1, 2025

This URL: https://blogs.oracle.com/developers/post/connecting-a-go-application-to-oracle-database in receiver/sqlqueryreceiver/README.md is consistently failing the check-links PR check.

@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch 5 times, most recently from 6d11cf9 to ba3feb5 Compare May 6, 2025 12:42
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch from ba3feb5 to 9ce48f0 Compare May 14, 2025 14:39
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch 2 times, most recently from fa6e884 to bb772e2 Compare May 22, 2025 15:26
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch from 8e66f70 to 023c902 Compare June 27, 2025 15:00
@justinianvoss22
Copy link
Contributor Author

@crobert-1 Good to merge?

@crobert-1 crobert-1 added the ready to merge Code review completed; ready to merge by maintainers label Jun 27, 2025
@justinianvoss22
Copy link
Contributor Author

@crobert-1 I'm not authorized to push this branch, could I have you push this? Thanks!

@crobert-1
Copy link
Member

@crobert-1 I'm not authorized to push this branch, could I have you push this? Thanks!

Only contrib maintainers have permission to merge PRs, I don't have permission either. I've added the ready to merge label to signal to them that this is ready, it should be merged relatively soon. Feel free to post in #otel-collector-dev on CNCF Slack if it sits too long. 👍

@justinianvoss22
Copy link
Contributor Author

Ahh gotcha, thanks!

@mx-psi mx-psi merged commit 87aafa3 into open-telemetry:main Jun 27, 2025
187 checks passed
@github-actions github-actions bot added this to the next release milestone Jun 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal/sqlquery ready to merge Code review completed; ready to merge by maintainers receiver/sqlquery SQL query receiver waiting-for-code-owners
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[receiver/sqlqueryreceiver]: break up datasource parameter into separate fields
6 participants