-
Notifications
You must be signed in to change notification settings - Fork 2.9k
[receiver/sqlserver] add scraper to record the currently executing query in mssql #38632
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Please resolve conflict and build issues. |
Co-authored-by: Chao Weng <[email protected]>
d483276
to
bbdd2f4
Compare
the other pr has been merged, so i updated it accordingly and did a force push |
…tor-contrib into sample-queries
@crobert-1 i have updated based on the comments. please take a look. thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor request, looks good otherwise!
…tor-contrib into sample-queries
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thanks for all your work here @cuichenli!
…ery in mssql (open-telemetry#38632) Co-authored-by: @sincejune (CLA signed) This PR is associated with open-telemetry#37958 Since the other PR is still under review, we created this one separately to accelerate the review process. Reviewers who have completed the review of the previous PR can prioritize this one. #### Description We introduced Query Sample collection in this PR. The scraper will record all the currently executing queries once (in most case) and report related metrics. This enables users to monitor executed queries and correlate them with Top Query data for deeper insights into troubleshooting and performance optimization. #### Configuration We introduced one configuration for the feature(see receiver's README for details): 1. max_sample_query_count: this one is also added in the other PR. In this change, it is mainly used to configure the cache size for the reported samples. It would report the sample only if the query (identified by queryHash and queryPlanHash) is not in the cache #### New Log Attributes - `sqlserver.db_name` - `sqlserver.client_address` - `sqlserver.client_port` - `sqlserver.query_start` - `sqlserver.session_id` - `sqlserver.session_status` - `sqlserver.request_status` - `sqlserver.host_name` - `sqlserver.command` - `db.query.text` - `sqlserver.blocking_session_id` - `sqlserver.wait_type` - `sqlserver.wait_time` - `sqlserver.wait_resource` - `sqlserver.open_transaction_count` - `sqlserver.transaction_id` - `sqlserver.percent_complete` - `sqlserver.estimated_completion_time` - `sqlserver.cpu_time` - `sqlserver.total_elapsed_time` - `sqlserver.reads` - `sqlserver.writes` - `sqlserver.logical_reads` - `sqlserver.transaction_isolation_level` - `sqlserver.lock_timeout` - `sqlserver.deadlock_priority` - `sqlserver.row_count` - `sqlserver.query_hash` - `sqlserver.query_plan_hash` - `sqlserver.context_info` - `sqlserver.username` - `sqlserver.wait_code` - `sqlserver.wait_category` #### Additional dependency - hashicorp/golang-lru/v2 - License: MPL-2.0 - Link: [pkg.go.dev/github.com/hashicorp/golang-lru/v2](https://pkg.go.dev/github.com/hashicorp/golang-lru/v2) - Already been used in the repo - DataDog/datadog-agent/pkg/obfuscate - License: Apache 2.0 - Link: [pkg.go.dev/github.com/DataDog/datadog-agent/pkg/obfuscate](https://pkg.go.dev/github.com/DataDog/datadog-agent/pkg/obfuscate) - Already been used in the repo #### Example Output ```yaml resourceLogs: - resource: attributes: - key: db.system.type value: stringValue: microsoft.sql_server scopeLogs: - logRecords: - attributes: - key: sqlserver.db_name value: stringValue: master - key: sqlserver.client_address value: stringValue: "172.19.0.1" - key: sqlserver.client_port value: intValue: 59286 - key: sqlserver.query_start value: stringValue: 2025-02-12T16:37:54.843+08:00 - key: sqlserver.session_id value: intValue: 60 - key: sqlserver.session_status value: stringValue: running - key: sqlserver.request_status value: stringValue: running - key: sqlserver.host_name value: stringValue: DESKTOP-GHAEGRD - key: sqlserver.command value: stringValue: SELECT - key: db.query.text value: stringValue: "SELECT DB_NAME ( r.database_id ), ISNULL ( c.client_net_address, ? ), ISNULL ( c.client_tcp_port, ? ), CONVERT ( NVARCHAR, TODATETIMEOFFSET ( r.start_time, DATEPART ( TZOFFSET, SYSDATETIMEOFFSET ( ) ) ), ? ), s.session_id, s.STATUS, r.STATUS, ISNULL ( s.host_name, ? ), r.command, SUBSTRING ( o.TEXT, ( r.statement_start_offset / ? ) + ? ( ( CASE r.statement_end_offset WHEN - ? THEN DATALENGTH ( o.TEXT ) ELSE r.statement_end_offset END - r.statement_start_offset ) / ? ) + ? ), r.blocking_session_id, ISNULL ( r.wait_type, ? ), r.wait_time, r.wait_resource, r.open_transaction_count, r.transaction_id, r.percent_complete, r.estimated_completion_time, r.cpu_time, r.total_elapsed_time, r.reads, r.writes, r.logical_reads, r.transaction_isolation_level, r.LOCK_TIMEOUT, r.DEADLOCK_PRIORITY, r.row_count, r.query_hash, r.query_plan_hash, ISNULL ( r.context_info, CONVERT ( VARBINARY, ? ) ), s.login_name FROM sys.dm_exec_requests r INNER JOIN sys.dm_exec_sessions s ON r.session_id = s.session_id INNER JOIN sys.dm_exec_connections c ON s.session_id = c.session_id CROSS APPLY sys.dm_exec_sql_text ( r.plan_handle )" - key: sqlserver.blocking_session_id value: intValue: 0 - key: sqlserver.wait_type value: stringValue: "" - key: sqlserver.wait_time value: intValue: 0 - key: sqlserver.wait_resource value: stringValue: "" - key: sqlserver.open_transaction_count value: intValue: 0 - key: sqlserver.transaction_id value: intValue: 11089 - key: sqlserver.percent_complete value: doubleValue: 0 - key: sqlserver.estimated_completion_time value: doubleValue: 0 - key: sqlserver.cpu_time value: intValue: 6 - key: sqlserver.total_elapsed_time value: intValue: 6 - key: sqlserver.reads value: intValue: 0 - key: sqlserver.writes value: intValue: 0 - key: sqlserver.logical_reads value: intValue: 38 - key: sqlserver.transaction_isolation_level value: intValue: 2 - key: sqlserver.lock_timeout value: intValue: -1 - key: sqlserver.deadlock_priority value: intValue: 0 - key: sqlserver.row_count value: intValue: 1 - key: sqlserver.query_hash value: stringValue: "307837304133423133304231303438443444" - key: sqlserver.query_plan_hash value: stringValue: "307831343032313046363442373838434239" - key: sqlserver.context_info value: stringValue: "" - key: sqlserver.username value: stringValue: sa - key: sqlserver.wait_code value: intValue: 0 - key: sqlserver.wait_category value: stringValue: Unknown body: stringValue: sample scope: name: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/sqlserverreceiver version: v0.0.1 ``` #### Link to tracking issue Part of open-telemetry#36462 #### Testing Added #### Documentation Updated --------- Co-authored-by: Chao Weng <[email protected]>
…ery in mssql (open-telemetry#38632) Co-authored-by: @sincejune (CLA signed) This PR is associated with open-telemetry#37958 Since the other PR is still under review, we created this one separately to accelerate the review process. Reviewers who have completed the review of the previous PR can prioritize this one. #### Description We introduced Query Sample collection in this PR. The scraper will record all the currently executing queries once (in most case) and report related metrics. This enables users to monitor executed queries and correlate them with Top Query data for deeper insights into troubleshooting and performance optimization. #### Configuration We introduced one configuration for the feature(see receiver's README for details): 1. max_sample_query_count: this one is also added in the other PR. In this change, it is mainly used to configure the cache size for the reported samples. It would report the sample only if the query (identified by queryHash and queryPlanHash) is not in the cache #### New Log Attributes - `sqlserver.db_name` - `sqlserver.client_address` - `sqlserver.client_port` - `sqlserver.query_start` - `sqlserver.session_id` - `sqlserver.session_status` - `sqlserver.request_status` - `sqlserver.host_name` - `sqlserver.command` - `db.query.text` - `sqlserver.blocking_session_id` - `sqlserver.wait_type` - `sqlserver.wait_time` - `sqlserver.wait_resource` - `sqlserver.open_transaction_count` - `sqlserver.transaction_id` - `sqlserver.percent_complete` - `sqlserver.estimated_completion_time` - `sqlserver.cpu_time` - `sqlserver.total_elapsed_time` - `sqlserver.reads` - `sqlserver.writes` - `sqlserver.logical_reads` - `sqlserver.transaction_isolation_level` - `sqlserver.lock_timeout` - `sqlserver.deadlock_priority` - `sqlserver.row_count` - `sqlserver.query_hash` - `sqlserver.query_plan_hash` - `sqlserver.context_info` - `sqlserver.username` - `sqlserver.wait_code` - `sqlserver.wait_category` #### Additional dependency - hashicorp/golang-lru/v2 - License: MPL-2.0 - Link: [pkg.go.dev/github.com/hashicorp/golang-lru/v2](https://pkg.go.dev/github.com/hashicorp/golang-lru/v2) - Already been used in the repo - DataDog/datadog-agent/pkg/obfuscate - License: Apache 2.0 - Link: [pkg.go.dev/github.com/DataDog/datadog-agent/pkg/obfuscate](https://pkg.go.dev/github.com/DataDog/datadog-agent/pkg/obfuscate) - Already been used in the repo #### Example Output ```yaml resourceLogs: - resource: attributes: - key: db.system.type value: stringValue: microsoft.sql_server scopeLogs: - logRecords: - attributes: - key: sqlserver.db_name value: stringValue: master - key: sqlserver.client_address value: stringValue: "172.19.0.1" - key: sqlserver.client_port value: intValue: 59286 - key: sqlserver.query_start value: stringValue: 2025-02-12T16:37:54.843+08:00 - key: sqlserver.session_id value: intValue: 60 - key: sqlserver.session_status value: stringValue: running - key: sqlserver.request_status value: stringValue: running - key: sqlserver.host_name value: stringValue: DESKTOP-GHAEGRD - key: sqlserver.command value: stringValue: SELECT - key: db.query.text value: stringValue: "SELECT DB_NAME ( r.database_id ), ISNULL ( c.client_net_address, ? ), ISNULL ( c.client_tcp_port, ? ), CONVERT ( NVARCHAR, TODATETIMEOFFSET ( r.start_time, DATEPART ( TZOFFSET, SYSDATETIMEOFFSET ( ) ) ), ? ), s.session_id, s.STATUS, r.STATUS, ISNULL ( s.host_name, ? ), r.command, SUBSTRING ( o.TEXT, ( r.statement_start_offset / ? ) + ? ( ( CASE r.statement_end_offset WHEN - ? THEN DATALENGTH ( o.TEXT ) ELSE r.statement_end_offset END - r.statement_start_offset ) / ? ) + ? ), r.blocking_session_id, ISNULL ( r.wait_type, ? ), r.wait_time, r.wait_resource, r.open_transaction_count, r.transaction_id, r.percent_complete, r.estimated_completion_time, r.cpu_time, r.total_elapsed_time, r.reads, r.writes, r.logical_reads, r.transaction_isolation_level, r.LOCK_TIMEOUT, r.DEADLOCK_PRIORITY, r.row_count, r.query_hash, r.query_plan_hash, ISNULL ( r.context_info, CONVERT ( VARBINARY, ? ) ), s.login_name FROM sys.dm_exec_requests r INNER JOIN sys.dm_exec_sessions s ON r.session_id = s.session_id INNER JOIN sys.dm_exec_connections c ON s.session_id = c.session_id CROSS APPLY sys.dm_exec_sql_text ( r.plan_handle )" - key: sqlserver.blocking_session_id value: intValue: 0 - key: sqlserver.wait_type value: stringValue: "" - key: sqlserver.wait_time value: intValue: 0 - key: sqlserver.wait_resource value: stringValue: "" - key: sqlserver.open_transaction_count value: intValue: 0 - key: sqlserver.transaction_id value: intValue: 11089 - key: sqlserver.percent_complete value: doubleValue: 0 - key: sqlserver.estimated_completion_time value: doubleValue: 0 - key: sqlserver.cpu_time value: intValue: 6 - key: sqlserver.total_elapsed_time value: intValue: 6 - key: sqlserver.reads value: intValue: 0 - key: sqlserver.writes value: intValue: 0 - key: sqlserver.logical_reads value: intValue: 38 - key: sqlserver.transaction_isolation_level value: intValue: 2 - key: sqlserver.lock_timeout value: intValue: -1 - key: sqlserver.deadlock_priority value: intValue: 0 - key: sqlserver.row_count value: intValue: 1 - key: sqlserver.query_hash value: stringValue: "307837304133423133304231303438443444" - key: sqlserver.query_plan_hash value: stringValue: "307831343032313046363442373838434239" - key: sqlserver.context_info value: stringValue: "" - key: sqlserver.username value: stringValue: sa - key: sqlserver.wait_code value: intValue: 0 - key: sqlserver.wait_category value: stringValue: Unknown body: stringValue: sample scope: name: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/sqlserverreceiver version: v0.0.1 ``` #### Link to tracking issue Part of open-telemetry#36462 #### Testing Added #### Documentation Updated --------- Co-authored-by: Chao Weng <[email protected]>
Co-authored-by: @sincejune (CLA signed)
This PR is associated with #37958 Since the other PR is still under review, we created this one separately to accelerate the review process. Reviewers who have completed the review of the previous PR can prioritize this one.
Description
We introduced Query Sample collection in this PR. The scraper will record all the currently executing queries once (in most case) and report related metrics. This enables users to monitor executed queries and correlate them with Top Query data for deeper insights into troubleshooting and performance optimization.
Configuration
We introduced one configuration for the feature(see receiver's README for details):
New Log Attributes
sqlserver.db_name
sqlserver.client_address
sqlserver.client_port
sqlserver.query_start
sqlserver.session_id
sqlserver.session_status
sqlserver.request_status
sqlserver.host_name
sqlserver.command
db.query.text
sqlserver.blocking_session_id
sqlserver.wait_type
sqlserver.wait_time
sqlserver.wait_resource
sqlserver.open_transaction_count
sqlserver.transaction_id
sqlserver.percent_complete
sqlserver.estimated_completion_time
sqlserver.cpu_time
sqlserver.total_elapsed_time
sqlserver.reads
sqlserver.writes
sqlserver.logical_reads
sqlserver.transaction_isolation_level
sqlserver.lock_timeout
sqlserver.deadlock_priority
sqlserver.row_count
sqlserver.query_hash
sqlserver.query_plan_hash
sqlserver.context_info
sqlserver.username
sqlserver.wait_code
sqlserver.wait_category
Additional dependency
Example Output
Link to tracking issue
Part of #36462
Testing
Added
Documentation
Updated