API to list console sessions #11016

bernardodemarco · 2025-06-12T15:37:30Z

Description

Currently, details about console sessions are stored in the cloud.console_session table in the database. Operators can only access this information by querying the database directly, while end users have no way to view console session data at all.

To address this, this PR proposes to create the listConsoleSessions API. It allows listing the console sessions, with optional filters by domain, account, user, host, instance, IP address, and date. The API is accessible to all account types and, thus, performs proper access validation on the queried resources.

The API supports the following parameters:

Parameter	Description	Required
`id`	Console session ID.	No
`activeonly`	Indicates whether only currently active console sessions should be listed. Defaults to `true`. Active sessions are the ones that have been acquired and have not been removed.	No
`isrecursive`	Lists console sessions recursively per domain. Defaults to `false`.	No
`clientaddress`	IP address of the client that accessed the console.	No
`consoleendpointcreatoraddress`	IP address of the client that generated the console session endpoint.	No
`hostid`	ID of the host where the VM was running when the console session endpoint was generated. Only available for Root Admins.	No
`instanceid`	ID of the VM associated with the console session.	No
`startdate`	If provided, only console sessions that were acquired from this date onward will be listed.	No
`enddate`	If provided, only console sessions that were acquired up to this date will be listed.	No
`domainid`	Domain ID of the account that generated the console session endpoint.	No
`accountid`	ID of the account that generated the console session endpoint.	No
`userid`	ID of the user that generated the console session endpoint.	No
`page`	Used for pagination along with `pagesize`.	No
`pagesize`	User for pagination along with `page`.	No

This PR only encompasses the creation of the API listConsoleSessions API. UI support will be implemented in a future PR.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Screenshots (if appropriate):

How Has This Been Tested?

Tests Setup

Created the following domain hierarchy:

ROOT
├── d1
│   └── d1-d1
└── d2

Created the following accounts:

Name	Role Type	Domain
admin	Admin	ROOT
u1	User	ROOT
d1-admin	DomainAdmin	ROOT/d1
d1-user	User	ROOT/d1
d1-d1-admin	DomainAdmin	ROOT/d1/d1-d1
d1-d1-user	User	ROOT/d1/d1-d1
d2-admin	DomainAdmin	ROOT/d2

Deployed a VM for each account:

ID	Name	Instance Name	Account Name
5	v-5-VM	v-5-VM	system
6	s-6-VM	s-6-VM	system
7	VM-fb5fa7ab-c5a0-4943-88bd-dedec9007c7b	i-7-7-VM	d1-d1-user
8	r-8-VM	r-8-VM	d1-d1-user
9	VM-16febbcd-a35e-4f05-87d3-1a27db077136	i-6-9-VM	d1-user
10	r-10-VM	r-10-VM	d1-user
11	VM-d3208889-d3fd-43fe-9087-3cf5d863a3b1	i-4-11-VM	d1-admin
12	r-12-VM	r-12-VM	d1-admin
13	VM-dea9f800-afd0-4054-9160-3a676a0df3d3	i-2-13-VM	admin
14	r-14-VM	r-14-VM	admin
15	VM-766b6485-330e-4ec9-9516-59e17b52c1c7	i-8-15-VM	u1
16	r-16-VM	r-16-VM	u1
17	VM-d2d7ded1-e210-4191-9a0e-75c3914de043	i-9-17-VM	d2-admin
18	r-18-VM	r-18-VM	d2-admin
19	VM-266b73ff-2684-46b9-b36e-830ac83c5c14	i-5-19-VM	d1-d1-admin
20	r-20-VM	r-20-VM	d1-d1-admin

With each account, I accessed all available VM consoles through the CPVM.

`listConsoleSessions` tests

With the admin account, verified that the API lists console sessions correctly according to the specified parameters
With the u1 account, verified that it is only possible to access the account's console sessions
With the d2-admin account, verified that it is only possible to list the console sessions of the d2 domain
With the d1-admin account, verified that it is only possible to list the console sessions of the d1 and d1/d1-d1 domains
With the d1-user account, verified that it is only possible to list the d1-user console sessions
With the d1-d1-admin account, verified that it is only possible to list the console sessions of the d1/d1-d1 domain
With the d1-d1-user account, verified that it is only possible to list the d1-d1-user console sessions
With accounts of the User type, verified that the hostid parameter is not considered in the API workflow
With accounts of the User type, verified that the hostid and hostname response attributes are not included in the API's return

bernardodemarco · 2025-06-12T15:38:23Z

@blueorangutan package

codecov · 2025-06-12T15:45:16Z

Codecov Report

Attention: Patch coverage is 55.30547% with 139 lines in your changes missing coverage. Please review.

Project coverage is 16.59%. Comparing base (0bd35a5) to head (69c21a0).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...loudstack/api/response/ConsoleSessionResponse.java	50.48%	51 Missing ⚠️
...n/java/com/cloud/vm/dao/ConsoleSessionDaoImpl.java	0.00%	37 Missing ⚠️
...mand/user/consoleproxy/ListConsoleSessionsCmd.java	52.63%	27 Missing ⚠️
...src/main/java/com/cloud/api/ApiResponseHelper.java	81.03%	5 Missing and 6 partials ⚠️
...a/src/main/java/com/cloud/vm/ConsoleSessionVO.java	14.28%	6 Missing ⚠️
...udstack/consoleproxy/ConsoleAccessManagerImpl.java	87.50%	5 Missing and 1 partial ⚠️
...in/java/com/cloud/server/ManagementServerImpl.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #11016      +/-   ##
============================================
+ Coverage     16.57%   16.59%   +0.02%     
- Complexity    13968    13999      +31     
============================================
  Files          5743     5745       +2     
  Lines        510470   510802     +332     
  Branches      62074    62092      +18     
============================================
+ Hits          84615    84793     +178     
- Misses       416393   416539     +146     
- Partials       9462     9470       +8

Flag	Coverage Δ
uitests	`3.90% <ø> (ø)`
unittests	`17.49% <55.30%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

bernardodemarco · 2025-06-12T18:02:52Z

@blueorangutan package

blueorangutan · 2025-06-12T18:04:03Z

@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-12T19:05:38Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13755

bernardodemarco · 2025-06-13T18:26:30Z

@blueorangutan package

blueorangutan · 2025-06-13T18:28:06Z

@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

DaanHoogland

clgtm, only doubt, is the active parameter needed? would we ever want to list no longer available sessions?

blueorangutan · 2025-06-13T18:54:26Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13766

blueorangutan · 2025-06-13T19:37:33Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13771

bernardodemarco · 2025-06-13T21:50:46Z

@DaanHoogland, thanks for the review!

is the active parameter needed? would we ever want to list no longer available sessions?

Yes, listing removed sessions is useful for audit and analysis purposes, as it allows users to track who generated a console endpoint, who accessed it, and when it was generated, acquired, and removed. Listing only active sessions, on the other hand, is helpful for verifying whether someone is currently using a VM.

blueorangutan · 2025-06-14T10:37:50Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13777

DaanHoogland · 2025-06-14T19:12:34Z

@blueorangutan test

blueorangutan · 2025-06-14T19:14:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-06-15T11:19:33Z

[SF] Trillian test result (tid-13527)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54933 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11016-t13527-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland · 2025-06-16T07:12:15Z

@blueorangutan test ol8 vmware-70u3 keepEnv

blueorangutan · 2025-06-16T07:14:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + vmware-70u3) has been kicked to run smoke tests

bernardodemarco · 2025-06-24T22:19:43Z

@blueorangutan package

blueorangutan · 2025-06-24T22:22:03Z

@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-24T23:29:02Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13902

DaanHoogland · 2025-06-25T07:41:28Z

@blueorangutan test ol8 vmware-70u3 keepEnv

blueorangutan · 2025-06-25T07:42:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + vmware-70u3) has been kicked to run smoke tests

blueorangutan · 2025-06-26T09:19:40Z

[SF] Trillian test result (tid-13599)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 89919 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11016-t13599-vmware-70u3.zip
Smoke tests completed. 130 look OK, 11 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_nic_secondaryip_add_remove	`Error`	1517.73	test_multipleips_per_nic.py
ContextSuite context=TestNestedVirtualization>:setup	`Error`	0.00	test_nested_virtualization.py
ContextSuite context=TestNetworkACL>:setup	`Error`	0.00	test_network_acl.py
ContextSuite context=TestIpv6Network>:setup	`Error`	0.00	test_network_ipv6.py
test_delete_account	`Error`	1517.96	test_network.py
test_delete_network_while_vm_on_it	`Error`	1.21	test_network.py
test_deploy_vm_l2network	`Error`	1.19	test_network.py
test_l2network_restart	`Error`	2.33	test_network.py
ContextSuite context=TestPortForwarding>:setup	`Error`	3.57	test_network.py
ContextSuite context=TestPublicIP>:setup	`Error`	10.98	test_network.py
test_reboot_router	`Failure`	0.10	test_network.py
test_releaseIP	`Error`	5.65	test_network.py
test_releaseIP_using_IP	`Error`	5.32	test_network.py
ContextSuite context=TestRouterRules>:setup	`Error`	5.39	test_network.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup	`Error`	1524.51	test_network.py
ContextSuite context=TestPrivateGwACL>:setup	`Error`	0.00	test_privategw_acl.py
ContextSuite context=TestAdapterTypeForNic>:setup	`Error`	0.00	test_nic_adapter_type.py
ContextSuite context=TestNonStrictAffinityGroups>:setup	`Error`	0.00	test_nonstrict_affinity_group.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup	`Error`	0.00	test_password_server.py
ContextSuite context=TestPortForwardingRules>:setup	`Error`	0.00	test_portforwardingrules.py
ContextSuite context=TestProjectSuspendActivate>:setup	`Error`	1528.61	test_projects.py

DaanHoogland · 2025-06-26T10:18:13Z

@bernardodemarco , I am trying this in a vmware lab env ^ but get an empty result. Is there some setting that I should apply first?

I am doing the very basic test:

cmk list consolesessions ==> empty result
start a console for the ssvm
cmk list consolesessions ==> empty result

I’d expect an extry on the second list command.

weizhouapache · 2025-06-26T10:54:58Z

@bernardodemarco , I am trying this in a vmware lab env ^ but get an empty result. Is there some setting that I should apply first?

I am doing the very basic test:

cmk list consolesessions ==> empty result

start a console for the ssvm

cmk list consolesessions ==> empty result

I’d expect an extry on the second list command.

maybe in step2, create a console endpoint using a cmk ?

nvazquez · 2025-06-26T11:07:15Z

@bernardodemarco I did some basic tests with the admin user only, and accessing console through the CloudStack UI and so far no issues listing the console sessions, will continue testing with other user types.
I have only one minor request: instance_id is not autocompleting on cmk, can this autocompletion be added?

DaanHoogland · 2025-06-26T11:12:40Z

maybe in step2, create a console endpoint using a cmk ?

shouldn’t all sessions be listed?

@bernardodemarco I did some basic tests with the admin user only, and accessing console through the CloudStack UI and so far no issues listing the console sessions, will continue testing with other user types. I have only one minor request: instance_id is not autocompleting on cmk, can this autocompletion be added?

So in your case you did see a session in the output, @nvazquez ? (/me not)

bernardodemarco · 2025-06-26T11:16:05Z

@DaanHoogland, thanks for testing!

shouldn’t all sessions be listed?

It is important to note that there is the activeonly parameter, that defaults to true and the isrecursive parameter, that defaults to false.

So, if the console session is still active (i.e., marked as acquired and not marked as removed) and has been generated by an account in the same domain of the caller's account, then the API should have returned the console session.

If the console session is not active, then the activeonly parameter needs to be explicitly defined as false

DaanHoogland · 2025-06-26T11:17:46Z

@DaanHoogland, thanks for testing!

shouldn’t all sessions be listed?

It is important to note that there is the activeonly parameter, that defaults to true and the isrecursive parameter, that defaults to false.

So, if the console session are still active (i.e., marked as acquired and not marked as removed) and has been generated by an account in the same domain of the caller's account, then the API should have returned the console session.

If the console session is not active, then the activeonly parameter needs to be explicitly defined as false

thanks @bernardodemarco , I’ll double check.

bernardodemarco · 2025-06-26T11:17:50Z

@nvazquez, thanks for testing!

I have only one minor request: instance_id is not autocompleting on cmk, can this autocompletion be added?

Sure, I'll try to send a patch to the CloudMonkey repo adding support for autocompletion for the instance_id parameter

DaanHoogland · 2025-06-26T11:21:02Z

@nvazquez, thanks for testing!

I have only one minor request: instance_id is not autocompleting on cmk, can this autocompletion be added?

Sure, I'll try to send a patch to the CloudMonkey repo adding support for autocompletion for the instance_id parameter

ok, the problem is that a console created in the ui is not marked as active somehow. nothing to do with this PR I’ll investigate (only a little) and create a fresh issue if needed.

DaanHoogland · 2025-06-26T11:23:41Z

actually

(randy) 🐱 > list consolesessions activeonly=false
{
  "consolesession": [
    {
      "account": "admin",
      "accountid": "73ae5df4-519b-11f0-afe0-1e00ce000291",
      "consoleendpointcreatoraddress": "10.0.3.251",
      "created": "2025-06-26T11:21:22+0000",
      "domain": "ROOT",
      "domainid": "2aa3227b-519b-11f0-afe0-1e00ce000291",
      "domainpath": "/",
      "hostid": "60e3d383-56f6-4658-9ae6-dded45d2d56f",
      "hostname": "10.0.34.126",
      "id": "358f993a-7b47-4d15-bb90-a2aed92dc127",
      "instance_id": "d62c3fc2-deaa-4b82-b60d-c8353508bb82",
      "user": "admin",
      "userid": "73af0672-519b-11f0-afe0-1e00ce000291"
    },
    {
      "account": "admin",
      "accountid": "73ae5df4-519b-11f0-afe0-1e00ce000291",
      "consoleendpointcreatoraddress": "10.0.3.251",
      "created": "2025-06-26T10:14:52+0000",
      "domain": "ROOT",
      "domainid": "2aa3227b-519b-11f0-afe0-1e00ce000291",
      "domainpath": "/",
      "hostid": "60e3d383-56f6-4658-9ae6-dded45d2d56f",
      "hostname": "10.0.34.126",
      "id": "a2d2557c-5777-4920-93bc-633b5958f3bd",
      "instance_id": "d62c3fc2-deaa-4b82-b60d-c8353508bb82",
      "removed": "2025-06-26T10:28:25+0000",
      "user": "admin",
      "userid": "73af0672-519b-11f0-afe0-1e00ce000291"
    }
  ],
  "count": 2
}
(randy) 🐱 > list consolesessions 
(randy) 🐱 >

note how removed field is not present in the newer session but still it is not shown without activeonly=false.

can you explain this @bernardodemarco ?

DaanHoogland · 2025-06-26T11:26:14Z

btw, the acquired field is null in the DB for both session. Maybe this has to do with it.

bernardodemarco · 2025-06-26T11:28:55Z

can you explain this @bernardodemarco ?

ok, the problem is that a console created in the ui is not marked as active somehow.

@DaanHoogland, yes. I've already investigated this behavior when I was developing this patch. The console session is (or should be) marked as acquired when the user actually access the console through the browser.

note how removed field is not present in the newer session but still it is not shown without activeonly=false.

That happens because it is not marked as acquired. If the session has been acquired and has not been removed, then it is active. If the session has not been acquired, it is considered as inactive. If it has been removed, it is also considered as inactive.

DaanHoogland · 2025-06-26T12:24:01Z

#11096 (for reference @bernardodemarco ) I think we need not scope creep this PR with this.

api/src/main/java/org/apache/cloudstack/api/response/ConsoleSessionResponse.java

bernardodemarco · 2025-06-26T13:12:11Z

#11096 (for reference @bernardodemarco ) I think we need not scope creep this PR with this.

@DaanHoogland, okay, thanks!

bernardodemarco · 2025-06-26T16:08:24Z

@blueorangutan package

blueorangutan · 2025-06-26T17:04:28Z

@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

sureshanaparti · 2025-06-27T14:54:35Z

@blueorangutan package

blueorangutan · 2025-06-27T14:56:09Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-27T16:51:12Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13934

DaanHoogland · 2025-06-30T06:32:06Z

@blueorangutan test

blueorangutan · 2025-06-30T06:34:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

boring-cyborg bot added component:api component:management-server Python Warning... Python code Ahead! labels Jun 12, 2025

bernardodemarco added this to the 4.21.0 milestone Jun 12, 2025

bernardodemarco requested review from weizhouapache, nvazquez, sureshanaparti, winterhazel and JoaoJandre and removed request for weizhouapache June 12, 2025 15:37

bernardodemarco requested a review from DaanHoogland June 12, 2025 15:38

bernardodemarco force-pushed the listconsolesessions branch from 79f6d2d to ad7af57 Compare June 13, 2025 18:26

DaanHoogland reviewed Jun 13, 2025

View reviewed changes

refactor createConsoleSessionResponse method

1334d03

DaanHoogland mentioned this pull request Jun 26, 2025

Console session should be marked as acquired when accessed through the browser. #11096

Open

nvazquez reviewed Jun 26, 2025

View reviewed changes

api/src/main/java/org/apache/cloudstack/api/response/ConsoleSessionResponse.java Show resolved Hide resolved

add instancename to the APIs response

69c21a0

API to list console sessions #11016

Are you sure you want to change the base?

API to list console sessions #11016

Uh oh!

Conversation

bernardodemarco commented Jun 12, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Screenshots (if appropriate):

How Has This Been Tested?

Tests Setup

listConsoleSessions tests

Uh oh!

bernardodemarco commented Jun 12, 2025

Uh oh!

codecov bot commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bernardodemarco commented Jun 12, 2025

Uh oh!

blueorangutan commented Jun 12, 2025

Uh oh!

blueorangutan commented Jun 12, 2025

Uh oh!

bernardodemarco commented Jun 13, 2025

Uh oh!

blueorangutan commented Jun 13, 2025

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Jun 13, 2025

Uh oh!

blueorangutan commented Jun 13, 2025

Uh oh!

bernardodemarco commented Jun 13, 2025

Uh oh!

blueorangutan commented Jun 14, 2025

Uh oh!

DaanHoogland commented Jun 14, 2025

Uh oh!

blueorangutan commented Jun 14, 2025

Uh oh!

blueorangutan commented Jun 15, 2025

Uh oh!

DaanHoogland commented Jun 16, 2025

Uh oh!

blueorangutan commented Jun 16, 2025

Uh oh!

bernardodemarco commented Jun 24, 2025

Uh oh!

blueorangutan commented Jun 24, 2025

Uh oh!

blueorangutan commented Jun 24, 2025

Uh oh!

DaanHoogland commented Jun 25, 2025

Uh oh!

blueorangutan commented Jun 25, 2025

Uh oh!

blueorangutan commented Jun 26, 2025

Uh oh!

DaanHoogland commented Jun 26, 2025

Uh oh!

weizhouapache commented Jun 26, 2025

Uh oh!

nvazquez commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DaanHoogland commented Jun 26, 2025

Uh oh!

bernardodemarco commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DaanHoogland commented Jun 26, 2025

Uh oh!

bernardodemarco commented Jun 26, 2025

Uh oh!

`listConsoleSessions` tests

codecov bot commented Jun 12, 2025 •

edited

Loading

nvazquez commented Jun 26, 2025 •

edited

Loading

bernardodemarco commented Jun 26, 2025 •

edited

Loading