User from domain can't deploy VM in pod dedicated to that domain

[edikevich]

ISSUE TYPE

• Bug Report

CLOUDSTACK VERSION

4.16.0

CONFIGURATION

XCP-NG 8.2
Advanced Zone

OS / ENVIRONMENT

Oracle Linux 8.4 - Management server, MYSQL 8.0.26
XCP-NG 8.2 - hypervisor

SUMMARY

User from domain1 can't deploy VM in dedicated to domain1 pod.

STEPS TO REPRODUCE

Create domain. Create user in this domain. Dedicate pod to this domain. Create Compute offering with ImplicitDedicationPlanner with strict policy and choose domain1.

EXPECTED RESULTS

All users from this domain can deploy VMs only in this Pod

ACTUAL RESULTS

User from this domain can't deploy VM in dedicated pod. 

[DaanHoogland]

Is this new? Did you test in 4.15 @edikevich?
@sureshanaparti is this one good for 4.16.1?

[edikevich]

Hi. Sorry for delay. I didn't test this feature in 4.15.2 :( I think problem is near Affinity group which is created after dedicate host/cluster/pod. I tried create VM from root Admin then grand this VM to user from domain but not user or root admin can't start VM in dedicated host/cluster/pod :(

[DaanHoogland]

Hey @edikevich do you mean to say that the issue is

Create domain.
Create user in this domain.
Dedicate pod to this domain.
Create Compute offering with ImplicitDedicationPlanner with strict policy and choose domain1.
Add an affinity group ???

now user can't a VM in this domain?

[edikevich]

Hi. Not exactly. When pod is dedicated affinity group is created automatically by CS. Other steps are correct.

[DaanHoogland]

@edikevich can you add some error logs and preferably a stack trace. I am trying to reproduce this, but haven't succeeded yet. Mostly due to environment build.

[edikevich]

Hi. I don't have any errors in log. I see only that no suitable hosts found.

[edikevich]

Sorry. Have Error:

2022-01-12 12:42:14,559 INFO [o.a.c.a.c.u.v.DeployVMCmd] (API-Job-Executor-17:ctx-10e86d4b job-3810 ctx-ed36cebb) (logid:7ebaa05f) com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {id: "334", name: "i-34-334-VM", uuid: "88d9976d-0b1a-4c78-a201-b6d153cf0e58", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1
2022-01-12 12:42:14,559 INFO [o.a.c.a.c.u.v.DeployVMCmd] (API-Job-Executor-17:ctx-10e86d4b job-3810 ctx-ed36cebb) (logid:7ebaa05f) Unable to create a deployment for VM instance {id: "334", name: "i-34-334-VM", uuid: "88d9976d-0b1a-4c78-a201-b6d153cf0e58", type="User"}
com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {id: "334", name: "i-34-334-VM", uuid: "88d9976d-0b1a-4c78-a201-b6d153cf0e58", type="User"}Scope=interface com.cloud.dc.DataCenter; id=1
at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.reserveVirtualMachine(VMEntityManagerImpl.java:225)
at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.reserve(VirtualMachineEntityImpl.java:202)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5207)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4693)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4682)
at jdk.internal.reflect.GeneratedMethodAccessor1658.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy127.startVirtualMachine(Unknown Source)
at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:696)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-01-12 12:42:14,561 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-17:ctx-10e86d4b job-3810) (logid:7ebaa05f) Complete async job-3810, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"533","errortext":"Unable to create a deployment for VM instance {id: "334", name: "i-34-334-VM", uuid: "88d9976d-0b1a-4c78-a201-b6d153cf0e58", type="User"}"}

[DaanHoogland]

This message says that there is no deployment plan suitable, but above this message there should be more info on how this was tried, @edikevich . If you grep for 'job-3810' in the log you should get all the info related to the failure. Can you do that and see if there is more relevant info?
What I am looking for is:
Is the lack of host the capacity problem or something else? (most likely storage then)
Did it even try the dedicated domain? and if so, why did it refuse them?