bucket creation fail

[PPisz]

Hello,

I'm testing setting up buckets via rgw ceph, and I have the following error. The user set up correctly, are there any special requirements for configuring the zone in rgw? Can something be additionally defined on the CS side?

2025-02-21 14:48:41,320 DEBUG [o.a.c.s.d.d.CephObjectStoreDriverImpl] (qtp2038105753-809:[ctx-56b9b564, ctx-bdfed81a]) (logid:5b06dfdd) Attempting to create Ceph RGW user for account PrjAcct-WB-MGMT-1 with UUID ad3777df-cb51-4353-be50-4e1cf2e14b12 2025-02-21 14:48:41,432 INFO [o.a.c.s.d.d.CephObjectStoreDriverImpl] (qtp2038105753-809:[ctx-56b9b564, ctx-bdfed81a]) (logid:5b06dfdd) User already exists in Ceph RGW: ad3777df-cb51-4353-be50-4e1cf2e14b12 2025-02-21 14:48:41,489 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-9:[]) (logid:) Seq 1-2013953458364743834: Routing from 37951217891278 2025-02-21 14:48:41,494 DEBUG [c.c.a.ApiServer] (qtp2038105753-809:[ctx-56b9b564, ctx-bdfed81a]) (logid:5b06dfdd) Retrieved cmdEventType from job info: BUCKET.CREATE 2025-02-21 14:48:41,501 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-31:[ctx-979eab10, job-84118]) (logid:b63bfa54) Add job-84118 into job monitoring 2025-02-21 14:48:41,507 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-31:[ctx-979eab10, job-84118]) (logid:2a132844) Executing AsyncJobVO: {id:84118, userId: 33, accountId: 14, instanceType: Bucket, instanceId: 7, cmd: org.apache.cloudstack.api.command.user.bucket.CreateBucketCmd, cmdInfo: {"sessionkey":"7MvY717MrZS9E0kIhtG9TrQYYro","httpmethod":"GET","ctxAccountId":"14","uuid":"88a2dcf9-7646-41f5-b12d-09430363c32f","cmdEventType":"BUCKET.CREATE","objectstorageid":"baf8bd23-82b5-4fde-ad11-a27bb17a4920","response":"json","ctxUserId":"33","name":"sdcsddcsc","ctxStartEventId":"428848","id":"7","ctxDetails":"{\"interface com.cloud.projects.Project\":\"198b6aa5-b909-4c5b-8899-7a7f1616b61d\",\"interface org.apache.cloudstack.storage.object.ObjectStore\":\"baf8bd23-82b5-4fde-ad11-a27bb17a4920\",\"interface org.apache.cloudstack.storage.object.Bucket\":\"88a2dcf9-7646-41f5-b12d-09430363c32f\"}","projectid":"198b6aa5-b909-4c5b-8899-7a7f1616b61d","policy":"Private"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 222320990092816, completeMsid: null, lastUpdated: null, lastPolled: null, created: null, removed: null} 2025-02-21 14:48:41,510 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (qtp2038105753-809:[ctx-56b9b564, ctx-bdfed81a]) (logid:5b06dfdd) submit async job-84118, details: AsyncJobVO: {id:84118, userId: 33, accountId: 14, instanceType: Bucket, instanceId: 7, cmd: org.apache.cloudstack.api.command.user.bucket.CreateBucketCmd, cmdInfo: {"sessionkey":"7MvY717MrZS9E0kIhtG9TrQYYro","httpmethod":"GET","ctxAccountId":"14","uuid":"88a2dcf9-7646-41f5-b12d-09430363c32f","cmdEventType":"BUCKET.CREATE","objectstorageid":"baf8bd23-82b5-4fde-ad11-a27bb17a4920","response":"json","ctxUserId":"33","name":"sdcsddcsc","ctxStartEventId":"428848","id":"7","ctxDetails":"{\"interface com.cloud.projects.Project\":\"198b6aa5-b909-4c5b-8899-7a7f1616b61d\",\"interface org.apache.cloudstack.storage.object.ObjectStore\":\"baf8bd23-82b5-4fde-ad11-a27bb17a4920\",\"interface org.apache.cloudstack.storage.object.Bucket\":\"88a2dcf9-7646-41f5-b12d-09430363c32f\"}","projectid":"198b6aa5-b909-4c5b-8899-7a7f1616b61d","policy":"Private"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 222320990092816, completeMsid: null, lastUpdated: null, lastPolled: null, created: null, removed: null} 2025-02-21 14:48:41,512 INFO [c.c.a.ApiServlet] (qtp2038105753-809:[ctx-56b9b564, ctx-bdfed81a]) (logid:5b06dfdd) (userId=33 accountId=14 sessionId=node0177oojxr7gbdi1717zlvgmeyh632) 10.10.101.11 -- GET name=sdcsddcsc&objectstorageid=baf8bd23-82b5-4fde-ad11-a27bb17a4920&policy=Private&command=createBucket&response=json&sessionkey=7MvY717MrZS9E0kIhtG9TrQYYro&projectid=198b6aa5-b909-4c5b-8899-7a7f1616b61d 200 {"createbucketresponse":{"id":"88a2dcf9-7646-41f5-b12d-09430363c32f","jobid":"2a132844-2af5-48c6-befa-eede21d310fa"}} 2025-02-21 14:48:41,512 DEBUG [c.c.a.ApiServlet] (qtp2038105753-809:[ctx-56b9b564, ctx-bdfed81a]) (logid:5b06dfdd) ===END=== 10.10.101.11 -- GET name=sdcsddcsc&objectstorageid=baf8bd23-82b5-4fde-ad11-a27bb17a4920&policy=Private&command=createBucket&response=json&sessionkey=7MvY717MrZS9E0kIhtG9TrQYYro&projectid=198b6aa5-b909-4c5b-8899-7a7f1616b61d 2025-02-21 14:48:41,568 DEBUG [c.c.a.ApiServlet] (qtp2038105753-1075:[ctx-7ca01584]) (logid:55c6f0fd) ===START=== 10.10.101.11 -- GET jobId=2a132844-2af5-48c6-befa-eede21d310fa&command=queryAsyncJobResult&response=json&sessionkey=7MvY717MrZS9E0kIhtG9TrQYYro&projectid=198b6aa5-b909-4c5b-8899-7a7f1616b61d 2025-02-21 14:48:41,569 DEBUG [c.c.a.ApiServlet] (qtp2038105753-1075:[ctx-7ca01584]) (logid:55c6f0fd) Two factor authentication is already verified for the user 33, so skipping 2025-02-21 14:48:41,577 DEBUG [c.c.a.ApiServer] (qtp2038105753-1075:[ctx-7ca01584, ctx-f8fafc8e]) (logid:55c6f0fd) CIDRs from which account 'Account [{"accountName":"ppisz","id":14,"uuid":"016415f5-344c-4f77-8ae3-f6097845237f"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-21 14:48:41,582 INFO [o.a.c.a.ProjectRoleBasedApiAccessChecker] (qtp2038105753-1075:[ctx-7ca01584, ctx-f8fafc8e]) (logid:55c6f0fd) Account [ppisz] is Root Admin or Domain Admin, all APIs are allowed. 2025-02-21 14:48:41,582 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp2038105753-1075:[ctx-7ca01584, ctx-f8fafc8e]) (logid:55c6f0fd) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-21 14:48:41,582 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp2038105753-1075:[ctx-7ca01584, ctx-f8fafc8e]) (logid:55c6f0fd) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-21 14:48:41,595 DEBUG [o.a.c.s.o.BucketApiServiceImpl] (API-Job-Executor-31:[ctx-979eab10, job-84118, ctx-e1748a40]) (logid:2a132844) Failed to create bucket with name: sdcsddcsc com.cloud.utils.exception.CloudRuntimeException: The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint; Request ID: tx00000f91052dcd922dc7e-0067b88439-61475656-default; S3 Extended Request ID: 61475656-default-default; Proxy: null) at org.apache.cloudstack.storage.datastore.driver.CephObjectStoreDriverImpl.createBucket(CephObjectStoreDriverImpl.java:120) at org.apache.cloudstack.storage.object.store.ObjectStoreImpl.createBucket(ObjectStoreImpl.java:107) at org.apache.cloudstack.storage.object.BucketApiServiceImpl.createBucket(BucketApiServiceImpl.java:147) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:105) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) at jdk.proxy3/jdk.proxy3.$Proxy434.createBucket(Unknown Source) at org.apache.cloudstack.api.command.user.bucket.CreateBucketCmd.execute(CreateBucketCmd.java:188) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173) at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:110) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:652) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:600) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint; Request ID: tx00000f91052dcd922dc7e-0067b88439-61475656-default; S3 Extended Request ID: 61475656-default-default; Proxy: null), S3 Extended Request ID: 61475656-default-default at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403) at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:1131) at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:1059) at org.apache.cloudstack.storage.datastore.driver.CephObjectStoreDriverImpl.createBucket(CephObjectStoreDriverImpl.java:109) ... 32 more

[abh1sar]

Hi @PPisz
This looks to be the same issue as discussed here : #10044 (comment)
Please see if the workaround given works for you.
The PR has been merged to 4.20.1 : #10772