Skip to content

WebSocket Next: enable users to update SecurityIdentity before previous bearer access token expires #47675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions docs/src/main/asciidoc/websockets-next-reference.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -1073,6 +1073,51 @@ When you plan to use bearer access tokens during the opening WebSocket handshake
* Use a custom WebSocket ticket system which supplies a random token with the HTML page which hosts the JavaScript WebSockets client which must provide this token during the initial handshake request as a query parameter.
====

Before the bearer access token sent on the initial HTTP request expires, you can send a new bearer access token as part of a message and update current `SecurityIdentity` attached to the WebSocket server connection:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this new token can be obtained ? The current one is bound to the connection at the upgrade time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you obtained original token used at the upgrade time? Obtain it the same way. You pass it in a DTO that contains message and optionally metadata. You can pass it into the endpoint or hide that in the decoder (but I prefer the endpoint, hence the example). Tests speak for itself

Copy link
Member Author

@michalvavrik michalvavrik May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can front end client use refresh token stored in a session storage and obtain access token?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalvavrik

How did you obtained original token used at the upgrade time?

I'm assuming SPA used the authorization code flow

Obtain it the same way.

I don't think it works the same way for the token refresh.

So the proposal is for SPA to run a background task and check the current access token expiry time and refresh ?
What I don't know if OIDC scripts allow SPA access the access token expiry time - as some of these tokens are binary tokens.
The other thing I don't know, can one do HTTP calls while working with WS API ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming SPA used the authorization code flow

I don't know how to make WS work with authorization code flow on the Backend. I have looked into it before and I also raised it in the linked issue comments. Even if authorization code flow started before the WebSocket was opened, you still don't have a have to link session cookie with opened WebSocket. Anyway, I opened this PR for a bearer token authentication only.

Obtain it the same way.
I don't think it works the same way for the token refresh.

[1]
I need more information to understand why not, I can trust you, but then there is no point continue because this PR just won't do. No point wasting time on a review. My thinking was that:

  • For client credentials (like server to server websocket communication) you can use client id and secret to obtain token in the background.
  • For implicit grant flow (if there is still someone using it) SPA can send in the background and gets a token based on valid browser session
  • For code flow on the SPA side, you store session tokens in the browser session store, you keep refreshing token and you have direct access to the access token. I have personal experience with it, because I was maintaining Angular SPA in production for 2 years that communicated with multiple backends and we wrote interceptor to generated front-end clients that always set authorization header for tokens retrieved from the browser storage. we used https://github.com/manfredsteyer/angular-oauth2-oidc, it was hell to get it right, but then it worked

So the proposal is for SPA to run a background task and check the current access token expiry time and refresh ?

proposal is for SPA to get access token itself in a ways described right above, please see [1] of this comment.

What I don't know if OIDC scripts allow SPA access the access token expiry time - as some of these tokens are binary tokens.

I have no idea about that, but I remember that in our SPA we got tokens from Keycloak and we knew token expiration time (I want to say 30 minutes for access token and 18 hours for the refresh token, but honestly, I am not sure if I am not imaging it). It didn't change for us, maybe they can just check it out before? E.g. they don't exactly need to automate it, just add timed task.

The other thing I don't know, can one do HTTP calls while working with WS API ?

Typescript has multithreading, I think having background tasks like updating feeds on your SPA is a standard (e.g. on the newspage you have breaking news and that can by done by polling and HTTP requests). But I do speak about frameworks like Angular and React, no idea what you do with vanilla JS or node.js etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalvavrik Sure, SPA can do token refreshes

Can it really work though such that the script that works with the current WS connection can accept a refreshed token and forward this token to Quarkus ?

If we could confirm it then it would be great, IMHO there is no rush with the fix being merged

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll write an example by the end of the week.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we could confirm it then it would be great, IMHO there is no rush with the fix being merged

I have more plans for WS Next and while I didn't mention it (no reason to mention it), what I have added to this PR will allow me to easily fix quarkiverse/quarkus-langchain4j#1418 in the follow up. I don't want to keep this PR on hold, but let's wait until I have the example front end application. I'll inform you when I am done.

Copy link
Member

@sberyozkin sberyozkin May 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalvavrik Thanks, IMHO it will be worth it because this sample SPA can be used as a base for Step-Up authentication demos.

We can also add some simple example, similar to https://quarkus.io/guides/security-oidc-bearer-token-authentication#single-page-applications, without expecting it to be complete, to give users an idea how a token can be passed over to the existing WS connection

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also add some simple example, similar to https://quarkus.io/guides/security-oidc-bearer-token-authentication#single-page-applications, without expecting it to be complete, to give users an idea how a token can be passed over to the existing WS connection

ok, I'll look into it, but no promises on the https://quarkus.io/guides/security-oidc-bearer-token-authentication#single-page-applications part because it may require complex stuff, I don't know until I do it


[source, java]
----
package io.quarkus.websockets.next.test.security;

import io.quarkus.security.Authenticated;
import io.quarkus.security.identity.SecurityIdentity;
import io.quarkus.websockets.next.OnTextMessage;
import io.quarkus.websockets.next.WebSocket;
import io.quarkus.websockets.next.WebSocketSecurity;
import jakarta.inject.Inject;

@Authenticated
@WebSocket(path = "/end")
public class Endpoint {

record RequestDto(String accessToken, String message) {}

@Inject
SecurityIdentity securityIdentity;

@Inject
WebSocketSecurity webSocketSecurity;

@OnTextMessage
String echo(RequestDto request) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalvavrik

I think this method will have to match the security level of the HTTP upgrade.

For example, if HTTP Upgrade happened at @RolesAllowed("Admin"), then only verifying the validity of the token which would only match @Authenticated level, would raise a risk of decreasing the original security identity strength, for example, the new token which is about to replace an admin level identity may only have a user role.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that's easy to do.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalvavrik Sounds good, we do it indirectly in the code flow too when refreshing tokens, since it is done in scope of the request.
In case of this PR, the new token is coming via the front channel so the extra care is needed.
We might end up doing extra checks as well...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if I understand what you mean, but my plan is: there is a check that assures principal name is same (so they can't change alice for bob), but for roles and permissions, we just need to reapply standard security annotations (HTTP policies sounds like out of scope to me), which I think is easy. There is a code that prevents repeated security checks, I'll just disable the code when the identity refresh is enabled.

Copy link
Member

@sberyozkin sberyozkin May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalvavrik I was actually thinking about it too, how do you prevent an identity swap, so having the principal name check is good, but a few more checks may be necessary, such as a sub claim - sub is a unique number associated with a given user that did an original login, or even a more thorough identity comparison, but we can think about it later.

IMHO, whatever the security constraints were applied to HTTP upgrade should be reapplied at the token injection time because this is how we do in the code flow refresh. For example, when the ID and access token are refreshed, the refreshed ID token goes through the complete authentication and authorization cycle before the method can be called.

Copy link
Member

@sberyozkin sberyozkin May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the token is binary then we can update OidcIdentityProvider to record sub value from the introspection response as an identity attribute, something like that, so that you can easily access this sub

if (request.accessToken != null) {
webSocketSecurity.updateSecurityIdentity(request.accessToken); <1>
}
String principalName = securityIdentity.getPrincipal().getName(); <2>
return request.message + " " + principalName;
}

}
----
<1> Asynchronously update the `SecurityIdentity` attached to the WebSocket server connection.
<2> The `SecurityIdentity` instance injected into the `Endpoint` will represent the updated identity after Quarkus has finished the asynchronous identity update.
The update process should be imperceptible, because the `SecurityIdentity` before and after update should only differ in the token expiration time.

The xref:security-oidc-bearer-token-authentication.adoc[OIDC Bearer token authentication] mechanism has builtin support for the `SecurityIdentity` update.
If you use other authentication mechanisms, you must implement the `io.quarkus.security.identity.IdentityProvider` provider that supports the `io.quarkus.websockets.next.runtime.spi.telemetry.WebSocketIdentityUpdateRequest` authentication request.

IMPORTANT: Always use the `wss` protocol to enforce encrypted HTTP connection via TLS when sending credentials as part of the WebSocket message.

=== Inspect and/or reject HTTP upgrade

To inspect an HTTP upgrade, you must provide a CDI bean implementing the `io.quarkus.websockets.next.HttpUpgradeCheck` interface.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@
import io.quarkus.oidc.runtime.OidcTokenCredentialProducer;
import io.quarkus.oidc.runtime.OidcUtils;
import io.quarkus.oidc.runtime.TenantConfigBean;
import io.quarkus.oidc.runtime.WebSocketIdentityUpdateProvider;
import io.quarkus.oidc.runtime.providers.AzureAccessTokenCustomizer;
import io.quarkus.runtime.configuration.ConfigurationException;
import io.quarkus.security.Authenticated;
Expand Down Expand Up @@ -472,6 +473,14 @@ public void registerAuthenticationContextInterceptor(Capabilities capabilities,
.builder(Authenticated.class).buildWithTarget(c))));
}

@BuildStep
void supportIdentityUpdateForWebSocketConnections(Capabilities capabilities,
BuildProducer<AdditionalBeanBuildItem> additionalBeanProducer) {
if (capabilities.isPresent(Capability.WEBSOCKETS_NEXT)) {
additionalBeanProducer.produce(AdditionalBeanBuildItem.unremovableOf(WebSocketIdentityUpdateProvider.class));
}
}

private static boolean areEagerSecInterceptorsSupported(Capabilities capabilities,
VertxHttpBuildTimeConfig httpBuildTimeConfig) {
if (httpBuildTimeConfig.auth().proactive()) {
Expand Down
4 changes: 4 additions & 0 deletions extensions/oidc/runtime/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-oidc-common</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-websockets-next-spi</artifactId>
</dependency>
<dependency>
<groupId>io.smallrye</groupId>
<artifactId>smallrye-jwt</artifactId>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
package io.quarkus.oidc.runtime;

import static io.quarkus.runtime.configuration.DurationConverter.parseDuration;
import static io.quarkus.vertx.http.runtime.security.HttpSecurityUtils.getRoutingContextAttribute;

import java.time.Duration;
import java.util.HashMap;
import java.util.Map;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.function.Supplier;
Expand All @@ -18,22 +15,15 @@

import io.quarkus.arc.Arc;
import io.quarkus.arc.SyntheticCreationalContext;
import io.quarkus.oidc.AccessTokenCredential;
import io.quarkus.oidc.OIDCException;
import io.quarkus.oidc.Oidc;
import io.quarkus.oidc.OidcTenantConfig;
import io.quarkus.oidc.TenantIdentityProvider;
import io.quarkus.runtime.annotations.Recorder;
import io.quarkus.runtime.annotations.RuntimeInit;
import io.quarkus.runtime.annotations.StaticInit;
import io.quarkus.security.AuthenticationFailedException;
import io.quarkus.security.identity.AuthenticationRequestContext;
import io.quarkus.security.identity.SecurityIdentity;
import io.quarkus.security.identity.request.TokenAuthenticationRequest;
import io.quarkus.security.runtime.SecurityConfig;
import io.quarkus.security.spi.runtime.BlockingSecurityExecutor;
import io.quarkus.tls.TlsConfigurationRegistry;
import io.smallrye.mutiny.Uni;
import io.vertx.core.Vertx;
import io.vertx.ext.web.RoutingContext;

Expand Down Expand Up @@ -173,52 +163,4 @@ public void accept(RoutingContext routingContext) {
}
};
}

private static final class TenantSpecificOidcIdentityProvider extends OidcIdentityProvider
implements TenantIdentityProvider {

private final String tenantId;
private final BlockingSecurityExecutor blockingExecutor;

private TenantSpecificOidcIdentityProvider(String tenantId) {
super(Arc.container().instance(DefaultTenantConfigResolver.class).get(),
Arc.container().instance(BlockingSecurityExecutor.class).get());
this.blockingExecutor = Arc.container().instance(BlockingSecurityExecutor.class).get();
this.tenantId = tenantId;
}

@Override
public Uni<SecurityIdentity> authenticate(AccessTokenCredential token) {
return authenticate(new TokenAuthenticationRequest(token));
}

@Override
protected Uni<TenantConfigContext> resolveTenantConfigContext(TokenAuthenticationRequest request,
AuthenticationRequestContext context) {
return tenantResolver.resolveContext(tenantId).onItem().ifNull().failWith(new Supplier<Throwable>() {
@Override
public Throwable get() {
return new OIDCException("Failed to resolve tenant context");
}
});
}

@Override
protected Map<String, Object> getRequestData(TokenAuthenticationRequest request) {
RoutingContext context = getRoutingContextAttribute(request);
if (context != null) {
return context.data();
}
return new HashMap<>();
}

private Uni<SecurityIdentity> authenticate(TokenAuthenticationRequest request) {
return authenticate(request, new AuthenticationRequestContext() {
@Override
public Uni<SecurityIdentity> runBlocking(Supplier<SecurityIdentity> function) {
return blockingExecutor.executeBlocking(function);
}
});
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
package io.quarkus.oidc.runtime;

import static io.quarkus.vertx.http.runtime.security.HttpSecurityUtils.getRoutingContextAttribute;

import java.util.HashMap;
import java.util.Map;
import java.util.function.Supplier;

import io.quarkus.arc.Arc;
import io.quarkus.oidc.AccessTokenCredential;
import io.quarkus.oidc.OIDCException;
import io.quarkus.oidc.TenantIdentityProvider;
import io.quarkus.security.identity.AuthenticationRequestContext;
import io.quarkus.security.identity.SecurityIdentity;
import io.quarkus.security.identity.request.TokenAuthenticationRequest;
import io.quarkus.security.spi.runtime.BlockingSecurityExecutor;
import io.smallrye.mutiny.Uni;
import io.vertx.ext.web.RoutingContext;

final class TenantSpecificOidcIdentityProvider extends OidcIdentityProvider implements TenantIdentityProvider {

private final String tenantId;
private final BlockingSecurityExecutor blockingExecutor;

TenantSpecificOidcIdentityProvider(String tenantId, DefaultTenantConfigResolver resolver,
BlockingSecurityExecutor blockingExecutor) {
super(resolver, blockingExecutor);
this.blockingExecutor = blockingExecutor;
this.tenantId = tenantId;
}

TenantSpecificOidcIdentityProvider(String tenantId) {
this(tenantId, Arc.container().instance(DefaultTenantConfigResolver.class).get(),
Arc.container().instance(BlockingSecurityExecutor.class).get());
}

@Override
public Uni<SecurityIdentity> authenticate(AccessTokenCredential token) {
return authenticate(new TokenAuthenticationRequest(token));
}

@Override
protected Uni<TenantConfigContext> resolveTenantConfigContext(TokenAuthenticationRequest request,
AuthenticationRequestContext context) {
return tenantResolver.resolveContext(tenantId).onItem().ifNull().failWith(new Supplier<Throwable>() {
@Override
public Throwable get() {
return new OIDCException("Failed to resolve tenant context");
}
});
}

@Override
protected Map<String, Object> getRequestData(TokenAuthenticationRequest request) {
RoutingContext context = getRoutingContextAttribute(request);
if (context != null) {
return context.data();
}
return new HashMap<>();
}

private Uni<SecurityIdentity> authenticate(TokenAuthenticationRequest request) {
return authenticate(request, new AuthenticationRequestContext() {
@Override
public Uni<SecurityIdentity> runBlocking(Supplier<SecurityIdentity> function) {
return blockingExecutor.executeBlocking(function);
}
});
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
package io.quarkus.oidc.runtime;

import static io.quarkus.vertx.http.runtime.security.HttpSecurityUtils.getRoutingContextAttribute;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;

import io.quarkus.oidc.AccessTokenCredential;
import io.quarkus.oidc.OidcTenantConfig;
import io.quarkus.security.AuthenticationFailedException;
import io.quarkus.security.identity.AuthenticationRequestContext;
import io.quarkus.security.identity.IdentityProvider;
import io.quarkus.security.identity.SecurityIdentity;
import io.quarkus.security.spi.runtime.BlockingSecurityExecutor;
import io.quarkus.websockets.next.runtime.spi.telemetry.WebSocketIdentityUpdateRequest;
import io.smallrye.mutiny.Uni;
import io.vertx.ext.web.RoutingContext;

@ApplicationScoped
public class WebSocketIdentityUpdateProvider implements IdentityProvider<WebSocketIdentityUpdateRequest> {

@Inject
DefaultTenantConfigResolver resolver;

@Inject
BlockingSecurityExecutor blockingExecutor;

WebSocketIdentityUpdateProvider() {
}

@Override
public Class<WebSocketIdentityUpdateRequest> getRequestType() {
return WebSocketIdentityUpdateRequest.class;
}

@Override
public Uni<SecurityIdentity> authenticate(WebSocketIdentityUpdateRequest request,
AuthenticationRequestContext authenticationRequestContext) {
return authenticate(request.getCredential().getToken(), getRoutingContextAttribute(request));
}

private Uni<SecurityIdentity> authenticate(String accessToken, RoutingContext routingContext) {
final OidcTenantConfig tenantConfig = routingContext.get(OidcTenantConfig.class.getName());
if (tenantConfig == null) {
return Uni.createFrom().failure(new AuthenticationFailedException(
"Cannot update SecurityIdentity because OIDC tenant wasn't resolved for current WebSocket connection"));
}
final var tenantId = tenantConfig.tenantId().get();
final var identityProvider = new TenantSpecificOidcIdentityProvider(tenantId, resolver, blockingExecutor);
final var credential = new AccessTokenCredential(accessToken);
return identityProvider.authenticate(credential);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.function.Consumer;
Expand All @@ -21,6 +22,7 @@
import java.util.regex.Pattern;
import java.util.stream.Collectors;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.context.SessionScoped;
import jakarta.enterprise.inject.Instance;
import jakarta.enterprise.invoke.Invoker;
Expand All @@ -40,6 +42,7 @@
import org.jboss.jandex.PrimitiveType;
import org.jboss.jandex.Type;
import org.jboss.jandex.Type.Kind;
import org.jboss.jandex.WildcardType;
import org.objectweb.asm.Opcodes;

import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
Expand Down Expand Up @@ -95,6 +98,8 @@
import io.quarkus.gizmo.ResultHandle;
import io.quarkus.gizmo.TryBlock;
import io.quarkus.runtime.metrics.MetricsFactory;
import io.quarkus.security.identity.IdentityProvider;
import io.quarkus.security.identity.IdentityProviderManager;
import io.quarkus.security.spi.ClassSecurityAnnotationBuildItem;
import io.quarkus.security.spi.ClassSecurityCheckStorageBuildItem;
import io.quarkus.security.spi.PermissionsAllowedMetaAnnotationBuildItem;
Expand All @@ -113,6 +118,7 @@
import io.quarkus.websockets.next.WebSocketClientException;
import io.quarkus.websockets.next.WebSocketConnection;
import io.quarkus.websockets.next.WebSocketException;
import io.quarkus.websockets.next.WebSocketSecurity;
import io.quarkus.websockets.next.WebSocketServerException;
import io.quarkus.websockets.next.deployment.Callback.MessageType;
import io.quarkus.websockets.next.deployment.Callback.Target;
Expand Down Expand Up @@ -161,6 +167,7 @@ public class WebSocketProcessor {
static final String CLIENT_ENDPOINT_SUFFIX = "_WebSocketClientEndpoint";
static final String NESTED_SEPARATOR = "$_";
static final DotName HTTP_UPGRADE_CHECK_NAME = DotName.createSimple(HttpUpgradeCheck.class);
private static final DotName WEBSOCKET_SECURITY_NAME = DotName.createSimple(WebSocketSecurity.class);

// Parameter names consist of alphanumeric characters and underscore
private static final Pattern PATH_PARAM_PATTERN = Pattern.compile("\\{[a-zA-Z0-9_]+\\}");
Expand Down Expand Up @@ -791,6 +798,32 @@ void createTelemetryProvider(BuildProducer<SyntheticBeanBuildItem> syntheticBean
}
}

@BuildStep
@Record(STATIC_INIT)
void supportSecurityIdentityUpdate(BeanDiscoveryFinishedBuildItem beanDiscoveryFinishedBuildItem,
WebSocketServerRecorder recorder, Capabilities capabilities,
BuildProducer<SyntheticBeanBuildItem> syntheticBeanProducer) {
if (capabilities.isMissing(Capability.SECURITY)) {
return;
}
boolean isWsSecurityInjected = beanDiscoveryFinishedBuildItem.getInjectionPoints().stream()
.map(InjectionPointInfo::getType)
.filter(Objects::nonNull)
.map(Type::name)
.anyMatch(WEBSOCKET_SECURITY_NAME::equals);
if (isWsSecurityInjected) {
syntheticBeanProducer.produce(SyntheticBeanBuildItem
.configure(WEBSOCKET_SECURITY_NAME)
.addInjectionPoint(ClassType.create(IdentityProviderManager.class))
// Instance<IdentityProvider<?>>
.addInjectionPoint(ParameterizedType.create(Instance.class,
ParameterizedType.create(DotName.createSimple(IdentityProvider.class), WildcardType.UNBOUNDED)))
.createWith(recorder.createWebSocketSecurity())
.scope(ApplicationScoped.class)
.done());
}
}

private static boolean isTracesSupportEnabled(Capabilities capabilities) {
return capabilities.isPresent(Capability.OPENTELEMETRY_TRACER);
}
Expand Down
Loading
Loading