Skip to content

Bmoric/extract notification api #18812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Nov 2, 2022
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
4ebf4ec
Tmp
benmoriceau Oct 21, 2022
6d8e907
Merge branch 'master' of github.com:airbytehq/airbyte into bmoric/con…
benmoriceau Oct 24, 2022
d930c98
Extract the Attempt API from the V1 API
benmoriceau Oct 24, 2022
2dba567
Add comments
benmoriceau Oct 24, 2022
7c36975
Move Connection API out of configuration API
benmoriceau Oct 24, 2022
0d87509
format
benmoriceau Oct 24, 2022
cf61847
Merge branch 'bmoric/convert-server-to-micronaut' of github.com:airby…
benmoriceau Oct 24, 2022
04e9bb2
format
benmoriceau Oct 24, 2022
c74db48
Rename to Controller
benmoriceau Oct 25, 2022
433dfe5
Merge branch 'bmoric/convert-server-to-micronaut' of github.com:airby…
benmoriceau Oct 25, 2022
415e0c7
Rename to Controller
benmoriceau Oct 25, 2022
7211da6
Add values to the factory
benmoriceau Oct 25, 2022
f503b4c
Change the constructor to use hadler instead of objects needed by the…
benmoriceau Oct 25, 2022
70ce157
Merge branch 'master' of github.com:airbytehq/airbyte into bmoric/ext…
benmoriceau Oct 25, 2022
688f812
Merge branch 'master' of github.com:airbytehq/airbyte into bmoric/ext…
benmoriceau Oct 25, 2022
2103e7d
Update with new tags.
benmoriceau Oct 25, 2022
a5c8522
tmp
benmoriceau Oct 25, 2022
c5aee8f
Fix PMD errors
benmoriceau Oct 25, 2022
50fc3ae
Extract DB migrator
benmoriceau Oct 25, 2022
6b11154
Merge branch 'bmoric/extract-connection-api' of github.com:airbytehq/…
benmoriceau Oct 25, 2022
4e8051e
Add something that I forgot
benmoriceau Oct 26, 2022
99d9d4b
Merge branch 'master' of github.com:airbytehq/airbyte into bmoric/ext…
benmoriceau Oct 26, 2022
aca18c9
Merge branch 'master' of github.com:airbytehq/airbyte into bmoric/ext…
benmoriceau Oct 26, 2022
7254ebb
extract destination definition api
benmoriceau Oct 26, 2022
4d748af
restore destination factory initialization
benmoriceau Oct 26, 2022
e6a56a2
extract destination definition specification api
benmoriceau Oct 26, 2022
709cdc4
format
benmoriceau Oct 26, 2022
aa06c20
Merge branch 'bmoric/extract-db-migration-api' of github.com:airbyteh…
benmoriceau Oct 26, 2022
82360dc
format
benmoriceau Oct 26, 2022
82ccd82
Merge branch 'bmoric/extract-destination-definition-api' of github.co…
benmoriceau Oct 26, 2022
6ecdb17
format
benmoriceau Oct 26, 2022
187c9ea
extract health check api
benmoriceau Oct 26, 2022
f6a5e9b
extract jobs api
benmoriceau Oct 26, 2022
8023bc9
fix test
benmoriceau Oct 27, 2022
bd206a9
Merge branch 'master' of github.com:airbytehq/airbyte into bmoric/ext…
benmoriceau Oct 27, 2022
293c187
format
benmoriceau Oct 27, 2022
89308f5
Merge branch 'master' into bmoric/extract-healt-api
benmoriceau Oct 28, 2022
60fa570
Merge branch 'bmoric/extract-healt-api' of github.com:airbytehq/airby…
benmoriceau Oct 28, 2022
7accc25
Extract logs api
benmoriceau Oct 28, 2022
5151909
Add missing declaration
benmoriceau Oct 28, 2022
0eb77e1
Merge branch 'bmoric/extract-jobs-api' of github.com:airbytehq/airbyt…
benmoriceau Oct 28, 2022
ffb770b
Fix build
benmoriceau Oct 28, 2022
c3cd08e
Merge branch 'master' into bmoric/extract-healt-api
benmoriceau Oct 28, 2022
c65a79b
Merge branch 'bmoric/extract-healt-api' into bmoric/extract-jobs-api
benmoriceau Oct 28, 2022
2142e26
Tmp
benmoriceau Oct 28, 2022
3ed2fee
format and PR comments
benmoriceau Oct 28, 2022
cef5c58
Merge branch 'bmoric/extract-jobs-api' of github.com:airbytehq/airbyt…
benmoriceau Oct 28, 2022
bfaf6b5
Merge branch 'master' of github.com:airbytehq/airbyte into bmoric/ext…
benmoriceau Nov 1, 2022
9b00d89
Extract notification API
benmoriceau Nov 1, 2022
8d07282
Merge branch 'bmoric/extract-jobs-api' of github.com:airbytehq/airbyt…
benmoriceau Nov 1, 2022
4b2361d
Merge branch 'bmoric/extract-logs-api' of github.com:airbytehq/airbyt…
benmoriceau Nov 1, 2022
fa4761d
Fix PMD
benmoriceau Nov 2, 2022
c41a1f9
Merge branch 'bmoric/extract-logs-api' of github.com:airbytehq/airbyt…
benmoriceau Nov 2, 2022
ca56b4a
Merge branch 'master' of github.com:airbytehq/airbyte into bmoric/ext…
benmoriceau Nov 2, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 42 additions & 1 deletion airbyte-server/src/main/java/io/airbyte/server/ServerApp.java
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,14 @@
import io.airbyte.server.handlers.DbMigrationHandler;
import io.airbyte.server.handlers.DestinationDefinitionsHandler;
import io.airbyte.server.handlers.DestinationHandler;
import io.airbyte.server.handlers.HealthCheckHandler;
import io.airbyte.server.handlers.JobHistoryHandler;
import io.airbyte.server.handlers.LogsHandler;
import io.airbyte.server.handlers.OperationsHandler;
import io.airbyte.server.handlers.SchedulerHandler;
import io.airbyte.server.handlers.SourceDefinitionsHandler;
import io.airbyte.server.handlers.SourceHandler;
import io.airbyte.server.handlers.WorkspacesHandler;
import io.airbyte.server.scheduler.DefaultSynchronousSchedulerClient;
import io.airbyte.server.scheduler.EventRunner;
import io.airbyte.server.scheduler.TemporalEventRunner;
Expand Down Expand Up @@ -299,6 +305,37 @@ public static ServerRunnable getServer(final ServerFactory apiFactory,
final DestinationDefinitionsHandler destinationDefinitionsHandler = new DestinationDefinitionsHandler(configRepository, syncSchedulerClient,
destinationHandler);

final HealthCheckHandler healthCheckHandler = new HealthCheckHandler(configRepository);

final SourceHandler sourceHandler = new SourceHandler(
configRepository,
secretsRepositoryReader,
secretsRepositoryWriter,
schemaValidator,
connectionsHandler);

final SourceDefinitionsHandler sourceDefinitionsHandler = new SourceDefinitionsHandler(configRepository, syncSchedulerClient, sourceHandler);

final JobHistoryHandler jobHistoryHandler = new JobHistoryHandler(
jobPersistence,
configs.getWorkerEnvironment(),
configs.getLogConfigs(),
connectionsHandler,
sourceHandler,
sourceDefinitionsHandler,
destinationHandler,
destinationDefinitionsHandler,
configs.getAirbyteVersion());

final LogsHandler logsHandler = new LogsHandler(configs);

final WorkspacesHandler workspacesHandler = new WorkspacesHandler(
configRepository,
secretsRepositoryWriter,
connectionsHandler,
destinationHandler,
sourceHandler);

LOGGER.info("Starting server...");

return apiFactory.create(
Expand All @@ -323,8 +360,12 @@ public static ServerRunnable getServer(final ServerFactory apiFactory,
dbMigrationHandler,
destinationDefinitionsHandler,
destinationHandler,
healthCheckHandler,
jobHistoryHandler,
logsHandler,
operationsHandler,
schedulerHandler);
schedulerHandler,
workspacesHandler);
}

@VisibleForTesting
Expand Down
68 changes: 60 additions & 8 deletions airbyte-server/src/main/java/io/airbyte/server/ServerFactory.java
Original file line number Diff line number Diff line change
Expand Up @@ -21,25 +21,41 @@
import io.airbyte.server.apis.DestinationApiController;
import io.airbyte.server.apis.DestinationDefinitionApiController;
import io.airbyte.server.apis.DestinationDefinitionSpecificationApiController;
import io.airbyte.server.apis.HealthApiController;
import io.airbyte.server.apis.JobsApiController;
import io.airbyte.server.apis.LogsApiController;
import io.airbyte.server.apis.NotificationsApiController;
import io.airbyte.server.apis.binders.AttemptApiBinder;
import io.airbyte.server.apis.binders.ConnectionApiBinder;
import io.airbyte.server.apis.binders.DbMigrationBinder;
import io.airbyte.server.apis.binders.DestinationApiBinder;
import io.airbyte.server.apis.binders.DestinationDefinitionApiBinder;
import io.airbyte.server.apis.binders.DestinationDefinitionSpecificationApiBinder;
import io.airbyte.server.apis.binders.HealthApiBinder;
import io.airbyte.server.apis.binders.JobsApiBinder;
import io.airbyte.server.apis.binders.LogsApiBinder;
import io.airbyte.server.apis.binders.NotificationApiBinder;
import io.airbyte.server.apis.factories.AttemptApiFactory;
import io.airbyte.server.apis.factories.ConnectionApiFactory;
import io.airbyte.server.apis.factories.DbMigrationApiFactory;
import io.airbyte.server.apis.factories.DestinationApiFactory;
import io.airbyte.server.apis.factories.DestinationDefinitionApiFactory;
import io.airbyte.server.apis.factories.DestinationDefinitionSpecificationApiFactory;
import io.airbyte.server.apis.factories.HealthApiFactory;
import io.airbyte.server.apis.factories.JobsApiFactory;
import io.airbyte.server.apis.factories.LogsApiFactory;
import io.airbyte.server.apis.factories.NotificationsApiFactory;
import io.airbyte.server.handlers.AttemptHandler;
import io.airbyte.server.handlers.ConnectionsHandler;
import io.airbyte.server.handlers.DbMigrationHandler;
import io.airbyte.server.handlers.DestinationDefinitionsHandler;
import io.airbyte.server.handlers.DestinationHandler;
import io.airbyte.server.handlers.HealthCheckHandler;
import io.airbyte.server.handlers.JobHistoryHandler;
import io.airbyte.server.handlers.LogsHandler;
import io.airbyte.server.handlers.OperationsHandler;
import io.airbyte.server.handlers.SchedulerHandler;
import io.airbyte.server.handlers.WorkspacesHandler;
import io.airbyte.server.scheduler.EventRunner;
import io.airbyte.server.scheduler.SynchronousSchedulerClient;
import java.net.http.HttpClient;
Expand Down Expand Up @@ -72,8 +88,12 @@ ServerRunnable create(final SynchronousSchedulerClient synchronousSchedulerClien
final DbMigrationHandler dbMigrationHandler,
final DestinationDefinitionsHandler destinationDefinitionsHandler,
final DestinationHandler destinationApiHandler,
final HealthCheckHandler healthCheckHandler,
final JobHistoryHandler jobHistoryHandler,
final LogsHandler logsHandler,
final OperationsHandler operationsHandler,
final SchedulerHandler schedulerHandler);
final SchedulerHandler schedulerHandler,
final WorkspacesHandler workspacesHandler);

class Api implements ServerFactory {

Expand All @@ -99,8 +119,12 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul
final DbMigrationHandler dbMigrationHandler,
final DestinationDefinitionsHandler destinationDefinitionsHandler,
final DestinationHandler destinationApiHandler,
final HealthCheckHandler healthCheckHandler,
final JobHistoryHandler jobHistoryHandler,
final LogsHandler logsHandler,
final OperationsHandler operationsHandler,
final SchedulerHandler schedulerHandler) {
final SchedulerHandler schedulerHandler,
final WorkspacesHandler workspacesHandler) {
final Map<String, String> mdc = MDC.getCopyOfContextMap();

// set static values for factory
Expand Down Expand Up @@ -140,13 +164,41 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul

DestinationDefinitionSpecificationApiFactory.setValues(schedulerHandler);

HealthApiFactory.setValues(healthCheckHandler);

JobsApiFactory.setValues(jobHistoryHandler, schedulerHandler);

LogsApiFactory.setValues(logsHandler);

NotificationsApiFactory.setValues(workspacesHandler);

// server configurations
final Set<Class<?>> componentClasses = Set.of(ConfigurationApi.class, AttemptApiController.class, ConnectionApiController.class,
DbMigrationApiController.class, DestinationApiController.class, DestinationDefinitionApiController.class,
DestinationDefinitionSpecificationApiController.class);
final Set<Object> components = Set.of(new CorsFilter(), new ConfigurationApiBinder(), new AttemptApiBinder(), new ConnectionApiBinder(),
new DbMigrationBinder(), new DestinationApiBinder(), new DestinationDefinitionApiBinder(),
new DestinationDefinitionSpecificationApiBinder());
final Set<Class<?>> componentClasses = Set.of(
ConfigurationApi.class,
AttemptApiController.class,
ConnectionApiController.class,
DbMigrationApiController.class,
DestinationApiController.class,
DestinationDefinitionApiController.class,
DestinationDefinitionSpecificationApiController.class,
HealthApiController.class,
JobsApiController.class,
LogsApiController.class,
NotificationsApiController.class);

final Set<Object> components = Set.of(
new CorsFilter(),
new ConfigurationApiBinder(),
new AttemptApiBinder(),
new ConnectionApiBinder(),
new DbMigrationBinder(),
new DestinationApiBinder(),
new DestinationDefinitionApiBinder(),
new DestinationDefinitionSpecificationApiBinder(),
new HealthApiBinder(),
new JobsApiBinder(),
new LogsApiBinder(),
new NotificationApiBinder());

// construct server
return new ServerApp(airbyteVersion, componentClasses, components);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,7 @@
import io.airbyte.server.handlers.ConnectionsHandler;
import io.airbyte.server.handlers.DestinationDefinitionsHandler;
import io.airbyte.server.handlers.DestinationHandler;
import io.airbyte.server.handlers.HealthCheckHandler;
import io.airbyte.server.handlers.JobHistoryHandler;
import io.airbyte.server.handlers.LogsHandler;
import io.airbyte.server.handlers.OAuthHandler;
import io.airbyte.server.handlers.OpenApiConfigHandler;
import io.airbyte.server.handlers.OperationsHandler;
Expand Down Expand Up @@ -157,8 +155,6 @@ public class ConfigurationApi implements io.airbyte.api.generated.V1Api {
private final JobHistoryHandler jobHistoryHandler;
private final WebBackendConnectionsHandler webBackendConnectionsHandler;
private final WebBackendGeographiesHandler webBackendGeographiesHandler;
private final HealthCheckHandler healthCheckHandler;
private final LogsHandler logsHandler;
private final OpenApiConfigHandler openApiConfigHandler;
private final OAuthHandler oAuthHandler;
private final WorkerEnvironment workerEnvironment;
Expand All @@ -169,7 +165,6 @@ public ConfigurationApi(final ConfigRepository configRepository,
final JobPersistence jobPersistence,
final SecretsRepositoryReader secretsRepositoryReader,
final SecretsRepositoryWriter secretsRepositoryWriter,

final SynchronousSchedulerClient synchronousSchedulerClient,
final StatePersistence statePersistence,
final TrackingClient trackingClient,
Expand Down Expand Up @@ -238,8 +233,6 @@ public ConfigurationApi(final ConfigRepository configRepository,
eventRunner,
configRepository);
webBackendGeographiesHandler = new WebBackendGeographiesHandler();
healthCheckHandler = new HealthCheckHandler(configRepository);
logsHandler = new LogsHandler();
openApiConfigHandler = new OpenApiConfigHandler();
}

Expand Down Expand Up @@ -291,9 +284,13 @@ public void updateWorkspaceFeedback(final WorkspaceGiveFeedback workspaceGiveFee
});
}

/**
* This implementation has been moved to {@link AttemptApiController}. Since the path of
* {@link AttemptApiController} is more granular, it will override this implementation
*/
@Override
public NotificationRead tryNotificationConfig(final Notification notification) {
return execute(() -> workspacesHandler.tryNotification(notification));
throw new NotImplementedException();
}

// SOURCE
Expand Down Expand Up @@ -893,41 +890,69 @@ public SourceDiscoverSchemaRead executeSourceDiscoverSchema(final SourceCoreConf
return execute(() -> schedulerHandler.discoverSchemaForSourceFromSourceCreate(sourceCreate));
}

/**
* This implementation has been moved to {@link JobsApiController}. Since the path of
* {@link JobsApiController} is more granular, it will override this implementation
*/
@Override
public JobInfoRead cancelJob(final JobIdRequestBody jobIdRequestBody) {
return execute(() -> schedulerHandler.cancelJob(jobIdRequestBody));
throw new NotImplementedException();
}

// JOB HISTORY

/**
* This implementation has been moved to {@link JobsApiController}. Since the path of
* {@link JobsApiController} is more granular, it will override this implementation
*/
@Override
public JobReadList listJobsFor(final JobListRequestBody jobListRequestBody) {
return execute(() -> jobHistoryHandler.listJobsFor(jobListRequestBody));
throw new NotImplementedException();
}

/**
* This implementation has been moved to {@link JobsApiController}. Since the path of
* {@link JobsApiController} is more granular, it will override this implementation
*/
@Override
public JobInfoRead getJobInfo(final JobIdRequestBody jobIdRequestBody) {
return execute(() -> jobHistoryHandler.getJobInfo(jobIdRequestBody));
throw new NotImplementedException();
}

/**
* This implementation has been moved to {@link JobsApiController}. Since the path of
* {@link JobsApiController} is more granular, it will override this implementation
*/
@Override
public JobInfoLightRead getJobInfoLight(final JobIdRequestBody jobIdRequestBody) {
return execute(() -> jobHistoryHandler.getJobInfoLight(jobIdRequestBody));
throw new NotImplementedException();
}

/**
* This implementation has been moved to {@link JobsApiController}. Since the path of
* {@link JobsApiController} is more granular, it will override this implementation
*/
@Override
public JobDebugInfoRead getJobDebugInfo(final JobIdRequestBody jobIdRequestBody) {
return execute(() -> jobHistoryHandler.getJobDebugInfo(jobIdRequestBody));
throw new NotImplementedException();
}

/**
* This implementation has been moved to {@link JobsApiController}. Since the path of
* {@link JobsApiController} is more granular, it will override this implementation
*/
@Override
public AttemptNormalizationStatusReadList getAttemptNormalizationStatusesForJob(final JobIdRequestBody jobIdRequestBody) {
return execute(() -> jobHistoryHandler.getAttemptNormalizationStatuses(jobIdRequestBody));
}

/**
* This implementation has been moved to {@link LogsApiController}. Since the path of
* {@link LogsApiController} is more granular, it will override this implementation
*/
@Override
public File getLogs(final LogsRequestBody logsRequestBody) {
return execute(() -> logsHandler.getLogs(workspaceRoot, workerEnvironment, logConfigs, logsRequestBody));
throw new NotImplementedException();
}

@Override
Expand All @@ -936,9 +961,13 @@ public File getOpenApiSpec() {
}

// HEALTH
/**
* This implementation has been moved to {@link HealthApiController}. Since the path of
* {@link HealthApiController} is more granular, it will override this implementation
*/
@Override
public HealthCheckRead getHealthCheck() {
return healthCheckHandler.health();
throw new NotImplementedException();
}

// WEB BACKEND
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/*
* Copyright (c) 2022 Airbyte, Inc., all rights reserved.
*/

package io.airbyte.server.apis;

import io.airbyte.api.generated.HealthApi;
import io.airbyte.api.model.generated.HealthCheckRead;
import io.airbyte.server.handlers.HealthCheckHandler;
import javax.ws.rs.Path;
import lombok.AllArgsConstructor;

@Path("/v1/health")
@AllArgsConstructor
public class HealthApiController implements HealthApi {

private final HealthCheckHandler healthCheckHandler;

@Override
public HealthCheckRead getHealthCheck() {
return healthCheckHandler.health();
}

}
Loading