Skip to content

Commit 7bb6c79

Browse files
mpeddada1gcf-owl-bot[bot]
mpeddada1
and
gcf-owl-bot[bot]
authored
docs(sample): Add sample for native image support in Bigquery (#1829)
* doc(java): add sample for native image support * rename test; remove unused deps * fix readme * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
1 parent bb4dd40 commit 7bb6c79

File tree

6 files changed

+447
-0
lines changed

6 files changed

+447
-0
lines changed

‎README.md

+1
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/java-bigquery/tree
111111

112112
| Sample | Source Code | Try it |
113113
| --------------------------- | --------------------------------- | ------ |
114+
| Native Image Bigquery Sample | [source code](https://github.com/googleapis/java-bigquery/blob/main/samples/native-image-sample/src/main/java/com/example/bigquery/NativeImageBigquerySample.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigquery&page=editor&open_in_editor=samples/native-image-sample/src/main/java/com/example/bigquery/NativeImageBigquerySample.java) |
114115
| Add Column Load Append | [source code](https://github.com/googleapis/java-bigquery/blob/main/samples/snippets/src/main/java/com/example/bigquery/AddColumnLoadAppend.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigquery&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigquery/AddColumnLoadAppend.java) |
115116
| Add Empty Column | [source code](https://github.com/googleapis/java-bigquery/blob/main/samples/snippets/src/main/java/com/example/bigquery/AddEmptyColumn.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigquery&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigquery/AddEmptyColumn.java) |
116117
| Auth Drive Scope | [source code](https://github.com/googleapis/java-bigquery/blob/main/samples/snippets/src/main/java/com/example/bigquery/AuthDriveScope.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-bigquery&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/bigquery/AuthDriveScope.java) |

‎samples/native-image-sample/README.md

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# BigQuery Sample Application with Native Image
2+
3+
The BigQuery sample application demonstrates some common operations with [Google Cloud BigQuery](https://cloud.google.com/bigquery) and is compatible with Native Image compilation.
4+
5+
## Setup Instructions
6+
7+
1. Follow the [GCP Project Authentication and Native Image Setup Instructions](../../README.md).
8+
9+
2. [Enable the BigQuery APIs](https://console.cloud.google.com/apis/api/bigquery.googleapis.com).
10+
11+
### Run with Native Image Support
12+
13+
Navigate to this directory in a new terminal.
14+
15+
1. Compile the application using the Native Image Compiler. This step may take a few minutes.
16+
17+
```
18+
mvn package -P native -DskipTests
19+
```
20+
21+
2. Run the application:
22+
23+
```
24+
./target/native-image-sample
25+
```
26+
27+
3. The application will create a sample BigQuery dataset in your GCP project called `nativeimage_test_dataset` and perform some simple operations like creating a table, inserting data, and running a query.
28+
29+
If you would like to delete the BigQuery dataset later, you can manage your BigQuery resources through [Google Cloud Console](https://console.cloud.google.com/bigquery) to clean up BigQuery resources under your project.
30+
31+
When you run the application, you'll see output like this in the terminal:
32+
33+
```
34+
Created new table: nativeimage_test_table_2351b0891d2f48af9309bd289c3bad13
35+
Successfully inserted test row.
36+
Queried the following records:
37+
User id: TestUser-2f39e3ec-d81a-483f-9ec0-b9bd54155710 | age: 40
38+
Deleted table: nativeimage_test_table_2351b0891d2f48af9309bd289c3bad13
39+
```
40+
41+
### Sample Integration test with Native Image Support
42+
43+
In order to run the sample integration test, call the following command:
44+
45+
```
46+
mvn test -Pnative
47+
```

‎samples/native-image-sample/pom.xml

+161
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
<?xml version='1.0' encoding='UTF-8'?>
2+
<!--
3+
~ Copyright 2022 Google LLC
4+
~
5+
~ Licensed under the Apache License, Version 2.0 (the "License");
6+
~ you may not use this file except in compliance with the License.
7+
~ You may obtain a copy of the License at
8+
~
9+
~ http://www.apache.org/licenses/LICENSE-2.0
10+
~
11+
~ Unless required by applicable law or agreed to in writing, software
12+
~ distributed under the License is distributed on an "AS IS" BASIS,
13+
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
~ See the License for the specific language governing permissions and
15+
~ limitations under the License.
16+
-->
17+
<project xmlns="http://maven.apache.org/POM/4.0.0"
18+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
19+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
20+
<modelVersion>4.0.0</modelVersion>
21+
<groupId>com.example.bigquery</groupId>
22+
<artifactId>native-image-sample</artifactId>
23+
<name>Native Image Sample</name>
24+
25+
<parent>
26+
<groupId>com.google.cloud.samples</groupId>
27+
<artifactId>shared-configuration</artifactId>
28+
<version>1.2.0</version>
29+
</parent>
30+
31+
<properties>
32+
<maven.compiler.target>1.8</maven.compiler.target>
33+
<maven.compiler.source>1.8</maven.compiler.source>
34+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
35+
</properties>
36+
37+
<dependencyManagement>
38+
<dependencies>
39+
<dependency>
40+
<groupId>com.google.cloud</groupId>
41+
<artifactId>libraries-bom</artifactId>
42+
<version>24.2.0</version>
43+
<type>pom</type>
44+
<scope>import</scope>
45+
</dependency>
46+
</dependencies>
47+
</dependencyManagement>
48+
49+
<dependencies>
50+
<dependency>
51+
<groupId>com.google.cloud</groupId>
52+
<artifactId>google-cloud-bigquery</artifactId>
53+
</dependency>
54+
55+
<!-- Test dependencies -->
56+
<dependency>
57+
<groupId>junit</groupId>
58+
<artifactId>junit</artifactId>
59+
<version>4.13.2</version>
60+
<scope>test</scope>
61+
</dependency>
62+
<dependency>
63+
<groupId>com.google.truth</groupId>
64+
<artifactId>truth</artifactId>
65+
<version>1.1.3</version>
66+
<scope>test</scope>
67+
</dependency>
68+
</dependencies>
69+
70+
<build>
71+
<plugins>
72+
<plugin>
73+
<groupId>org.apache.maven.plugins</groupId>
74+
<artifactId>maven-jar-plugin</artifactId>
75+
<configuration>
76+
<archive>
77+
<manifest>
78+
<mainClass>com.example.bigquery.NativeImageBigquerySample
79+
</mainClass>
80+
</manifest>
81+
</archive>
82+
</configuration>
83+
</plugin>
84+
</plugins>
85+
</build>
86+
87+
<!-- Native Profile-->
88+
<profiles>
89+
<profile>
90+
<id>native</id>
91+
92+
<dependencies>
93+
<dependency>
94+
<groupId>com.google.cloud</groupId>
95+
<artifactId>native-image-support</artifactId>
96+
<version>0.10.0</version>
97+
</dependency>
98+
<dependency>
99+
<groupId>org.junit.vintage</groupId>
100+
<artifactId>junit-vintage-engine</artifactId>
101+
<version>5.8.2</version>
102+
<scope>test</scope>
103+
</dependency>
104+
<dependency>
105+
<groupId>org.graalvm.buildtools</groupId>
106+
<artifactId>junit-platform-native</artifactId>
107+
<version>0.9.9</version>
108+
<scope>test</scope>
109+
</dependency>
110+
111+
</dependencies>
112+
113+
<build>
114+
<plugins>
115+
<plugin>
116+
<groupId>org.apache.maven.plugins</groupId>
117+
<artifactId>maven-surefire-plugin
118+
</artifactId> <!-- Must use older version of surefire plugin for native-image testing. -->
119+
<version>2.22.2</version>
120+
<configuration>
121+
<includes>
122+
<include>**/*IT</include>
123+
</includes>
124+
</configuration>
125+
</plugin>
126+
<plugin>
127+
<groupId>org.graalvm.buildtools</groupId>
128+
<artifactId>native-maven-plugin</artifactId>
129+
<version>0.9.9</version>
130+
<extensions>true</extensions>
131+
<configuration>
132+
<mainClass>com.example.bigquery.NativeImageBigquerySample
133+
</mainClass>
134+
<buildArgs>
135+
<buildArg>--no-fallback</buildArg>
136+
<buildArg>--no-server</buildArg>
137+
</buildArgs>
138+
</configuration>
139+
<executions>
140+
<execution>
141+
<id>build-native</id>
142+
<goals>
143+
<goal>build</goal>
144+
<goal>test</goal>
145+
</goals>
146+
<phase>package</phase>
147+
</execution>
148+
<execution>
149+
<id>test-native</id>
150+
<goals>
151+
<goal>test</goal>
152+
</goals>
153+
<phase>test</phase>
154+
</execution>
155+
</executions>
156+
</plugin>
157+
</plugins>
158+
</build>
159+
</profile>
160+
</profiles>
161+
</project>

‎samples/native-image-sample/src/main/java/com/example/bigquery/NativeImageBigquerySample.java

+170
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
/*
2+
* Copyright 2022 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
package com.example.bigquery;
18+
19+
import com.google.api.gax.paging.Page;
20+
import com.google.cloud.bigquery.BigQuery;
21+
import com.google.cloud.bigquery.BigQueryError;
22+
import com.google.cloud.bigquery.BigQueryOptions;
23+
import com.google.cloud.bigquery.Dataset;
24+
import com.google.cloud.bigquery.DatasetInfo;
25+
import com.google.cloud.bigquery.Field;
26+
import com.google.cloud.bigquery.FieldValueList;
27+
import com.google.cloud.bigquery.InsertAllRequest;
28+
import com.google.cloud.bigquery.InsertAllResponse;
29+
import com.google.cloud.bigquery.QueryJobConfiguration;
30+
import com.google.cloud.bigquery.Schema;
31+
import com.google.cloud.bigquery.StandardSQLTypeName;
32+
import com.google.cloud.bigquery.StandardTableDefinition;
33+
import com.google.cloud.bigquery.Table;
34+
import com.google.cloud.bigquery.TableDefinition;
35+
import com.google.cloud.bigquery.TableId;
36+
import com.google.cloud.bigquery.TableInfo;
37+
import com.google.cloud.bigquery.TableResult;
38+
import java.util.HashMap;
39+
import java.util.List;
40+
import java.util.Map;
41+
import java.util.UUID;
42+
43+
/**
44+
* Sample application demonstrating BigQuery operations.
45+
*
46+
* <p>Note: This application will create a BigQuery dataset in your GCP project. You can delete this
47+
* by viewing BigQuery in Cloud Console https://console.cloud.google.com/bigquery or by uncommenting
48+
* the call to `deleteDataset(..)` made in main().
49+
*/
50+
public class NativeImageBigquerySample {
51+
52+
private static final String DATASET_ID = "nativeimage_test_dataset";
53+
54+
private static final String TABLE_ID = "nativeimage_test_table";
55+
56+
private static final Schema TABLE_SCHEMA =
57+
Schema.of(
58+
Field.of("id", StandardSQLTypeName.STRING), Field.of("age", StandardSQLTypeName.INT64));
59+
60+
/** Entrypoint to the application. */
61+
public static void main(String[] args) throws InterruptedException {
62+
BigQuery bigQuery = BigQueryOptions.getDefaultInstance().getService();
63+
64+
if (!hasDataset(bigQuery, DATASET_ID)) {
65+
createDataset(bigQuery, DATASET_ID);
66+
}
67+
68+
String tableName = TABLE_ID + "_" + UUID.randomUUID().toString().replace("-", "");
69+
createTable(bigQuery, DATASET_ID, tableName, TABLE_SCHEMA);
70+
String testId = "TestUser-" + UUID.randomUUID().toString();
71+
int testAge = 40;
72+
insertTestRecord(bigQuery, DATASET_ID, tableName, testId, testAge);
73+
queryTable(bigQuery, DATASET_ID, tableName);
74+
75+
// Clean up resources.
76+
deleteTable(bigQuery, DATASET_ID, tableName);
77+
78+
// Uncomment this to delete the created dataset.
79+
// deleteDataset(bigQuery, DATASET_ID);
80+
}
81+
82+
static String queryTable(BigQuery bigQuery, String datasetName, String tableName)
83+
throws InterruptedException {
84+
String fullyQualifiedTable = datasetName + "." + tableName;
85+
String query = "SELECT * FROM " + fullyQualifiedTable;
86+
87+
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query).build();
88+
TableResult results = bigQuery.query(queryConfig);
89+
90+
String result = "";
91+
System.out.println("Queried the following records: ");
92+
for (FieldValueList row : results.iterateAll()) {
93+
String rowStatement =
94+
String.format(
95+
"User id: %s | age: %d\n",
96+
row.get("id").getStringValue(), row.get("age").getLongValue());
97+
result += rowStatement;
98+
System.out.println(row);
99+
}
100+
return result;
101+
}
102+
103+
static void insertTestRecord(
104+
BigQuery bigQuery, String datasetName, String tableName, String id, int age) {
105+
106+
Map<String, Object> rowContent = new HashMap<>();
107+
rowContent.put("id", id);
108+
rowContent.put("age", age);
109+
110+
InsertAllRequest request =
111+
InsertAllRequest.newBuilder(datasetName, tableName).addRow(rowContent).build();
112+
113+
InsertAllResponse response = bigQuery.insertAll(request);
114+
115+
if (response.hasErrors()) {
116+
System.out.println("Insert resulted in errors:");
117+
for (Map.Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
118+
System.out.println("Response error: \n" + entry.getValue());
119+
}
120+
} else {
121+
System.out.println("Successfully inserted test row.");
122+
}
123+
}
124+
125+
static void createTable(BigQuery bigQuery, String datasetName, String tableName, Schema schema) {
126+
127+
TableId tableId = TableId.of(datasetName, tableName);
128+
TableDefinition tableDefinition = StandardTableDefinition.of(schema);
129+
TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
130+
bigQuery.create(tableInfo);
131+
System.out.println("Created new table: " + tableName);
132+
}
133+
134+
static boolean hasTable(BigQuery bigQuery, String datasetName, String tableName) {
135+
136+
Page<Table> tables = bigQuery.listTables(datasetName);
137+
for (Table table : tables.iterateAll()) {
138+
if (tableName.equals(table.getTableId().getTable())) {
139+
return true;
140+
}
141+
}
142+
return false;
143+
}
144+
145+
static void createDataset(BigQuery bigQuery, String datasetName) {
146+
DatasetInfo datasetInfo = DatasetInfo.newBuilder(datasetName).build();
147+
Dataset newDataset = bigQuery.create(datasetInfo);
148+
System.out.println("Created new dataset: " + newDataset.getDatasetId().getDataset());
149+
}
150+
151+
static boolean hasDataset(BigQuery bigQuery, String datasetName) {
152+
Page<Dataset> datasets = bigQuery.listDatasets();
153+
for (Dataset dataset : datasets.iterateAll()) {
154+
if (datasetName.equals(dataset.getDatasetId().getDataset())) {
155+
return true;
156+
}
157+
}
158+
return false;
159+
}
160+
161+
static void deleteTable(BigQuery bigQuery, String datasetName, String tableName) {
162+
bigQuery.getTable(datasetName, tableName).delete();
163+
System.out.println("Deleted table: " + tableName);
164+
}
165+
166+
static void deleteDataset(BigQuery bigQuery, String datasetName) {
167+
bigQuery.getDataset(datasetName).delete();
168+
System.out.println("Deleting dataset " + datasetName);
169+
}
170+
}

0 commit comments

Comments
 (0)