Skip to content

Add core 'ports' functionality to docker context exporter, fix cross-platform Dockerfile write inconsistency #438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Jun 25, 2018
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
0b82cf6
Add core changes for exposed port configuration
TadCordle Jun 20, 2018
dd44f8b
Update integration tests
TadCordle Jun 20, 2018
fcfa21a
Javadoc
TadCordle Jun 20, 2018
39c34a4
Merge branch 'master' of github.com:GoogleContainerTools/jib into add…
TadCordle Jun 20, 2018
ac8d674
Merge branch 'master' of github.com:GoogleContainerTools/jib into add…
TadCordle Jun 22, 2018
218bce1
Feedback
TadCordle Jun 22, 2018
5e68c1b
Add core exportPorts functionality to docker context exporter
TadCordle Jun 22, 2018
d60d7eb
Remove EmptyStruct
TadCordle Jun 22, 2018
6b7fba5
Feedback
TadCordle Jun 22, 2018
fefd342
Add support for udp
TadCordle Jun 22, 2018
7fa64bb
Use builders and expand imports
TadCordle Jun 22, 2018
a71e7bb
i m p o r t s
TadCordle Jun 22, 2018
519d720
Merge branch 'add_ports_config' of github.com:GoogleContainerTools/ji…
TadCordle Jun 22, 2018
901c7d8
Feedback/protocol test
TadCordle Jun 22, 2018
cc25cd4
Use regex for splitting
TadCordle Jun 22, 2018
7c6f7f1
Stray newline stuff
TadCordle Jun 22, 2018
d2ab9d8
Merge branch 'add_ports_config' of github.com:GoogleContainerTools/ji…
TadCordle Jun 22, 2018
c13c6c2
Merge branch 'master' of github.com:GoogleContainerTools/jib into por…
TadCordle Jun 22, 2018
bfec9bd
?
TadCordle Jun 22, 2018
fb2fc0b
??
TadCordle Jun 22, 2018
c8dc74d
Try generating dockerfile from string instead of template file
TadCordle Jun 25, 2018
1a71919
Try specifying carriage return
TadCordle Jun 25, 2018
ddf8336
Revert "Try specifying carriage return"
TadCordle Jun 25, 2018
aab244a
Revert "Try generating dockerfile from string instead of template file"
TadCordle Jun 25, 2018
964d85a
testing
TadCordle Jun 25, 2018
9185594
Try another thing
TadCordle Jun 25, 2018
da15400
hm
TadCordle Jun 25, 2018
165a0d3
maybe
TadCordle Jun 25, 2018
213d32c
Try reading/writing line by line
TadCordle Jun 25, 2018
8036cb3
Consolidate joinAsJsonArray
TadCordle Jun 25, 2018
0324e2a
Remove template, cleanup
TadCordle Jun 25, 2018
a0aed3d
Add javadoc example for makeDockerfile()
TadCordle Jun 25, 2018
0c550c8
Use ObjectMapper
TadCordle Jun 25, 2018
5c19ef5
Remove one line function
TadCordle Jun 25, 2018
b68aebb
Move makeExposeInstructions to makeDockerfile
TadCordle Jun 25, 2018
b9bd25a
Merge branch 'master' of github.com:GoogleContainerTools/jib into por…
TadCordle Jun 25, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Preconditions;
import com.google.common.io.MoreFiles;
import com.google.common.io.Resources;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.DirectoryNotEmptyException;
Expand All @@ -31,6 +30,7 @@
import java.util.Collections;
import java.util.List;
import java.util.regex.Matcher;
import java.util.stream.Collectors;
import javax.annotation.Nullable;

/**
Expand All @@ -46,12 +46,45 @@
*/
public class DockerContextGenerator {

/**
* Formats a list for the Dockerfile's ENTRYPOINT or CMD.
*
* @see <a
* href="https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example">https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example</a>
* @param items the list of items to join into an array.
* @return a string in the format: ["item1","item2",...]
*/
@VisibleForTesting
static String joinAsJsonArray(List<String> items) {
return "["
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh for this we could actually use ObjectMapper from the JSON library we use (I forgot about it when implementing this). It will automatically generate the JSON string from a list of strings.

You could do like

Blobs.writeToString(JsonTemplateMapper.toBlob(thelist))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it needs a ListOfJsonTemplate, not a list of strings?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, maybe just use new ObjectMapper().writeValueAsString directly.

+ String.join(
",",
items
.stream()
.map(item -> "\"" + item.replaceAll("\"", Matcher.quoteReplacement("\\\"")) + "\"")
.collect(Collectors.toList()))
+ "]";
}

/**
* Builds a list of Dockerfile "EXPOSE" instructions.
*
* @param exposedPorts the list of ports numbers/ranges to expose
* @return a string containing an EXPOSE instruction for each of the entries
*/
@VisibleForTesting
static String makeExposeInstructions(List<String> exposedPorts) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also be moved to makeDockerfile and change that to append to a StringBuilder as an alternative.

return String.join(
"\n", exposedPorts.stream().map(port -> "EXPOSE " + port).collect(Collectors.toList()));
}

private final SourceFilesConfiguration sourceFilesConfiguration;

@Nullable private String baseImage;
private List<String> jvmFlags = Collections.emptyList();
private String mainClass = "";
private List<String> javaArguments = Collections.emptyList();
private List<String> exposedPorts = Collections.emptyList();

public DockerContextGenerator(SourceFilesConfiguration sourceFilesConfiguration) {
this.sourceFilesConfiguration = sourceFilesConfiguration;
Expand Down Expand Up @@ -102,11 +135,22 @@ public DockerContextGenerator setJavaArguments(List<String> javaArguments) {
return this;
}

/**
* Sets the exposed ports.
*
* @param exposedPorts the list of port numbers/port ranges to expose
* @return this
*/
public DockerContextGenerator setExposedPorts(List<String> exposedPorts) {
this.exposedPorts = exposedPorts;
return this;
}

/**
* Creates the Docker context in {@code #targetDirectory}.
*
* @param targetDirectory the directory to generate the Docker context in.
* @throws IOException if the export fails.
* @param targetDirectory the directory to generate the Docker context in
* @throws IOException if the export fails
*/
public void generate(Path targetDirectory) throws IOException {
Preconditions.checkNotNull(baseImage);
Expand Down Expand Up @@ -141,56 +185,26 @@ public void generate(Path targetDirectory) throws IOException {
}

/**
* Makes a {@code Dockerfile} from the {@code DockerfileTemplate}.
*
* @return the {@code Dockerfile} contents.
* @throws IOException if reading the Dockerfile template fails.
*/
@VisibleForTesting
String makeDockerfile() throws IOException {
Preconditions.checkNotNull(baseImage);

String dockerfileTemplate =
Resources.toString(Resources.getResource("DockerfileTemplate"), StandardCharsets.UTF_8);

return dockerfileTemplate
.replace("@@BASE_IMAGE@@", baseImage)
.replace(
"@@DEPENDENCIES_PATH_ON_IMAGE@@", sourceFilesConfiguration.getDependenciesPathOnImage())
.replace("@@RESOURCES_PATH_ON_IMAGE@@", sourceFilesConfiguration.getResourcesPathOnImage())
.replace("@@CLASSES_PATH_ON_IMAGE@@", sourceFilesConfiguration.getClassesPathOnImage())
.replace(
"@@ENTRYPOINT@@",
joinAsJsonArray(
EntrypointBuilder.makeEntrypoint(sourceFilesConfiguration, jvmFlags, mainClass)))
.replace("@@CMD@@", joinAsJsonArray(javaArguments));
}

/**
* Formats a list for the Dockerfile's ENTRYPOINT or CMD.
* Makes the contents of a {@code Dockerfile} using configuration data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're putting this in code, would be good to have in the javadoc an example of what it would look like.

*
* @see <a
* href="https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example">https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example</a>
* @param items the list of items to join into an array.
* @return a string in the format: ["item1","item2",...]
* @return the {@code Dockerfile} contents
*/
@VisibleForTesting
static String joinAsJsonArray(List<String> items) {
StringBuilder resultString = new StringBuilder("[");
boolean firstComponent = true;
for (String item : items) {
if (!firstComponent) {
resultString.append(',');
}

// Escapes quotes.
item = item.replaceAll("\"", Matcher.quoteReplacement("\\\""));

resultString.append('"').append(item).append('"');
firstComponent = false;
}
resultString.append(']');

return resultString.toString();
String makeDockerfile() {
return "FROM "
+ Preconditions.checkNotNull(baseImage)
+ "\n\nCOPY libs "
+ sourceFilesConfiguration.getDependenciesPathOnImage()
+ "\nCOPY resources "
+ sourceFilesConfiguration.getResourcesPathOnImage()
+ "\nCOPY classes "
+ sourceFilesConfiguration.getClassesPathOnImage()
+ "\n\n"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, wouldn't this add 3 newlines if there weren't any exposed ports?

+ makeExposeInstructions(exposedPorts)
+ "\nENTRYPOINT "
+ joinAsJsonArray(
EntrypointBuilder.makeEntrypoint(sourceFilesConfiguration, jvmFlags, mainClass))
+ "\nCMD "
+ joinAsJsonArray(javaArguments);
}
}
8 changes: 0 additions & 8 deletions jib-core/src/main/resources/DockerfileTemplate

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ public void testGenerate() throws IOException, URISyntaxException {
}

@Test
public void testMakeDockerList() {
public void testJoinAsJsonArray() {
Assert.assertEquals(
"[\"java\",\"-cp\",\"/app/libs/*:/app/resources/:/app/classes/\",\"\"]",
DockerContextGenerator.joinAsJsonArray(
Expand All @@ -129,22 +129,34 @@ public void testMakeDockerList() {
}

@Test
public void testMakeDockerfile() throws IOException, URISyntaxException {
public void testMakeExposeInstructions() {
Assert.assertEquals(
"EXPOSE 1000\nEXPOSE 2000-2010",
DockerContextGenerator.makeExposeInstructions(ImmutableList.of("1000", "2000-2010")));
}

@Test
public void testMakeDockerfile() throws IOException {
String expectedBaseImage = "somebaseimage";
List<String> expectedJvmFlags = Arrays.asList("-flag", "another\"Flag");
String expectedMainClass = "SomeMainClass";
List<String> expectedJavaArguments = Arrays.asList("arg1", "arg2");
List<String> exposedPorts = Arrays.asList("1000/tcp", "2000-2010/udp");

String dockerfile =
new DockerContextGenerator(mockSourceFilesConfiguration)
.setBaseImage(expectedBaseImage)
.setJvmFlags(expectedJvmFlags)
.setMainClass(expectedMainClass)
.setJavaArguments(expectedJavaArguments)
.setExposedPorts(exposedPorts)
.makeDockerfile();

Path sampleDockerfile = Paths.get(Resources.getResource("sampleDockerfile").toURI());
// Need to split/rejoin the string here to avoid cross-platform troubles
List<String> sampleDockerfile =
Resources.readLines(Resources.getResource("sampleDockerfile"), StandardCharsets.UTF_8);
Assert.assertArrayEquals(
Files.readAllBytes(sampleDockerfile), dockerfile.getBytes(StandardCharsets.UTF_8));
String.join("\n", sampleDockerfile).getBytes(StandardCharsets.UTF_8),
dockerfile.getBytes(StandardCharsets.UTF_8));
}
}
2 changes: 2 additions & 0 deletions jib-core/src/test/resources/sampleDockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,7 @@ COPY libs /app/libs/
COPY resources /app/resources/
COPY classes /app/classes/

EXPOSE 1000/tcp
EXPOSE 2000-2010/udp
ENTRYPOINT ["java","-flag","another\"Flag","-cp","/app/libs/*:/app/resources/:/app/classes/","SomeMainClass"]
CMD ["arg1","arg2"]