add `features test ...` command #11

joshspicer · 2022-05-11T13:56:53Z

https://github.com/github/codespaces/issues/8002

Adds devcontainer features test

devcontainer features test

Test features

Options:
      --help               Show help                                                                           [boolean]
      --version            Show version number                                                                 [boolean]
  -i, --base-image         Base Image                                                 [string] [default: "ubuntu:focal"]
  -f, --features           Feature(s) to test as space-separated parameters. Omit to auto-detect all features in
                           collection directory.  Cannot be combined with '-s'.                                  [array]
  -s, --scenarios          Path to scenario test directory (containing scenarios.json).  Cannot be combined with '-f'.
                                                                                                                [string]
  -u, --remote-user        Remote user                                                        [string] [default: "root"]
  -c, --collection-folder  Path to folder containing 'src' and 'test' sub-folders.               [string] [default: "."]
      --log-level          Log level.                              [choices: "info", "debug", "trace"] [default: "info"]
  -q, --quiet              Quiets output                                                      [boolean] [default: false]

(Folks with visibility can check out https://github.com/devcontainers/features/tree/main/test and https://github.com/devcontainers/features/tree/main/test-scenarios)

Example usage

joshspicer · 2022-05-11T21:33:22Z

example running in CI: https://github.com/devcontainers/features/runs/6396166078?check_suite_focus=true#step:3:1

stuartleeks · 2022-05-12T07:31:01Z

There's some work underway that @chrmarti and I have been discussing to enable dev container images to be build with inline cache data. This is to enable dev container image caching to work with container-features in environments such as CI where intermediate images from previous builds aren't available.

To enable this, we plan to combine the container features into a single, multi-stage build step with main image build, so I'm not sure how well that would work with this command?

An alternative might be a helper function that takes the feature details and generates the devcontainer.json for you, and then call dev-containers-cli build?

joshspicer · 2022-05-12T16:31:53Z

Having a devcontainer-cli features test ... command is definitely something we want to promote features authoring and adoption.

To enable this, we plan to combine the container features into a single, multi-stage build step with main image build, so I'm not sure how well that would work with this command?

I don't think it'll really change this? Right now this is essentially just doing a devcontainer-cli up ... with an auto-generated devcontainer.json. So long as we can do a devcontainer-cli up with a feature-filled devcontainer.json this technique should still work. (And looking forward to the multi-stage build!). To run the test scripts, it's basically just a devcontainer-cli exec.

I'd love your feedback @chrmarti on the best way to "hook into" building a devcontainer from a CLI command (you can see i'm using the launch(..) command to build and start the container, but open to suggestions on doing this better.)

stuartleeks · 2022-05-12T17:40:58Z

Ah, I'd misunderstood the appraoch - sorry!

src/spec-node/devContainersSpecCLI.ts

chrmarti · 2022-05-17T11:59:00Z

src/spec-node/featuresCLI/testContainerFeatures.ts

+    const output = `${prefix} ${msg}\n`;
+
+    if (options?.stderr) {
+        process.stderr.write(chalk.red(output));


Let's use a Log object like other commands.

I'm finding that the Log is printing out a lot more information than I want (timestamp, header, etc..). Also, I can't easily color the message.

What reason do you have for using Log in this case?

src/spec-node/featuresCLI/testContainerFeatures.ts

chrmarti · 2022-05-17T12:21:18Z

src/spec-node/featuresCLI/testContainerFeatures.ts

+            ...args
+        ]
+    };
+    const result = await doExec(execArgs);


You could probably inline launch and use the parameter objects that gets from resolve to call runRemoteCommand directly. Maybe check if that makes things clearer.

I tried to do this and found that I was duplicating a lot of code from doExec, and it was overall not working as well ($PATH wasn't accessible). I very likely was doing something wrong, but leaving this here for now. Happy to chat more about this though

(and would be nice to find a more standardized pattern. to invoke one CLI command from another)

joshspicer · 2022-05-25T17:08:42Z

eg output, testing two features go and dotnet against base image ubuntu:focal:

@joshspicer ➜ /workspaces/cli (jospicer/devcontainer-features-test ✗) $ ./devcontainer.js features test -f go -f dotnet -c ~/example

┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
|    dev container 'features' |   
│     Testing v0.3.0          │
└ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘

>  baseImage:         ubuntu:focal
>  Target Folder:     /home/node/example
>  features:          go, dotnet
>  workspaceFolder:   /tmp/vsch/container-features-test/1653498316110

⏳ Building test container...

[1 ms] @devcontainers/cli 0.3.0.
[224 ms] Start: Run: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/vsch/container-features/0.3.0-1653498316364 --build-arg _DEV_CONTAINERS_BASE_IMAGE=ubuntu:focal --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp -t vsc-1653498316110-8876ba1b135628075a1cea53a9f66029-features -f /tmp/vsch/container-features/0.3.0-1653498316364/Dockerfile.extended /tmp/__dev-containers-build-empty
[+] Building 133.2s (14/14) FINISHED                                            
 => [internal] load build definition from Dockerfile.extended              0.1s
 => => transferring dockerfile: 762B                                       0.0s
 => [internal] load .dockerignore                                          0.2s
 => => transferring context: 2B                                            0.0s
 => resolve image config for docker.io/docker/dockerfile:1.4               0.1s
 => CACHED docker-image://docker.io/docker/dockerfile:1.4@sha256:443aab4c  0.0s
 => [internal] load build definition from Dockerfile.extended              0.0s
 => [internal] load .dockerignore                                          0.0s
 => [internal] load metadata for docker.io/library/ubuntu:focal            0.0s
 => [context dev_containers_feature_content_source] load .dockerignore     0.1s
 => => transferring dev_containers_feature_content_source: 2B              0.0s
 => [context dev_containers_feature_content_source] load from client       0.1s
 => => transferring dev_containers_feature_content_source: 29.99kB         0.0s
 => CACHED [stage-0 1/4] FROM docker.io/library/ubuntu:focal               0.0s
 => [stage-0 2/4] COPY --from=dev_containers_feature_content_source . /tm  0.2s
 => [stage-0 3/4] RUN cd /tmp/build-features/go_1 && export $(cat devco  101.1s
 => [stage-0 4/4] RUN cd /tmp/build-features/dotnet_2 && export $(cat de  24.5s
 => exporting to image                                                     6.5s 
 => => exporting layers                                                    6.5s 
 => => writing image sha256:9c6df1c9f4b46ef3eff83a7472016d552f6b61279647d  0.0s 
 => => naming to docker.io/library/vsc-1653498316110-8876ba1b135628075a1c  0.0s 
[133652 ms] Start: Run: docker run --sig-proxy=false -a STDOUT -a STDERR --mount type=bind,source=/tmp/vsch/container-features-test/1653498316110,target=/workspaces/1653498316110 -l devcontainer.local_folder=/tmp/vsch/container-features-test/1653498316110 --entrypoint /bin/sh vsc-1653498316110-8876ba1b135628075a1cea53a9f66029-features -c echo Container started
Container started

🚀 Launched container.
>  containerId:          710daf4a6a71c146abe02415424cf1a9f3ecd70bdac2b6e7dea4661c8456a6a0
>  remoteUser:           root

🏃 Starting test(s)...

🧪 Executing 'go' test...


🔄 Testing 'version'

go version go1.18.2 linux/amd64
✅  Passed 'version'!
🧪 Executing 'dotnet' test...


🔄 Testing 'run dotnet info'

.NET SDK (reflecting any global.json):
 Version:   6.0.300
 Commit:    8473146e7d

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  20.04
 OS Platform: Linux
 RID:         ubuntu.20.04-x64
 Base Path:   /usr/share/dotnet/sdk/6.0.300/

Host (useful for support):
  Version: 6.0.5
  Commit:  70ae3df4a6

.NET SDKs installed:
  6.0.300 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.5 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.5 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download
✅  Passed 'run dotnet info'!


🔄 Testing 'dotnet sdks'

6.0.300 [/usr/share/dotnet/sdk]
✅  Passed 'dotnet sdks'!

✅ All tests passed!

* support fetching v2 features from github repo * update test

Changes for Features V2

joshspicer · 2022-06-29T23:29:19Z

src/spec-node/featuresCLI/utils.ts

@@ -0,0 +1,118 @@
+export const staticProvisionParams = {


I would love if we refactored the code a bit to make the provision and exec command easier to call internally. I couldn't seem to find a good blend of "easy to use" without moving or copying a bunch of code :)

chrmarti · 2022-07-08T07:43:59Z

Reapplied on main as a single commit in #81.

edgonmsft added 8 commits May 9, 2022 12:04

First changes for features

eebacc5

Missing reference.

e825692

Fix test

1047e01

Fixes.

40aa793

Run config

83afa1b

Comment out test

39cff54

Fixes for absolute paths

12d39a2

Comments on pr.

e769e66

joshspicer marked this pull request as ready for review May 11, 2022 14:01

joshspicer marked this pull request as draft May 11, 2022 14:02

joshspicer marked this pull request as ready for review May 11, 2022 20:42

Fix for github features.

b35174c

stuartleeks reviewed May 13, 2022

View reviewed changes

src/spec-node/devContainersSpecCLI.ts Outdated Show resolved Hide resolved

chrmarti reviewed May 17, 2022

View reviewed changes

chrmarti self-assigned this May 20, 2022

edgonmsft and others added 9 commits May 23, 2022 10:42

Merge remote-tracking branch 'origin/main' into edgon/features-1

6b59032

Fixes for locally cacched features.

1e9a5bb

Merge remote-tracking branch 'origin' into edgon/features-1

9b98c33

Fix old features.

5923ef7

Other fixes.

23e87ff

Some comments and formating.

9bce15f

Test fix

91edf38

Fix

971bd56

Fix test.

4e0fc7e

joshspicer force-pushed the jospicer/devcontainer-features-test branch from 1419216 to 03a2cae Compare May 25, 2022 00:50

joshspicer added 11 commits June 2, 2022 18:22

add a multi repo auth for devcontainers/* content read

104a768

multi repo auth

048ca53

update readme

122fb1b

multi repo auth

3297384

dont need devcontainers/*

acc1bd0

revert .gitignore

808633e

restore gitignore

9bcc954

revert accidentally formatted and move new command to bottom

ac36e20

remove omit

766fa68

didnt help

e6ca45c

remove print statement

7be3288

joshspicer force-pushed the jospicer/devcontainer-features-test branch from 2d54320 to 7be3288 Compare June 2, 2022 18:22

joshspicer and others added 7 commits June 3, 2022 14:28

support fetching v2 features from github repo (#48)

267ef67

* support fetching v2 features from github repo * update test

clean up tracing and formatting

df7575b

Merge remote-tracking branch 'origin/main' into edgon/features-1

1b42a63

Changes from the last spec proposal changes.

68e68b1

Comments and fixes from PR.

39a1b22

Unit test fixes.

e92dd24

Merge pull request #6 from devcontainers/edgon/features-1

66eec95

Changes for Features V2

joshspicer commented Jun 29, 2022

View reviewed changes

joshspicer force-pushed the jospicer/devcontainer-features-test branch 2 times, most recently from c76c5a1 to 52de545 Compare June 29, 2022 23:54

merge

761a232

joshspicer force-pushed the jospicer/devcontainer-features-test branch from 52de545 to 761a232 Compare June 30, 2022 00:05

joshspicer changed the base branch from edgon/features-1 to main June 30, 2022 00:14

update test scenario to support object notation

4ea8a7c

chrmarti force-pushed the main branch from 66eec95 to 60dd3bd Compare July 1, 2022 06:43

chrmarti closed this Jul 8, 2022

kieferrm mentioned this pull request Jul 9, 2022

Remote: Iteration Plan for July 2022 microsoft/vscode-remote-release#6919

Closed

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add `features test ...` command #11

add `features test ...` command #11

Uh oh!

joshspicer commented May 11, 2022 •

edited

Loading

Uh oh!

joshspicer commented May 11, 2022

Uh oh!

stuartleeks commented May 12, 2022

Uh oh!

joshspicer commented May 12, 2022 •

edited

Loading

Uh oh!

stuartleeks commented May 12, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chrmarti May 17, 2022

Uh oh!

joshspicer May 25, 2022

Uh oh!

Uh oh!

Uh oh!

chrmarti May 17, 2022

Uh oh!

joshspicer May 25, 2022

Uh oh!

joshspicer commented May 25, 2022

Uh oh!

joshspicer Jun 29, 2022

Uh oh!

chrmarti commented Jul 8, 2022

Uh oh!

Uh oh!

add features test ... command #11

add features test ... command #11

Uh oh!

Conversation

joshspicer commented May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Example usage

Uh oh!

joshspicer commented May 11, 2022

Uh oh!

stuartleeks commented May 12, 2022

Uh oh!

joshspicer commented May 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stuartleeks commented May 12, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chrmarti May 17, 2022

Choose a reason for hiding this comment

Uh oh!

joshspicer May 25, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

chrmarti May 17, 2022

Choose a reason for hiding this comment

Uh oh!

joshspicer May 25, 2022

Choose a reason for hiding this comment

Uh oh!

joshspicer commented May 25, 2022

Uh oh!

joshspicer Jun 29, 2022

Choose a reason for hiding this comment

Uh oh!

chrmarti commented Jul 8, 2022

Uh oh!

Uh oh!

add `features test ...` command #11

add `features test ...` command #11

joshspicer commented May 11, 2022 •

edited

Loading

joshspicer commented May 12, 2022 •

edited

Loading