Attempt running singer taps from a docker container #44
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Singer binaries (and their environments) used in java tests should be the exact same as what we ship in our docker container. While it's easy to isolate Singer binaries (they're stored in a single folder), it's not easy to isolate their dependencies. For example,
tap-postgres
requirespg_config
andgcc
to be installed on the host machine/container. By default,alpine-linux
does not ship with these dependencies, so we'd need a custom instruction in the Dockerfile to install them. But without building the container then running tests inside it (or encountering a bug while running it), we don't know if the local env in which tests succeeded accurately mirrors the docker container we produce.Potential solutions
This draft PR is me exploring solution no.2. Resolution of this problem is currently blocking writing any real tests for singer workers, so I'll need to figure this out to be able to meaningfully complete workers.
EDIT: I think i will go for solution 1 since I realized the docker container runs tests as part of its build process, so we'll catch any issues I'm worried about here. Ignore the docker file for now, will generate singer binaries as part of the build.