Add gzip compression option for exporter

[kw7oe]

Address #123. WIP, just PR to get early feedback.

Todo:

• Add test to verify http exporter with compression enabled
• Implement gzip compression for grpc.
• Add test to verify grpc exporter with compression enabled

[codecov]

Codecov Report

Merging #306 (94ddb7f) into main (0bc0f4d) will increase coverage by 0.12%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #306      +/-   ##
==========================================
+ Coverage   38.29%   38.41%   +0.12%     
==========================================
  Files          54       54              
  Lines        3562     3569       +7     
==========================================
+ Hits         1364     1371       +7     
  Misses       2198     2198              

Flag	Coverage Δ
api	65.29% <ø> (ø)
elixir	15.29% <ø> (ø)
erlang	38.43% <100.00%> (+0.12%)	⬆️
exporter	19.66% <100.00%> (+0.24%)	⬆️
sdk	76.64% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ntelemetry_exporter/src/opentelemetry_exporter.erl	72.15% <100.00%> (+1.29%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0bc0f4d...94ddb7f. Read the comment docs.

[kw7oe]

For passing gzip into grpcbox encoding configuration, I did some research and came across grpcbox README. Given that, I'm still not really sure how to pass it.

I'm thinking is either here:

init(Opts) ->
    %% ...
    Compression = maps:get(compression, Opts1, undefined),
    case maps:get(protocol, Opts1, http_protobuf) of
        grpc ->
            %% Pass in here?
            ChannelOpts = maps:get(channel_opts, Opts1, #{}),
            case grpcbox_channel:start_link(?MODULE, grpcbox_endpoints(Endpoints), ChannelOpts) of

or here:

export(Tab, Resource, #state{protocol=grpc,
                             grpc_metadata=Metadata,
                             channel_pid=_ChannelPid}) ->
    %% ...
    %% Pass in along side `#{channel => ?MODULE}`?
    case opentelemetry_trace_service:export(Ctx, ExportRequest, #{channel => ?MODULE}) of

Would be great if anyone can point me to the right direction here

[kw7oe]

@tsloughter I'm trying to add the test for grpc but ending up have failing test on the second grpc test group due to {error, no_endpoints}. Test pass if I'm just having one grpc test group.

Do you have any idea what might caused this?

Here's my diff:

diff --git a/apps/opentelemetry_exporter/src/opentelemetry_exporter.erl b/apps/opentelemetry_exporter/src/opentelemetry_exporter.erl
index 79c89a9..59651e9 100644
--- a/apps/opentelemetry_exporter/src/opentelemetry_exporter.erl
+++ b/apps/opentelemetry_exporter/src/opentelemetry_exporter.erl
@@ -180,7 +180,11 @@ export(Tab, Resource, #state{protocol=grpc,
                              channel_pid=_ChannelPid}) ->
     ExportRequest = tab_to_proto(Tab, Resource),
     Ctx = grpcbox_metadata:append_to_outgoing_ctx(ctx:new(), Metadata),
-    case opentelemetry_trace_service:export(Ctx, ExportRequest, #{channel => ?MODULE}) of
+    Result = opentelemetry_trace_service:export(Ctx, ExportRequest, #{channel
+                                                                      =>
+                                                                      ?MODULE}),
+    ct:print("~p~n", [Result]),
+    case Result  of
         {ok, _Response, _ResponseMetadata} ->
             ok;
         {error, {Status, Message}, _} ->
diff --git a/apps/opentelemetry_exporter/test/opentelemetry_exporter_SUITE.erl b/apps/opentelemetry_exporter/test/opentelemetry_exporter_SUITE.erl
index d2f73ac..e8afaff 100644
--- a/apps/opentelemetry_exporter/test/opentelemetry_exporter_SUITE.erl
+++ b/apps/opentelemetry_exporter/test/opentelemetry_exporter_SUITE.erl
@@ -9,12 +9,12 @@
 -include_lib("opentelemetry/include/otel_span.hrl").
 
 all() ->
-    [{group, functional}, {group, http_protobuf}, {group,
-                                                   http_protobuf_gzip}, {group, grpc}].
+    [{group, functional}, {group, http_protobuf}, {group, http_protobuf_gzip}, {group, grpc}, {group, grpc_gzip}].
 
 groups() ->
     [{functional, [], [configuration, span_round_trip, ets_instrumentation_info]},
      {grpc, [], [verify_export]},
+     {grpc_gzip, [], [verify_export]},
      {http_protobuf, [], [verify_export]},
      {http_protobuf_gzip, [], [verify_export]}].
 
@@ -31,6 +31,9 @@ init_per_group(Group, Config) when Group =:= grpc ;
 init_per_group(http_protobuf_gzip, Config) ->
     application:ensure_all_started(opentelemetry_exporter),
     [{protocol, http_protobuf}, {compression, gzip} | Config];
+init_per_group(grpc_gzip, Config) ->
+    application:ensure_all_started(opentelemetry_exporter),
+    [{protocol, grpc}, {compression, gzip} | Config];
 init_per_group(_, _) ->
     application:load(opentelemetry_exporter),
     ok.

Output:

----------------------------------------------------
2021-11-14 20:34:26.825
{error,no_endpoints}

[kw7oe]

CI test failed due to collector:

ERROR: for opentelemetry-erlang_otel_1  Cannot start service otel: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/otelcontribcol": permission denied: unknown

ERROR: for otel  Cannot start service otel: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/otelcontribcol": permission denied: unknown
Encountered errors while bringing up the project.
Error: Process completed with exit code 1.

[tsloughter]

@kw7oe yea, I got that too and am not able to reproduce locally so haven't figured out how to debug it :(

[kw7oe]

@kw7oe yea, I got that too and am not able to reproduce locally so haven't figured out how to debug it :(

This is referring to the CI/CD fail right? I think is a recent issues: open-telemetry/opentelemetry-collector-contrib#6238

I am able to replicate it using a different machine:

I guess our local machine is cached or something so we can't replicate it

Updates:

After a few testing, the latest otel/opentelemetry-collector-contrib-dev version that works is 93a9885459c9406db8ac446f77f290b02542e8d5.