Make all results from amps plugins exportable

[amard33p]

Ref issue: #1556
Currently amps plugins are not exportable:

glances/glances/exports/glances_export.py

Lines 26 to 31 in 13f6db0

	# List of non exportable plugins
	# @TODO: remove this part and make all plugins exportable (see issue #1556)
	# @TODO: also make this list configurable by the user (see issue #1443)
	non_exportable_plugins = [
	'alert',
	'amps',

At work we require lot of custom metrics and AMPs are great for that usecase. It would be great if we could export the results of AMP scripts.
Here's the proposal:
A single AMP script may export single or multiple metrics.
For a single metric, the output CSV should only contain the column amps_AMP_name_result.
Example:

[amp_conntrack]
enable=false
refresh=30
one_line=false
command=sysctl net.netfilter.nf_conntrack_count

Expected output CSV:

amps_conntrack_result
100

Now if the AMP script result contains multiple fields, it needs to output those fields in a predefined format.
Example format: value_1,100;value_2,400

To implement this we can probably create a new attribute for AMP like result_fields. Example:

[amp_multi_field_amp]
enable=true
refresh=15
one_line=false
command=/usr/local/bin/mycommand.sh
result_fields=value_1,value_2

Expected output CSV:

amps_multi_field_amp_value_1, amps_multi_field_amp_value_2
100, 400

[nicolargo]

Hi @amard33p

what is the output format of /usr/local/bin/mycommand.sh (when it ran from the command line) ?

[amard33p]

@nicolargo Here's an example:

#!/bin/bash
# Example amp script which exports multiple values
# /usr/local/bin/mycommand.sh
nf_conntrack_count=$(sysctl net.netfilter.nf_conntrack_count | cut -d'=' -f2 | xargs)
nf_conntrack_max=$(sysctl net.netfilter.nf_conntrack_max | cut -d'=' -f2 | xargs)
echo "nf_conntrack_count,$nf_conntrack_count; nf_conntrack_max,$nf_conntrack_max"

Output:
nf_conntrack_count,681; nf_conntrack_max,1835008

Again, this is just my idea...we can discuss on a more optimal output format.

[nicolargo]

Ok i understand, let me reformulate: you want a way to configure AMP plugin output in order to export it in a key=value format.

I propose to implement the following behavors:

Output format of the AMP script is a JSON

Ex: output of myamp2.sh

{'key1': ''value1", key2": "value2"}

Glances configuration file:

[amp_myamp2]
enable=true
refresh=15
one_line=false
command=/usr/local/bin/myamp1.sh
output_format=json

Expected output CSV for the AMP plugin:

amps_myamp1_key1, amps_myamp1_key1
value1, value2

Note: another fields will also be added as meta data for the AMP.

Output format of the AMP script is a CSV

Ex: output of myamp1.sh

key1,key2
value1,value2

Glances configuration file:

[amp_myamp1]
enable=true
refresh=15
one_line=false
command=/usr/local/bin/myamp1.sh
output_format=csv

Expected output CSV for the AMP plugin:

amps_myamp1_key1, amps_myamp1_key1
value1, value2

Note: another fields will also be added as meta data for the AMP.

Additional information

Just for the record, what you want to do in your amp_conntrack example already exist as a non AMP plugin:

glances --stdout-csv connections --enable-plugin connections
connections.net_connections_enabled,connections.nf_conntrack_enabled,connections.LISTEN,connections.ESTABLISHED,connections.SYN_SENT,connections.SYN_RECV,connections.initiated,connections.terminated,connections.nf_conntrack_count,connections.nf_conntrack_max,connections.nf_conntrack_percent
True,True,39,15,0,0,0,0,250.0,262144.0,0.095367431640625
...

[amard33p]

@nicolargo Overall I do like your idea to specify the output format but wanted to point out that outputting JSON in bash is not straightforward. This is the easiest way I found. We should mention this in the docs when the feature becomes available.
printf '{"key1":"%s","key2":"%s"}\n' "$val1" "$val2"

Also I used the amp_conntrack just as an example...glad to know it's supported natively.

[nicolargo]

First part of the job done on the issue2394 branch.

AMPs export is enable. Not the behavor regarding the AMPs script output.

[nicolargo]

Test bed:

./venv/bin/python -m glances --disable-plugin all --enable-plugin cpu,mem --export json --export-json-file /tmp/glances.json

Works in progress in branch issue2394.

[nicolargo]

Merged into develop.

All plugins are now exportable.

Note: for CSV export, header field now include the plugin name (ex: cpu.total).

Please test the develop version @amard33p