parseNameValue: fix no quoting support #234
Conversation
|
Hi, Is Someone can take a look on the travis build problem ? Benoit |
|
please rebase to master, than this should go away |
bdolez
force-pushed
the
fix/name-value-list
branch
from
090e2d2 to
efce0c2
Compare
|
Rebase done but seems to have a greater problem with travis ...
Le 14/02/2017 à 11:37, Rainer Gerhards a écrit :
… please rebase to master, than this should go away
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#234 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AXTAUd-whFeQ_2JP-fIhEzcgdQ_UQpnaks5rcYQDgaJpZM4LOR2l>.
--
Benoit DOLEZ, POM Monitoring, http://www.pom-monitoring.com/
|
Function description announce support of quoted variable :
- name=value
- name="value"
- name='value'
... but there is nothing in the code to really support quoted string.
I had small fixes to support it.
My tests :
$ cat > test.txt <<EOF
a=
a=123
a=123
a='123'
a="123"
a=123 b=456
a=123 b="456"
a=123 b="456" c=
a=123 b="456'789" c=
a=123 b="456 789" c=
a=123 b="456 789' c=
a=123 b=456 789 c=
EOF
$ cat > test.rb << EOF
version=2
rule=test-ok:%[
{"type": "name-value-list"}
]%
EOF
$ cat test.txt | ./lognormalizer -r test.rb -T
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "originalmsg": "a=123 b=\"456 789' c= ", "unparsed-data": "a=123 b=\"456 789' c= " }
{ "originalmsg": "a=123 b=456 789 c= ", "unparsed-data": "a=123 b=456 789 c= " }
bdolez
force-pushed
the
fix/name-value-list
branch
from
efce0c2 to
69392da
Compare
|
Ok, I have found the problem : a extraneous whitespace at end of line. |
|
Hi, Do you think it can be merged ? Regards |
|
Sorry for the long silence. I would like to do some additional verification that nothing in existing use is broken. I am not sure if the current testbench is sufficient in this regards. Pre-merging this PR the state is known good and stable. So I think I'll craft a release with existing code today, then work on some tests and see that I merge the new changes within the next two weeks (pls ping me if I don't do!!!). Then, we can craft yet another release. |
|
Hi |
|
@solhuebner, yes but your sample log isn't compliant because of spaces in key fields |
|
@rgerhards Would it be possible to have this PR merged? Is there more work to be done or any help needed with testing this issue? Thanks! |
|
This PR is very important for us, could you please consider merging it ? |
|
mhhh.. somehow I have lost track. Going through the comments my concern still is that we may break existing implementations. I think the new behavior needs to be guarded by an option which explicitly needs to be opted in. |
I've been trying to implement a key-value @rgerhards - Unless existing implementations have unbalanced quotes for values, my reading of this patch is that it won't hurt anything. It correctly parses quoted values only if the quotes exist in the first and last character positions, and are the same type of quotes. Can you think of a particular implementation that might be broken by this? |
|
Hi, it has been a long time ... I rebase this PR on last master, but I do not have access to old repo (inactive company one). Can you change the reference to this one : https://github.com/bdolez/liblognorm/tree/fix/name-value-list As I understand, you want an option to prevent inconsistant operation ? How would you define this option ? in rulebase file ? Perhaps duplicate code to a new type ? Regards Benoit |
|
Hello, Any update on the PR? This would be extremely useful!! We are attempting to parse Fortigate logs which come in a lot of different varieties, but always in key value pairs. The key value pairs come in all sorts of different formats that this PR accomplishes. And I agree with @mjbnz, this just seems to extend support as opposed to alter it. Unless there is a specific use case in mind, then maybe have a new keyword for this or some opt-in keyword in the RB. I would like to note, I pulled this and tested and verified that this works as intended. |
I would just expect a param option to enable the feature something like %type:name-value-list{"extended":"yes"}%. If we needed to keep it behind an option. not sure its needed but would solve backwards compatibility concerns i think. |
|
Hi, I have no response from @rgerhards about changing PR reference. Can someone tell him to check this problem ? Ok for the option (extended:yes). There is more code to change to add an option to type name-value-list ... I'm not sure what to do. Can someone confirm before I begin ?
Benoit |
|
Check PR #335. It seems to be doing the same thing and is already planning options. |
|
Thx for the heads-up. I will check #335 this week (if you don't hear back pls ping me). If it is backwards-compatible, it should be easy to merge. |
|
replaced by #335 |
liblognorm pull request #234 parseNameValue: fix no quoting support rsyslog/liblognorm#234
Function description announce support of quoted variable :
... but there is nothing in the code to really support quoted string.
I had small fixes to support it.
My tests :
$ cat > test.txt <<EOF
a=
a=123
a=123
a='123'
a="123"
a=123 b=456
a=123 b="456"
a=123 b="456" c=
a=123 b="456'789" c=
a=123 b="456 789" c=
a=123 b="456 789' c=
a=123 b=456 789 c=
EOF
$ cat > test.rb << EOF
version=2
rule=test-ok:%[
{"type": "name-value-list"}
]%
EOF
$ cat test.txt | ./lognormalizer -r test.rb -T
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "event.tags": [ "test-ok" ] }
{ "originalmsg": "a=123 b="456 789' c= ", "unparsed-data": "a=123 b="456 789' c= " }
{ "originalmsg": "a=123 b=456 789 c= ", "unparsed-data": "a=123 b=456 789 c= " }