UUID from Chef fails on RHEL 7.3

[clintoncwolfe]

Description

Reported on inspec/inspec#2985 by @djdees

When trying to read a Chef-generated UUID, the file content detection logic fails and passes a nil to the JSON parser, throwing an exception.

Looks like it's this line:
(https://github.com/chef/train/blame/master/lib/train/platforms/detect/helpers/os_common.rb#L102)

Offhand that seems fine; perhaps there are SELinux constraints preventing us from reading the file?

Train and Platform Version

1.4.1 targeting RHEL 7.3

Introduced on #270

Unknown SELinux status

Replication Case

See inspec/inspec#2985

Possible Solutions

Could check nil? on the file content

Check to verify that the file is really a file (not a dir)

Stacktrace

Traceback (most recent call last):
        30: from /ebiz/tools/ruby/bin/inspec:23:in `<main>'
        29: from /ebiz/tools/ruby/bin/inspec:23:in `load'
        28: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/bin/inspec:12:in `<top (required)>'
        27: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/base.rb:466:in `start'
        26: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'
        25: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
        24: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
        23: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/cli.rb:168:in `exec'
        22: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/runner.rb:104:in `run'
        21: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/runner.rb:132:in `run_tests'
        20: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/runner_rspec.rb:77:in `run'
        19: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:112:in `run_specs'
        18: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:81:in `report'
        17: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:171:in `finish'
        16: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:191:in `close_after'
        15: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:172:in `block in finish'
        14: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:199:in `stop'
        13: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:205:in `notify'
        12: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:205:in `each'
        11: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:206:in `block in notify'
        10: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/formatters/base.rb:72:in `stop'
         9: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/inspec/formatters/base.rb:190:in `platform'
         8: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/inspec-2.1.54/lib/resources/platform.rb:41:in `[]'
         7: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/platform.rb:52:in `[]'
         6: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/platform.rb:45:in `uuid'
         5: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/detect/uuid.rb:21:in `find_or_create_uuid'
         4: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/detect/helpers/os_common.rb:92:in `unix_uuid'
         3: from /ebiz/tools/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/train-1.4.1/lib/train/platforms/detect/helpers/os_common.rb:102:in `unix_uuid_from_chef'
         2: from /ebiz/tools/ruby-2.5.1/lib/ruby/2.5.0/json/common.rb:156:in `parse'
         1: from /ebiz/tools/ruby-2.5.1/lib/ruby/2.5.0/json/common.rb:156:in `new'
/ebiz/tools/ruby-2.5.1/lib/ruby/2.5.0/json/common.rb:156:in `initialize': no implicit conversion of nil into String (TypeError)

[clintoncwolfe]

@djdees , could you tell us a bit more:

• Are you using Chef to manage the machine? If so, anything unusual about the path /var/chef/cache/data_collector_metadata.json?
• Are you using SElinux, or any other secondary RBAC system which might result in being able to detect but not read the above path?
• Are you connecting to the machine with sudo enabled?

Thanks!

[djdees]

C. - - Chef is not actively managing the machine. It's used as part of the initial build, but not beyond that. Nothing unusual with that path, the file is there and accessible by my management account. - Nope. - Sudo is enabled and the service account I use has passwordless sudo.

…

On Mon, Apr 30, 2018 at 11:49 AM, Clinton Wolfe ***@***.***> wrote: @djdees <https://github.com/djdees> , could you tell us a bit more: - Are you using Chef to manage the machine? If so, anything unusual about the path /var/chef/cache/data_collector_metadata.json? - Are you using SElinux, or any other secondary RBAC system which might result in being able to detect but not read the above path? - Are you connecting to the machine with sudo enabled? Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#292 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIWF7WJySpl2j-d1SYWX-NXSe9_dtu9lks5tt0CogaJpZM4Ts4FT> .

-- Derek ========= djdees@gmail.com There is no frigate like a book to take us lands away. Nor any Coursers like a page of prancing poetry. This traverse may the poorest take without oppress of toll. How frugal is the chariot that bears the Human Soul? - Emily Dickinson

[rojomisin]

I am seeing this as well consistently now on centos-69 and latest chefdk 3.0.36. I am using test-kitchen ec2 driver on a hardened centos image. I have verified uuidgen is installed on centos, but I am unsure why it's failing as there's no debugging info other than:

[2018-06-28T20:46:44-07:00] DEBUG: Loading controls/main.rb into #<Inspec::ProfileContext:0x00007fab1877e370>
[2018-06-28T20:46:44-07:00] DEBUG: Loading <anonymous content> into #<Inspec::ProfileContext:0x00007fab17b1b678>
[2018-06-28T20:46:54-07:00] ERROR: Cannot find a UUID for your node.

I was thinking of adding a debugging print statement in train, but not sure how that would work with my local chefdk install and gems.

What would my nodes be missing to not be able to generate a uuid?

[chris-rock]

@rojomisin The implementation of that features is located here:

train/lib/train/platforms/detect/helpers/os_common.rb

Lines 91 to 121 in 16f70fc

	def unix_uuid
	uuid = unix_uuid_from_chef
	uuid = unix_uuid_from_machine_file if uuid.nil?
	uuid = uuid_from_command if uuid.nil?
	raise Train::TransportError, 'Cannot find a UUID for your node.' if uuid.nil?
	uuid
	end
	def unix_uuid_from_chef
	file = @backend.file('/var/chef/cache/data_collector_metadata.json')
	if file.exist? && !file.size.zero?
	json = ::JSON.parse(file.content)
	return json['node_uuid'] if json['node_uuid']
	end
	end
	def unix_uuid_from_machine_file
	%W(
	/etc/chef/chef_guid
	#{ENV['HOME']}/.chef/chef_guid
	/etc/machine-id
	/var/lib/dbus/machine-id
	/var/db/dbus/machine-id
	).each do |path|
	file = @backend.file(path)
	next unless file.exist? && !file.size.zero?
	return file.content.chomp if path =~ /guid/
	return uuid_from_string(file.content.chomp)
	end
	nil
	end

Essentially we try to read the following files:

/etc/chef/chef_guid
#{ENV['HOME']}/.chef/chef_guid
/etc/machine-id
/var/lib/dbus/machine-id
/var/db/dbus/machine-id

@rojomisin could you help us understand if your system has none of those files?
@jquick Can we document how we document how to override the uuid?

This in addition to the issue that the machine-id is not always unique (cloned vms), we may add an easy mechanism to override that?

[jquick]

This is a odd one. This error should not be hurting your setup unless your trying to use the A2 report with a non-chef node. We try to attach a UUID here:

https://github.com/inspec/inspec/blob/master/lib/inspec/formatters/base.rb#L192

but if we cannot find one we set to nil and continue. The only time its mandatory is when we are reporting up to A2. You can override it for that case using the documentation here: https://www.inspec.io/docs/reference/reporters/

[jquick]

The error message [2018-06-28T20:46:54-07:00] ERROR: Cannot find a UUID for your node. should not be stopping your run at all. @rojomisin can you share your kitchen stack trace? I am assuming the failure is from something else.

[jquick]

@djdees @clintoncwolfe What I assume is happening is we have a empty "/var/chef/cache/data_collector_metadata.json" file from the chef startup. We should add a check in train to make sure the file has data before we try to parse it.

[rojomisin]

hi @chris-rock @jquick thanks for info.

I'm using test-kitchen + chef-zero so those files are not there. That being said if I do a straight chef exec inspec exec ../profile/ -t ssh://user@<kitchen ip> --sudo --attrs ../attrs.yml it works. But in test kitchen it is reporting the error.

Could it be that the test-kitchen method of generating the uuid on my workstation (os x el cap 10.11.6 ... kinda old)?

train/lib/train/platforms/detect/specifications/os.rb

Lines 439 to 496 in de3ec80

	# bsd family
	plat.family('bsd').in_family('unix')
	.detect {
	# we need a better way to determin this family
	# for now we are going to just try each platform
	true
	}
	plat.family('darwin').in_family('bsd')
	.detect {
	if unix_uname_s =~ /darwin/i
	cmd = unix_file_contents('/usr/bin/sw_vers')
	unless cmd.nil?
	m = cmd.match(/^ProductVersion:\s+(.+)$/)
	@platform[:release] = m.nil? ? nil : m[1]
	m = cmd.match(/^BuildVersion:\s+(.+)$/)
	@platform[:build] = m.nil? ? nil : m[1]
	end
	@platform[:release] = unix_uname_r.lines[0].chomp if @platform[:release].nil?
	@platform[:arch] = unix_uname_m
	true
	end
	}
	plat.name('mac_os_x').title('macOS X').in_family('darwin')
	.detect {
	cmd = unix_file_contents('/System/Library/CoreServices/SystemVersion.plist')
	@platform[:uuid_command] = "system_profiler SPHardwareDataType | awk '/UUID/ { print $3; }'"
	true if cmd =~ /Mac OS X/i
	}
	plat.name('darwin').title('Darwin').in_family('darwin')
	.detect {
	# must be some other type of darwin
	@platform[:name] = unix_uname_s.lines[0].chomp
	true
	}
	plat.name('freebsd').title('Freebsd').in_family('bsd')
	.detect {
	if unix_uname_s =~ /freebsd/i
	@platform[:name] = unix_uname_s.lines[0].chomp
	@platform[:release] = unix_uname_r.lines[0].chomp
	true
	end
	}
	plat.name('openbsd').title('Openbsd').in_family('bsd')
	.detect {
	if unix_uname_s =~ /openbsd/i
	@platform[:name] = unix_uname_s.lines[0].chomp
	@platform[:release] = unix_uname_r.lines[0].chomp
	true
	end
	}
	plat.name('netbsd').title('Netbsd').in_family('bsd')
	.detect {
	if unix_uname_s =~ /netbsd/i
	@platform[:name] = unix_uname_s.lines[0].chomp
	@platform[:release] = unix_uname_r.lines[0].chomp
	true
	end
	}

I've been querying this issue in the test-kitchen slack channel a bit too, because it doesn't seem to be inspec.

will post debug output in a bit
the test-kitchen -D does not show any info really

-----> Verifying <default-centos-69>...
       Detected alternative framework tests for `inspec`
       Loaded tests from {:path=>"*"} 
       Loaded config_oss 
[2018-06-29T11:06:46-07:00] ERROR: Cannot find a UUID for your node.

Profile: tests from {:path=>"*"} (tests from {:path=>"*"})
Version: (not specified)
Target:  ssh://ec2-user@10.197.41.46:22

     No tests executed.

Profile: InSpec Profile (config_oss)
Version: 0.1.9
Target:  ssh://ec2-user@10.197.41.46:22

     No tests executed.

Profile: InSpec Java in system (java)
Version: 0.0.1
Target:  ssh://ec2-user@10.197.41.46:22

     No tests executed.

Test Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <default-centos-69> (0m24.16s).
-----> Kitchen is finished. (8m5.29s)