Orbit authentication with Fleet

[sharvilshah]

Goal

In order to us to remotely manage osquery flags (CLI flags in osquery which requires restarting osquery on the host), orbit should "enroll" and authenticate with Fleet

Parent Epic

• Manage flags for osquery remotely via Orbit #6851

How?

• Create a new /orbit endpoint on Fleet
  • This new endpoint will be used for all subsequent orbit related API calls (/orbit/enroll, /orbit/<team|host>/flags, etc)
• EnrollRequest <> EnrollResponse
  • When orbit requests an enroll on the new endpoint, Fleet should create a new orbit_node_key similar to osquery's node_key
  • orbit should use the existing enroll_secret to request an enrollment with Fleet
  • orbit should save the node_key locally (initially on filesystem, later migrate to something like Pebble by CockroachDB (https://pkg.go.dev/github.com/cockroachdb/pebble)
  • re-enrollment and invalidation:
    • Ideally both orbit and osquery node-keys should handle invalidation and reenrollment in tandem (TBD)
    • Failure scenario: what happens when Fleet is unreachable (retry?) (TBD)
• Extend host MySQL datastore to store the following: host_id, node_key, orbit_node_key
  • orbit and osquery on the same host, shouldn't cause duplicate hosts

[lucasmrod]

Hi @sharvilshah.

We've discussed a way in which Fleet could match an Orbit instance and its child osqueryd instance (basically to determine if both instances are coming from the same host).

Osquery enroll API is of the following form:

{
  "enroll_secret": "...", // Optional.
  "host_identifier": "..." // Determined by the --host_identifier flag
  "host_details": { // A dictionary of keys mapping to helpful osquery tables.
    "os_version": {},
    "osquery_info": {},
    "system_info": {},
    "platform_info": {}
  }
}

Fleet stores the host_identifier as hosts.osquery_host_id in MySQL. (And that's how Fleet identifies the host in case of e.g. re-enroll.)

Orbit always starts osqueryd with --host_identifier=uuid:

fleet/orbit/pkg/osquery/flags.go

Lines 9 to 13 in acf6018

	func FleetFlags(fleetURL *url.URL) []string {
	hostname, prefix := fleetURL.Host, fleetURL.Path
	return []string{
	// Use uuid as the default identifier -- users can override this in their flagfile
	"--host_identifier=uuid",

Such uuid value for --host_identifier is the UUID of the hardware:
https://github.com/osquery/osquery/blob/master/osquery/core/system.cpp#L126

Proposal: Orbit could run the following command before starting the osqueryd daemon:

# /opt/orbit/bin/osqueryd/macos-app/stable/osquery.app/Contents/MacOS/osqueryd -S --json "select uuid from system_info;"
[
  {"uuid":"3EBADBB4-B0B1-4BB2-ADAB-E82A53D6045D"}
]

# (system_info.uuid is the hardware UUID.)

Then, in the Orbit "enroll request", Orbit would send such hardware UUID + enroll secret. That way, Fleet has a way to match the two in MySQL.

Let me know if it makes sense to you.

[lukeheath]

@zhumo Question from @mna: Does this issue include the updates necessary for Orbit to receive agent options/flags or do we need to open a new ticket for that?

(As related to #7377)

cc @noahtalerman

[zhumo]

Hey @lukeheath @mna, No this issue does not. This is a sub-issue of

• Manage flags for osquery remotely via Orbit #6851
  which is parent epic encompassing this whole effort.

This ticket only reflects the one-off work needed for orbit to authenticate and communicate with fleet server. A follow-on ticket will be created, when @sharvilshah starts working on it, to address the actual reading/updating of flags. Does that answer your question?

[zhumo]

@lukeheath @mna never mind!

#7246 covers both the flags and the authentication piece.

[zhumo]

@sharvilshah @zwass Hey! has this been addressed and ready for release? Its sibling issues are, but this one is not marked.

[zhumo]

@zwass

[zwass]

Yes, this was done as part of #7246.