Question about VMM process lifecycle

[nohajc]

It is documented that calling krun_start_enter effectively transfers control over the process to libkrun which calls exit when the virtual machine is stopped (and therefore krun_start_enter does not return).

My question is twofold.

1. Why do you call _exit instead of the more "friendly" exit which among other things runs callbacks registered by atexit?
2. Would it be possible to modify libkrun so that it can be used in a context where process termination cannot be controlled by the hypervisor? I'm thinking macOS (user-space) system extension for example.

[slp]

1. Why do you call _exit instead of the more "friendly" exit which among other things runs callbacks registered by atexit?

TBH, I don't know. That code is part of the little remains from Firecracker that are still in libkrun codebase. Can't say why they choose to use _exit there, instead of std::process::exit or any other option. I would be in favor of changing it if there is a benefit without side effects.

2. Would it be possible to modify libkrun so that it can be used in a context where process termination cannot be controlled by the hypervisor? I'm thinking macOS (user-space) system extension for example.

The possibility of having a different entry point other than krun_start_enter that would not exit the process and, instead return to the caller, has been discussed before. So far the consensus was to wait for the API revamp in v2, but if there's a use case and a PR implement such entry point as a new function alternative to krun_start_enter, I'd be in favor of merging it.

[nohajc]

@slp So there aren't any fundamental reasons why this would be problematic? I mean in a similar sense such as when you have POSIX exec which basically replaces your current process memory so there's nothing to return to?

I know executing a virtual machine doesn't exactly do this but I'm no expert either. Basically, the question is whether you can easily restore CPU/thread/memory (or whatever) state when returning from the VM entry point after the hypervisor terminated.

[mtjhrc]

Why do you call _exit instead of the more "friendly" exit

I'm not sure either, maybe we could, but there probably could be some issues. _exit seems safer to me for now.

Would it be possible to modify libkrun so that it can be used in a context where process termination cannot be controlled by the hypervisor

I think it should be possible, and I think it would be nice feature. The main problem would actually be the Rust code, where we probably don't have proper Drop implementations in places, don't handle a lot of objects lifetimes (e.g. using RawFd) and we don't have mechanisms for stopping worker threads cleanly... Basically the clean shutdown paths are untested or are not implemented at all, hence we just _exit the process currently.