RFC: Refactor Crystal::EventLoop to disconnect it from LibEvent

[lbguilherme]

LibEvent is all about detecting when a file/socket is ready to be read or written without blocking. A non-blocking read from a socket is thus a sequence of two steps: first wait for it to be readable, then read from it without blocking. If the system happens to return a EWOULDBLOCK error, just try waiting for it to be readable again. This is the reason IO::Evented exists. For example:

def evented_read(slice : Bytes, errno_msg : String) : Int32
  loop do
    bytes_read = yield slice
    if bytes_read != -1
      # `to_i32` is acceptable because `Slice#size` is an Int32
      return bytes_read.to_i32
    end

    if Errno.value == Errno::EAGAIN
      wait_readable
    else
      raise IO::Error.from_errno(errno_msg)
    end
  end
ensure
  resume_pending_readers
end

The problem is that not all async IO work like that. Windows' Overlapped IO works by submitting a "read request" to the system and them waiting for an event when it completes in a single step. Linux's io_uring is very similar.

It is not viable to implement efficient async IO on Windows or on Linux's io_uring on top of the current IO::Evented interface. Please refer to these comments by ysbaddaden and RX14 as well: #8651.

---

I propose changing Crystal::EventLoops interface to this:

module Crystal::EventLoop
  # Runs the event loop.
  def self.run_once : Nil
  end

  {% unless flag?(:preview_mt) %}
    # Reinitializes the event loop after a fork.
    def self.after_fork : Nil
    end
  {% end %}

  # All methods below will block the current fiber until the operation is complete.
  # run_once is responsible for enqueueing fibers that are ready to return.

  # Blocks the current fiber for `time`.
  def self.sleep(time : Time::Span) : Nil
  end

  # Reads at least one byte from a file or pipe
  def self.read(fd, slice : Bytes) : Int32
  end

  # Writes at least one byte to a file or pipe
  def self.write(fd, slice : Bytes) : Int32
  end

  # Reads at least one byte from a socket
  def self.receive(fd, slice : Bytes) : Int32
  end

  # Reads at least one byte from a socket, obtaining the source address of the packet (UDP)
  def self.receive_from(fd, slice : Bytes) : {Int32, ::Socket::Address}
  end

  # Writes at least one byte to a socket
  def self.send(fd, slice : Bytes) : Int32
  end

  # Writes at least one byte to a socket with a target address (UDP)
  def self.send_to(fd, slice : Bytes, address : ::Socket::Address) : Int32
  end

  # Accepts a incomming TCP connection
  def self.accept(fd) : Int32
  end

  # Opens a connection
  def self.connect(fd, address : ::Socket::Address) : Int32
  end

  # Closes a file
  def self.close(fd) : Int32
  end
  
  # Synchronizes data and metadata of a file to persistent storage
  def self.fsync(fd) : Int32
  end

  # Synchronizes data of a file to persistent storage
  def self.fdatasync(fd) : Int32
  end

  # Waits until a file is ready for a non-blocking read operation
  def self.wait_readable(fd) : Nil
  end
  
  # Waits until a file is ready for a non-blocking write operation
  def self.wait_writable(fd) : Nil
  end
end

All of these can be implemented for Unix either by blocking or by using wait_readable/wait_writable and calling into LibC when ready. This interface allows other async implementations to exist without changes to files outside Crystal::System.

I believe this makes the Windows implementation cleaner as well. What do you think @straight-shoota?

[straight-shoota]

Are you suggesting to move IO operations directly into the event loop? Not sure that's a wise idea. I think they should stay confined to the respective IO implementations because this tends to be very application-specific. There are differences between IO operations on a socket vs. on a file, even if they technically use the same fd interface and functions.

For the win32 port I think I have this worked out quite well. The IOCP variant of the event loop build the basis, and I introduced an IO::Overlapped module similar to IO::Evented which handles and abstracts the overlapped IO for the socket implementation, and integrates with the event loop.
There is some overlap with IO::Evented, though, and I believe some of the interfaces can be more refined.

I understand the integration of io_uring is going to be a bit more complex, because we essentially need to support it together with libevent in a single program. With libevent and iocp this is not an issue, because they're confined to POSIX and win32 platforms, respectively.

[yxhuvud]

# Runs the event loop.
  def self.run_once : Nil
  end

Eh, I would consider run_once to be just an implementation detail of libevent. What would be needed is a loop specific implementation of Scheduler#reschedule. It really is not advisable to view the event loop separately from the scheduler - especially not in the case of io_uring where the same syscall is used for both submitting and fetching events, and you can do both in the same call. (EDIT to clarify: there are certainly some parts of reschedule that still belongs in Scheduler).

I think they should stay confined to the respective IO implementations because this tends to be very application-specific.

Problem is that they simultaneously also end up very event-loop specific. Always mixing everything into the different IO classes is not super great. It could be that the nicest structure available is to have a second object in between what we now call the event loop and have all interactions with the event loop in that.

EDIT: BTW, the refactorings already done for sockets related to the windows port is a good step in the right direction here, it is really nice to have everything collected in the same place, even if it is in a module and thus not so self contained as it could be.

[straight-shoota]

Yeah, the typical system-specific implementations are based on modules, because there has only ever been a single viable strategy for any target triple. But when we need to select between libevent and io_uring at runtime, that no longer works. We need a different strategy then.

I suppose it shouldn't be a big change to go from system_* methods in a module to a dedicated type that encapsulates all the IO operations and event loop interaction. The API wouldn't need to change much for that.
This isn't necessarily the only option, but it should be pretty doable.

[straight-shoota]

For reference, this is the relevant event loop API in my IOCP port:

module Crystal::EventLoop
  # Runs the event loop.
  def self.run_once : Nil
  end

  # Reinitializes the event loop after a fork.
  def self.after_fork : Nil
  end

  def self.enqueue(event : Event)
  end

  def self.dequeue(event : Event)
  end

  # Create a new resume event for a fiber.
  def self.create_resume_event(fiber : Fiber) : Crystal::Event
  end
end

struct Crystal::Event
  getter fiber
  getter wake_at

  def initialize(@fiber : Fiber, @wake_at = Time.monotonic)
  end

  # Frees the event
  def free : Nil
  end

  def add(time_span : Time::Span) : Nil
  end
end

[ysbaddaden]

I'm growing fond of the initial proposed idea by @lbguilherme where we treat the event loop as some a blackbox.

This is close to how we treat the GC: allocate N bytes and return a pointer, I don't care how you do it. We can switch the implementation and everything's working as long as the interface is implemented, without leaking any internal details.

It would be great if we could do the same for non-blocking IO operations!

I also like your Event + EventLoop abstraction @straight-shoota. It's simple and concise, and having an actual Event object is likely a good idea!

The problem is that it's leaking implementation details from the EventLoop into the IO objects themselves, so depending on the target-triple we either have IO::Overlapped or IO::Evented that each assume a specific mechanism for the event-loop, which may not stand: on a quick glance io_uring seem closer to kqueue and iocp than epoll for eample.

[yxhuvud]

It would be great if we could do the same for non-blocking IO operations!

One problem is that the set of non blocking operations are not the same everywhere. Meaning the interface would need to either be so pessimal that it may as well not exist or contain almost everything under the sun. There are also some operations that is simply impossible or really hard to implement on certain platforms, like wait_readable or wait_writable on windows. Meanwhile they sortof have to exist on linux in some way as they are the ideomatic way to do a lot of interaction with the operating system and close-to-operating system level stuff.

Agreed on the Event being quite implementation dependent, it is definitely possible to build an event loop that have no reason at all to keep track of that because it is gotten for free by the general functionality of everything, but instead have to keep track of for example received events until they are processed ( example, half-functioning together with nested_scheduler shard. Fiber is monkey patched to keep the received event around).

[ysbaddaden]

Would it be so bad to have N implementations? If there are repetitions could they not be abstracted behind the event loop / nonblock operations, instead of leaking their details?

Replying to myself: let's learn about kqueue and IOCP in addition to epoll 📚