[BUG] ADDON service.libreelec.settings is messing with python asyncio event loop.

[x-stride]

Describe the bug

ADDON service.libreelec.settings is messing with python asyncio event loop.

To Reproduce

1. Create an ADDON creating or requesting an asyncio event loop, like service.libreelec.settings
2. Start it...
3. Asyncio is now broken

Informations

• LE Version: 10.0.2
• Hardware Platform: x86-64

Log file

2022-04-27 12:28:03.992 T:1187    DEBUG <general>: CPythonInvoker(1, /storage/.kodi/addons/service.aiohttp.service/service.py): setting the Python path to /storage/.kodi/addons/service.aiohttp.service:/storage/.kodi/addons/script.module.aiohttp/lib:/storage/.kodi/addons/script.module.aiosignal/lib:/storage/.kodi/addons/script.module.async_timeout/lib:/storage/.kodi/addons/script.module.charset_normalizer/lib:/storage/.kodi/addons/script.module.frozenlist/lib:/storage/.kodi/addons/script.module.idna/lib:/storage/.kodi/addons/script.module.multidict/lib:/storage/.kodi/addons/script.module.typing_extensions/lib:/storage/.kodi/addons/script.module.yarl/lib:/usr/lib/python38.zip:/usr/lib/python3.8:/usr/lib/python3.8/lib-dynload:/usr/lib/python3.8/site-packages
2022-04-27 12:28:03.992 T:1187    DEBUG <general>: CPythonInvoker(1, /storage/.kodi/addons/service.aiohttp.service/service.py): entering source directory /storage/.kodi/addons/service.aiohttp.service
2022-04-27 12:28:03.992 T:1187    DEBUG <general>: CPythonInvoker(1, /storage/.kodi/addons/service.aiohttp.service/service.py): instantiating addon using automatically obtained id of "service.aiohttp.service" dependent on version 3.0.0 of the xbmc.python api
2022-04-27 12:28:04.078 T:1188    ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                                    - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                                   Error Type: <class 'RuntimeError'>
                                                   Error Contents: There is no current event loop in thread 'Dummy-1'.
                                                   Traceback (most recent call last):
                                                     File "/storage/.kodi/addons/service.libreelec.settings/service.py", line 7, in <module>
                                                       import dbus_utils
                                                     File "/usr/share/kodi/addons/service.libreelec.settings/resources/lib/dbus_utils.py", line 127, in <module>
                                                     File "/usr/lib/python3.8/asyncio/events.py", line 639, in get_event_loop
                                                   RuntimeError: There is no current event loop in thread 'Dummy-1'.
                                                   -->End of Python script error report<--
 

A similar offending ADDON

service.asyncio.loop, which will claim and start it:

import asyncio
import xbmc

class Monitor(xbmc.Monitor):
    def run(self):
        loop = asyncio.get_event_loop()
        while not self.abortRequested():
            if self.waitForAbort(60):
                break

if __name__ == '__main__':
    Monitor().run()

What I really wanted to do :)

service.aiohttp.server, will get never get any loop :(

import asyncio
import xbmc

async def main():
    await asyncio.sleep(120)

class Monitor(xbmc.Monitor):
    def run(self):
        # It's a race... First addon started requesting it will get it, others will fail.
        loop = asyncio.get_running_loop()
        loop.create_task(main())
        while not self.abortRequested():
            if self.waitForAbort(60):
                break

if __name__ == '__main__':
    Monitor().run()

And it becomes all messy as the event-loop already stolen here:
https://github.com/LibreELEC/service.libreelec.settings/blob/master/resources/lib/dbus_utils.py#L127

Possible solution:

• I guess Kodi's main process can hold one single loop created the right way, or start it itself
• ADDON's must do get_running_loop or fork off and do whatever?
• asyncio is amazing, when it works :)

[x-stride]

Made some addons for testing over in a repo

[romanvm]

According to docs get_running_loop can only be run from within a coroutine.
The following example will fail even if run without Kodi:

import asyncio


async def sleeper():
    await asyncio.sleep(60)
    print('#### tester 1 sleep finished')


loop = asyncio.get_running_loop()
loop.create_task(sleeper)

[romanvm]

On the other side, there is indeed a conflict if several addons try to run async tasks in an event loop. However, this is a general Kodi problem not specific to LibreELEC. I guess the reason is the way how addons are run using sub-interpreters and probably can be fixed only on Python level.

[romanvm]

I have confirmed that this is a Python problem related to the use of sub-interpreters. The following code produces the same error outside Kodi and LE:

#include <iostream>
#include <thread>
#include <string>

#include "Python.h"


const std::string pyCode = R"""(
import asyncio
import threading

async def coro():
    await asyncio.sleep(3)
    print(f'Sleep in thread {threading.get_ident()} done')

asyncio.run(coro())
)""";


void runPython() {
  PyThreadState* ts = PyThreadState_New(PyInterpreterState_Main());
  PyEval_RestoreThread(ts);

  PyThreadState* interpTs = Py_NewInterpreter();

  PyRun_SimpleString(pyCode.c_str());
  PyObject* error = PyErr_Occurred();
  if (error) {
   PyErr_PrintEx(0);
  }

  Py_EndInterpreter(interpTs);

  PyThreadState_Swap(ts);
  PyEval_ReleaseThread(ts);
}


int main(int argc, char *argv[]) {
  Py_Initialize();
  Py_BEGIN_ALLOW_THREADS

  std::thread thread1(runPython);
  std::thread thread2(runPython);
  thread1.join();
  thread2.join();

  Py_END_ALLOW_THREADS
  Py_Finalize();

  std::cout << "All done" << std::endl;

  return 0;
}

[romanvm]

As it turned out, it is known Python issue python/cpython#91375 that has no fix so far. The only possible workaround is to use a pure-Python implementation of asyncio event loop by disabling its C-based implementation. It can be done adding the following code at the top of your addon entrypoint:

import sys
sys.modules['_asyncio'] = None

[heitbaum]

Closing as completed - noting upstream closure. If still an issue please reopen or reference this issue.