Skip to content

Asynchronous start method compatible with new Scrapy versions #305

Open
@lexxai

Description

@lexxai
import asyncio

class RedisMixin:   

...
     async def start(self):
        """Asynchronous start method compatible with new Scrapy versions."""
        for req in await asyncio.to_thread(self._collect_start_requests):
            yield req

    def _collect_start_requests(self):
        """Collect start requests synchronously from next_requests()."""
        return list(self.next_requests())  # collect all yielded requests

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions