Skip to content

Stats ValueError: invalid literal for int() with base 10: b'1739222295.11384' #303

Open
@lexxai

Description

@lexxai

Problem:

  File ".../lib/python3.11/site-packages/pydispatch/robustapply.py", line 55, in robustApply
    return receiver(*arguments, **named)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/scrapy/extensions/logstats.py", line 78, in spider_closed
    rpm_final, ipm_final = self.calculate_final_stats(spider)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/scrapy/extensions/logstats.py", line 85, in calculate_final_stats
    start_time = self.stats.get_value("start_time")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/scrapy_redis/stats.py", line 42, in get_value
    return int(self.server.hget(self._get_key(spider), key))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: b'1739222295.11384'

Solution:

>>> a=b'1739222295.11384'
>>> int(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: b'1739222295.11384'
>>> float(a) 
1739222295.11384
>>> int(float(a))
1739222295

Changes:

    def get_value(self, key, default=None, spider=None):
        """Return the value of hash stats"""
        if self.server.hexists(self._get_key(spider), key):
            return int(float(self.server.hget(self._get_key(spider), key)))
        else:
            return default

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions