Open
Description
Problem:
File ".../lib/python3.11/site-packages/pydispatch/robustapply.py", line 55, in robustApply
return receiver(*arguments, **named)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/scrapy/extensions/logstats.py", line 78, in spider_closed
rpm_final, ipm_final = self.calculate_final_stats(spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/scrapy/extensions/logstats.py", line 85, in calculate_final_stats
start_time = self.stats.get_value("start_time")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/scrapy_redis/stats.py", line 42, in get_value
return int(self.server.hget(self._get_key(spider), key))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: b'1739222295.11384'
Solution:
>>> a=b'1739222295.11384'
>>> int(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: b'1739222295.11384'
>>> float(a)
1739222295.11384
>>> int(float(a))
1739222295
Changes:
def get_value(self, key, default=None, spider=None):
"""Return the value of hash stats"""
if self.server.hexists(self._get_key(spider), key):
return int(float(self.server.hget(self._get_key(spider), key)))
else:
return default
Metadata
Metadata
Assignees
Labels
No labels