Skip to content

Fix /languages timing out + make it cache #10747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 12, 2025

Conversation

cdrini
Copy link
Collaborator

@cdrini cdrini commented May 9, 2025

Fix /languages page timing out, and make it more performant by caching the results to avoid frequent solr hits of this expensive query.

Technical

Testing

Before: https://openlibrary.org/languages (times out, re-fetched every time)

After: https://testing.openlibrary.org/languages

Also not still works with i18n: https://testing.openlibrary.org/languages?lang=es

Screenshot

Stakeholders

@cdrini cdrini marked this pull request as ready for review May 9, 2025 17:20
@Copilot Copilot AI review requested due to automatic review settings May 9, 2025 17:20
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request addresses timeouts observed on the /languages endpoint by introducing a configurable timeout parameter for Solr queries and adding caching to improve performance.

  • Added a _timeout parameter in solr.py to allow customizable query timeouts
  • Introduced caching for language count queries in languages.py using a memoization decorator with a one-hour expiration

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
openlibrary/utils/solr.py Added _timeout parameter and replaced literal timeout values with variable reference
openlibrary/plugins/worksearch/languages.py Updated get_top_languages for type hints and implemented caching in get_all_top_language_counts
Comments suppressed due to low confidence (1)

openlibrary/plugins/worksearch/languages.py:51

  • [nitpick] The cache key 'get_top_language_counts' does not correspond with the function name 'get_all_top_language_counts'; consider renaming the key for better clarity and consistency.
@cache.memoize("memcache", key='get_top_language_counts', expires=60 * 60)

if _timeout is not None:
timeout = _timeout
else:
timeout = 10
Copy link
Preview

Copilot AI May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider defining a shared constant for the default timeout (10 seconds) rather than hardcoding the value in multiple places, which can improve maintainability.

Suggested change
timeout = 10
timeout = DEFAULT_TIMEOUT

Copilot uses AI. Check for mistakes.

@cdrini cdrini force-pushed the fix/languages-page branch from 05e22d6 to f8b0b43 Compare May 9, 2025 17:24
@cdrini cdrini force-pushed the fix/languages-page branch from f8b0b43 to e26f8e7 Compare May 10, 2025 09:05
@mekarpeles mekarpeles merged commit aa43c83 into internetarchive:master May 12, 2025
4 checks passed
@cdrini cdrini deleted the fix/languages-page branch May 12, 2025 17:51
@cdrini cdrini mentioned this pull request May 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants