Skip to content

GitHub: use_commit_name option #213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
cache-name: cache-pip
with:
path: ~/.cache/pip
key: ${{ runner.os }}-${{ env.cache-name }}-${{ matrix.deps }}-${{ hashFiles('setup.py') }}
key: ${{ runner.os }}-${{ env.cache-name }}-${{ matrix.deps }}-${{ hashFiles('pyproject.toml', 'setup.cfg') }}
restore-keys: |
${{ runner.os }}-${{ env.cache-name }}-${{ matrix.deps }}-
${{ runner.os }}-${{ env.cache-name }}-
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
*.egg-info/
__pycache__/
/build/
/dist/
.cache/
.eggs/
*.pyc
*.pyo
.travis.pub
.pytest_cache/
.tox/
keyfile.toml
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ To install::

To use the latest code, you can also clone this repository and run::

python3 setup.py install
pip install .

To see available options::

Expand Down
22 changes: 18 additions & 4 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,8 @@ httptoken
A personal authorization token used to fetch the url with the ``Authorization`` header.
The type of token depends on the authorization required.

- For Bearer token set : ``Bearer <Your_bearer_token>``
- For Basic token set : ``Basic <Your_base64_encoded_token>``
- For Bearer token set \: ``Bearer <Your_bearer_token>``
- For Basic token set \: ``Basic <Your_base64_encoded_token>``

In the keyfile add ``httptoken_{name}`` token.

Expand Down Expand Up @@ -365,8 +365,8 @@ Check GitHub
source = "github"

Check `GitHub <https://github.com/>`_ for updates. The version returned is in
date format ``%Y%m%d.%H%M%S``, e.g. ``20130701.012212``, unless ``use_latest_release``
or ``use_max_tag`` is used. See below.
date format ``%Y%m%d.%H%M%S``, e.g. ``20130701.012212``, unless ``use_latest_release``,
``use_max_tag``, or ``use_commit_name`` is used. See below.

github
The github repository, with author, e.g. ``lilydjwg/nvchecker``.
Expand All @@ -393,6 +393,12 @@ use_latest_tag

This requires a token because it's using the v4 GraphQL API.

use_commit_name
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about use_commit_hash? It's not obvious what a commit name is.

Set this to ``true`` to append a plus and the commit name to the version, e.g.
``20130701.012212+e1457aadd30f53f4d50d6c4828d517355c09b8ae``.

If this isn't showing up, provide a token so it can use the v4 GraphQL API.

query
When ``use_latest_tag`` is ``true``, this sets a query for the tag. The exact
matching method is not documented by GitHub.
Expand All @@ -403,6 +409,12 @@ use_max_tag
lightweight ones, and return the largest one sorted by the
``sort_version_key`` option. Will return the tag name instead of date.

This defaults ``list_count`` to 100.

list_count
When supporting :ref:`list options` through the v4 GraphQL API, this sets a
maximum count of items in the list. By default, ``list_count`` is set to 1.

token
A personal authorization token used to call the API.

Expand All @@ -415,6 +427,8 @@ To set an authorization token, you can set:
- the token option

This source supports :ref:`list options` when ``use_max_tag`` is set.
Options of this source that support :ref:`list options` may be effected by
``list_count``.

Check Gitea
~~~~~~~~~~~
Expand Down
26 changes: 19 additions & 7 deletions nvchecker/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,14 @@ async def run(self) -> None:
'''Run the `tasks`. Subclasses should implement this method.'''
raise NotImplementedError

def _normalize(x: Any) -> Any:
if isinstance(x, list):
return tuple(sorted(_normalize(y) for y in x))
elif isinstance(x, dict):
return tuple(sorted((_normalize(k), _normalize(v)) for k, v in x.items()))
else:
return x
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not obvious what this function does. It needs a docstring or a better name.


class AsyncCache:
'''A cache for use with async functions.'''
cache: Dict[Hashable, Any]
Expand All @@ -156,28 +164,32 @@ def __init__(self) -> None:
self.lock = asyncio.Lock()

async def _get_json(
self, key: Tuple[str, str, Tuple[Tuple[str, str], ...]],
self, key: Tuple[str, str, Tuple[Tuple[str, str], ...], object], extra: Any,
) -> Any:
_, url, headers = key
res = await session.get(url, headers=dict(headers))
_, url, headers, json = key
json = extra # denormalizing json would be a pain, so we sneak it through
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about passing json as a str or bytes? We can set the Content-Type header and directly pass it to session as body

res = await (session.get(url=url, headers=dict(headers)) if json is None \
else session.post(url=url, headers=dict(headers), json=json))
return res.json()

async def get_json(
self, url: str, *,
headers: Dict[str, str] = {},
json: Optional[object] = None,
) -> Any:
'''Get specified ``url`` and return the response content as JSON.

The returned data will be cached for reuse.
'''
key = '_jsonurl', url, tuple(sorted(headers.items()))
key = '_jsonurl', url, _normalize(headers), _normalize(json)
return await self.get(
key , self._get_json) # type: ignore
key, self._get_json, extra=json) # type: ignore

async def get(
self,
key: Hashable,
func: Callable[[Hashable], Coroutine[Any, Any, Any]],
func: Callable[[Hashable, Optional[Any]], Coroutine[Any, Any, Any]],
extra: Optional[Any] = None,
) -> Any:
'''Run async ``func`` and cache its return value by ``key``.

Expand All @@ -189,7 +201,7 @@ async def get(
async with self.lock:
cached = self.cache.get(key)
if cached is None:
coro = func(key)
coro = func(key, extra)
fu = asyncio.create_task(coro)
self.cache[key] = fu

Expand Down
Loading