Skip to content

Add support for Magic Kernel Sharp resizing modes #8811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

drhead
Copy link

@drhead drhead commented Mar 11, 2025

Original formula is by John Costella, link includes paper with explanation of why it works better than Lanczos and demonstrations of its performance: https://johncostella.com/magic/

This is a simple implementation of the Magic Kernel Sharp 2021 resize filter. It should produce higher quality results for resampling, either upsampling or downsampling, than Lanczos.

This PR is incomplete, I still need to test it against the reference implementation of MKS, write tests for it, update documentation samples, and run linting. The outputs that I have gotten from preliminary testing overall look as expected and look better than Lanczos in the expected ways (reduced moiré patterning on downsampling, reduced blockiness on upscaling), so it should be in a state where it is suitable for experimentation if wanted.

@drhead
Copy link
Author

drhead commented Mar 12, 2025

A notable discrepancy:
image
("pillow (linear)" is applying the resize after a manual conversion to linear sRGB space, "reference (sRGB)" is the reference resizer run with --no-gamma-handling.)

The correct behavior would be to convert images to a linear color space before resizing. If the situation has changed at all since #1604 was filed it may be worth revisiting, but in either case it's out of scope for this. As you can see, it matches the reference implementation without gamma handling by default, and when applied in linear sRGB space it matches the reference with gamma handling, both within reasonable quantization error.

Just for reference, this is the amount of difference this produces on a test image:
image

@drhead
Copy link
Author

drhead commented Mar 12, 2025

Another discrepancy I am noticing is a number of spots with relatively large differences along the edges of images, this might indicate that there's a problem with conv padding. Comparisons of Lanczos as implemented in MKS's reference code produces some artifacts like this too, but the artifacts are lower magnitude. This is out of scope for this PR but it needs to be checked on.

After some more experimentation (in the form of doing a separate basic implementation of the whole resizing process), I've determined that this is most likely just due to implementation differences. MKS's reference code does the resize as three separate kernels, but here they are composed into a single kernel, and that causes boundary differences. My understanding is that neither is necessarily more visually correct than the other, and despite some of the pixel differences being fairly high magnitude they are still sub perceptual. edit 3/18/25: Further empirical testing has shown me that single kernel implementation is generally more accurate when it comes to handling edges. With that, I am completely satisfied with the correctness of the implementation itself.

(if you're having trouble seeing it, they're especially prominent on the top of the image)
image

@drhead drhead changed the title Add support for MKS2021 (Magic Kernel Sharp) resizing mode Add support for Magic Kernel Sharp resizing modes Apr 4, 2025
@drhead
Copy link
Author

drhead commented Apr 4, 2025

I have updated this to include Magic Kernel Sharp 2013 (at the suggestion of the creator -- it should have the same effects in combatting moire/blockiness while being slightly faster than Lanczos 3) as well as the 2021 implementation.

I have also added these kernels to documentation at appropriate points and added them to tests. The additions to concepts.rst may need revision. I have also not added the test_reduce_x and test_enlarge_x tests because I am not completely certain of how those tests need to be properly set up and would want some guidance on that first -- as far as I am aware the current implementation right now is correct and could be trusted to generate testing code for this. The only other point in the code that I think might need some advisement is whether MKS2021 should replace LANCZOS at src/PIL/IcoImagePlugin.py:77. In any case, I'm opening the issue for review.

@drhead drhead marked this pull request as ready for review April 4, 2025 20:24
radarhere and others added 2 commits April 5, 2025 14:53
@radarhere radarhere requested a review from homm April 14, 2025 08:53
@drhead drhead requested a review from radarhere May 8, 2025 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants