Skip to content

Lua palette quantization #802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

warmCabin
Copy link

Don't mind me, just using your appveyor to avoid setting up Visual Studio on my new PC :)

Just want to see what happens. If the performance sucks, I'll try it with 5
or 6 bits. If the difference is negligible, I'll remove it properly.
@warmCabin
Copy link
Author

warmCabin commented Aug 13, 2025

Definitely seeing some improvements by switching away from the green channel rods n' cones biasing thing. When I completely turn off the color caching and run this test script, there is a noticeable performance hit when turboing. I'd say it goes from 25x to 10x speed. As such, I changed the color cache to 5 bits per channel and it seems fine... But I'm still a bit leery, so I might make a Lua API that can turn this on and off just in case 5 bits still makes things weird.

Here are some comparison results. Hopefully it's not too hard to see. With the green bias, the ghosts get kind of yellowy when they stack up.

3-bit green bias (the ways things are in master)

3bit-green_bias.mp4

no cache euclid (best results, noticeable performance dip when turboing)

no_cache-euclid.mp4

5-bit Euclid

5bit-euclid.mp4

3 bit euclid

3bit-euclid.mp4

no cache green bias

no_cache-green_bias.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant