Converting to index color gives incorrect results

Bug Reports
12

Hi @ash3s,

If possible, please post a screen capture and say whether you’re using Octree or Table RGB 5 bits to convert.

For some posters on this thread, the issue is complicated by the fact that images created in Photoshop may have a different color profile than standard RGB. Go to Sprite > Properties to check. If the profile does not say sRGB, test if Converting to SRGB before creating a palette makes a difference.

If you want to try a script that creates a palette, see if this gives you different results:

-- Change these settings to preference.
local removeAlpha = false
local prependMask = true
local clampTo256 = true

local activeSprite = app.activeSprite
if not activeSprite then return end

-- Optional. Remove double hyphen to re-enable.
-- activeSprite:convertColorSpace(ColorSpace { sRGB = true })

local activeSpec = activeSprite.spec
if activeSpec.colorMode ~= ColorMode.RGB then
    return
end

local alphaMask = 0x0
if removeAlpha then
    alphaMask = 0xff000000
end

local dictionary = {}
local idx = 1

for _, activeFrame in ipairs(activeSprite.frames) do
    local flatImage = Image(activeSpec)
    flatImage:drawSprite(
        activeSprite,
        activeFrame,
        Point(0, 0))
    local itr = flatImage:pixels()

    for elm in itr do
        local hex = elm()
        if ((hex >> 0x18) & 0xff) > 0 then
            hex = alphaMask | hex
            if not dictionary[hex] then
                dictionary[hex] = idx
                idx = idx + 1
            end
        end
    end
end

local hexes = {}
for k, v in pairs(dictionary) do
    hexes[v] = k
end

if prependMask then
    local maskIdx = dictionary[0x0]
    if maskIdx then
        if maskIdx > 1 then
            table.remove(hexes, maskIdx)
            table.insert(hexes, 1, 0x0)
        end
    else
        table.insert(hexes, 1, 0x0)
    end
end

local hexesLen = #hexes
if hexesLen > 0 then
    local palLen = hexesLen
    if clampTo256 then
        palLen = math.min(256, hexesLen)
    end
    local palette = Palette(palLen)
    for i = 1, palLen, 1 do
        local hex = hexes[i]
        local aseColor = Color(
                      hex & 0xff,
            (hex >> 0x08) & 0xff,
            (hex >> 0x10) & 0xff,
            (hex >> 0x18) & 0xff)
        palette:setColor(i - 1, aseColor)
    end
    activeSprite:setPalette(palette)
end

app.refresh()

I don’t recommend using this script with a sprite that could generate a large palette (i.e., greater than 256). It could be slow and lock up Aseprite until it’s finished. It is not designed to work with any color mode other than RGB.

See also: Octree Color Indexed Conversion Testing ,
New Script for perfect palette generation from RGB sprite .

Cheers,
Jeremy