Hi @blaze_kick,
Welcome aboard and congratulations on releasing your Aseprite extension! 
Regarding any tips, I’m not in the least an expert on performance, so take all advice to follow with a grain of salt.
I’d say start by having a look at this post:
As far as I know, color matching can be sped up with a space partitioning strategy. I’m aware of octrees, I believe people use k-d trees as well, and there may be yet others.
I also suggest finding the unique colors in an image, then matching the palette to uniques, rather than looping over all the pixels. Unique colors can be found by using a boolean as a dummy value for each key, e.g., dictionary[rgbaInteger] = true. Palette matches can be assigned to colors from the image in a similar manner, e.g., dictionary[uniqueImageColor] = paletteMatchInteger, then pixelIterator(dictionary[pixelIterator()]). In the examples, by integer I’m referring to the 32-bit integers that store the color channels, each in [0, 255], in the order 0xAABBGGRR.
Perhaps not as important: if you use Euclidean distance, try seeing if the distance-squared gives you the same results. It might spare you some math.sqrt calls. Also, if I remember correctly, you don’t need to find the math.abs of the delta when the exponent is even, i.e. is 2. For example, check that with local deltasq = delta * delta. You may want to retain the abs, however, if you want to generalize to some other distance metric later. To see what I mean, compare the formulas for Manhattan, Minkowski, Chebyshev and Euclidean distance.
(The distance metric for CIE LAB is a whole other rabbit hole, and I prefer to simplify by choosing a variant LAB space instead of tackling it directly.)
The go-to doc for general Lua performance tips is a pdf by Roberto Ierusalimschy: https://www.lua.org/gems/sample.pdf . It includes advice such as localizing global functions that are used in loops, e.g., local sqrt = math.sqrt and simplifying tables, e.g., rgbaTable[1 + i * channelCount + rChannelOffset] = red instead of rgbaTable[1 + i] = { r = red, g = green, b = blue }.
Measure elapsed time for your script. I just use os.clock, then subtract the start time from the end. Are there better ways? Probably.
Lastly, I guess others were looking into / found a way to write in another programming language, with the Lua handling primarily the UI? Not sure though. See Calling a C function from Lua inside Aseprite? .
Hope that helps some,
Jeremy