Remove/clean jaggy function

Pretty self explanatory. I don’t really like using the pixel perfect option for the brush, yet there are still situations where you need to clean something after the fact. It would be nice if there was a function that did the same thing to already existing lines. It wouldn’t do a perfect job since it can’t know the best artistic choice of pixel to remove, but it would serve as a good starting point for further cleaning.

i wonder if there’s some algorithm for this already. i didn’t find anything yet, but i’m sure this could be done with script. i made a quick hand-made test:
_temp-cleanup2 _temp-cleanup2-source
and it seems like it could work rather fine.
the rules are simple:

  • check colour of pixel on position n. if its colour is selected colour c, then:
  • check position n+1 in specified direction
  • if there’s no colour c on that position, do nothing
  • if there is c, move to n+1 and check all other +1 positions except the position n - if there’s c present on any of these positions[*], delete n+1 pixel.
  • move to n+1 and repeat

in this example there are only two problematic cases:
– ‘down’ direction: erasing one pixel at the right end of the line (that may or may not be what user wanted)
– ‘left’ direction: leaving out one pixel in the same area instead of erasing it. this issue can be solved by going through the process at least twice in both horizontal and vertical directions. or possibly all of them - as seen on examples, the order of selected directions would matter.

this would of course work only on layers with just lines and transparency.
if the line is surrounded by other colours then the process would be more complex, as it would have to check other areas for colours and then change the colour of a pixel instead of deleting it.
also when applied to filled areas instead of lines, it could give really bad results.

i would try to write this script, but i am too busy this month.
meanwhile someone else may find it interesting enough to write it?

edit: [*] this is actually incorrect.

I assume it will be better to analyze normal vector (or just angle) at each selected color pixel as distance-weighted average of perpendicular vectors (or angles) of surrouning pixels of same color (only connected i.e. use spread fill to find them), then (one at a time) find and remove most off-line “jaggy” pixels (compare by sum of distance-weighted distances to angled line of each of surrounding pixels) until there will be no pixels with “jaggy” neightbours.

you might be right, but i don’t know anything about vectors. wouldn’t it be more time consuming though? the scripts aren’t very fast.

Not really. Most time consuming will be pixels scanning process to collect them. After that we’ll have array of all pixels with information about them (averaged angle, number of connections around) to find pixels to remove in continous passes.

We also can act in Selection only to avoid touching too much and restricting what could be modified.

Also maybe use simpler approach and just always find and compare next two horizontal or vertical pixels which form two side connectivity (with one or two pixels at each side). And then decide which one to remove by comparing distances averaged angles lines of surrounding pixels and pick one which is most off-line.

i see. would you give it a try?

I may implement the algorithm, but without user interface, color picking and filtering by Selection. My implementation will always look for specific color and at current active layer and frame. Would you add those features after that if the algorithm will work?

3 Likes

sure thing!

3 Likes

I know nothing about scripting this sort of thing but,
I think you’re right that doing it to the selected color is the best bet. Would it be possible to select everything of that color on the layer (like using wand), cut it, put it on it’s own layer with transparency, do the effect, and then paste/merge the contents back to the original layer? Internally as part of the process, I mean.

It may also make sense to add an option that adds the cleaned lines to a new layer and keeps it separate, so that you can choose what to do with it.

Though in terms of practical use, I can also imagine it being useful if you can lasso something and hit a hotkey without a window even popping up, fixing only that spot.

In general it would be nice if you could save a filter’s settings to a hotkey, so that you can do things like “apply outline” with presets, without the window ever even coming up. Though I guess that’s a seperate feature in and of itself.

well, i’m not a programmer, so… yeah :]]
however, the gui-less version which would take the current foreground colour as input is no problem.
i’ve never used it before, but it seems like we can pack the script as a plugin, which should be able to store preferences. one can add hotkeys to both scripts and plugins, so thats fine.
and i’d say lifting the lines to another layer should be doable as well.
but let’s not get ahead of ourselves. first we need basic functionality and then we’ll see what to do next with it.

First working version (eg_jagger_v1.zip): eg_jagger_v1.zip - Google Drive

It looks at current sprite current cel and searches for color cc(0,0,0,255) and replaces it with color ct(255,0,0,255) (red color for easy testing) at jaggy places.

You may add panel with colors selection or make a gui-less version that uses foreground and background colors.

3 Likes

you’re super fast! thanks

no ui - uses foreground and background colours for detection and replacement: eg_jagger - no ui - Pastebin.com

simple ui - two colours, one button: eg_jagger - simple ui - Pastebin.com

no ui on selection: eg_jagger - no ui - selection - Pastebin.com
note: this version works correctly only on rectangular selections. unsurprisingly, selection made with lasso tool is treated as rectangle.

3 Likes

Now let’s wait for Kyrieru to evaluate our efforts =)

1 Like

i still plan to tweak selection version when i get a bit more time. i think the content outside of selection can be filtered out through selection:contains(point)
but other than that it seems to work just great. i didn’t think i’d need something like this until now :]

2 Likes

Quick pixel sketching made fun :]

1 Like

Thanks guys, I’ll take a look.

I think that there’s times when you would want it to auto handle layers and deleting of the jaggy pixels (aka select, hit the hotkey, and they’re just gone), whereas the current behavior of setting the pixels to red is useful in other situations.

selection version now affects only pixels inside selection (not just rectangle):


-- Enginya's Jagger

local function Engy_Jagger()
  local spr = app.activeSprite;
  if not spr then
    app.alert "No active sprite!"
    return
  end
  
  --if spr.colorMode ~= ColorMode.RGB then
  --   app.alert "Not RGB mode!"
  --   return
  --end
  
  
  local sprite = app.activeSprite
  local selection = sprite.selection 
  
  local sw = spr.width
  local sh = spr.height
  
  local pc = app.pixelColor
  
  local fg = app.fgColor
  local bg = app.bgColor
  
  local cel = app.activeCel
  if cel == nil then
    app.alert "No image!"
    return
  end
  
  local img = cel.image:clone()

	
  local rx1 = selection.bounds.x --cel.bounds.x
  local rx2 = rx1 + selection.bounds.width - 1 --rx1 + cel.bounds.width - 1
  local ry1 = selection.bounds.y --cel.bounds.y
  local ry2 = ry1 + selection.bounds.height - 1 --ry1 + cel.bounds.height - 1
  
  local cc = fg.rgbaPixel --pc.rgba(0,0,0,255)
  local ct = bg --pc.rgba(255,0,0,0)
  --local ct = pc.rgba(0,0,0,0)

  function ff(xx,yy) return img:getPixel(xx,yy) == cc end
  function nn(xx,yy) return img:getPixel(xx,yy) ~= cc end
  
  
  function check(xx,yy) return selection:contains(xx,yy) end 
  
  local gx = {}
  gx[0] = -1
  gx[1] =  1
  gx[2] =  1
  gx[3] = -1
  gx[4] = -1
  gx[5] =  1
  gx[6] =  0
  gx[7] =  0

  local gy = {}
  gy[0] = -1
  gy[1] = -1
  gy[2] =  1
  gy[3] =  1
  gy[4] =  0
  gy[5] =  0
  gy[6] = -1
  gy[7] =  1
  
  --function gg(xx,yy,ii) return ff(xx+gx[ii],yy+gy[i]) end
  
  function clr(tab) for k,v in pairs(tab) do tab[k]=nil end end

  local pts = {}
  local was = {}
  local WW = sw*2
  
  function coll(xx,yy,kk)
    table.insert(pts,{x=xx,y=yy})
    if kk >= 3 then return end
    was[xx+yy*WW] = 1
    for ii=0,7 do
      tx=xx+gx[ii]
      ty=yy+gy[ii]
      if ff(tx,ty) and was[tx+ty*WW] ~= 1 then coll(tx,ty,kk+1) end
    end
  end
  
  function avga(xx,yy)
    mx = 0
    my = 0
    mn = 0
    
    for k,v in pairs(pts) do
      --print( v.x..','..v.y )
      
      mx = mx + v.x
      my = my + v.y
      mn = mn + 1
    end
    
    if mn <= 0 then return 0 end   
   
    mx = mx / mn
    my = my / mn
    
    local aaa = math.atan( -(my-yy), mx-xx ) * 180 / 3.1415926
    if aaa < 0 then aaa = aaa + 360 end
    
    --print( "ANG="..aaa )    
    return aaa
  end
  
  function ang(xx,yy,zx,zy,ax,ay,bx,by)
    clr(pts)
    clr(was)
    was[xx+yy*WW] = 1
    was[zx+zy*WW] = 1
    coll(xx+ax,yy+ay,1)
    local a1 = avga(xx,yy)
    
    clr(pts)
    clr(was)
    was[xx+yy*WW] = 1
    was[zx+zy*WW] = 1
    coll(xx+bx,yy+by,1)
    local a2 = avga(xx,yy)
    
    local d1 = a2-a1; if d1 < 0 then d1 = 360+d1 end
    local d2 = a1-a2; if d2 < 0 then d2 = 360+d2 end
    
    if d2 < d1 then d1 = d2 end
    --print( "DA="..d1 )
    return d1
  end
  
  local ttt = 10
  local any = 1
  while any > 0 do
    any = 0

    -- rule T: triple junctions

    for cy=ry1,ry2 do
      for cx=rx1,rx2 do
        local px = cx-rx1 + selection.bounds.x - cel.bounds.x
        local py = cy-ry1 + selection.bounds.y - cel.bounds.y
		 
        if check(cx,cy) and ff(px,py) then
          
          if ff(px-1,py) and ff(px+1,py) and ff(px,py-1) and nn(px,py+1) and nn(px-1,py-1) and nn(px+1,py-1) then
             any = 1
             img:drawPixel( px, py, ct )
          end

          if ff(px-1,py) and ff(px+1,py) and ff(px,py+1) and nn(px,py-1) and nn(px-1,py+1) and nn(px+1,py+1) then
             any = 1
             img:drawPixel( px, py, ct )
          end

          if ff(px,py-1) and ff(px,py+1) and ff(px+1,py) and nn(px-1,py) and nn(px+1,py-1) and nn(px+1,py+1) then
             any = 1
             img:drawPixel( px, py, ct )
          end

          if ff(px,py-1) and ff(px,py+1) and ff(px-1,py) and nn(px+1,py) and nn(px-1,py-1) and nn(px-1,py+1) then
             any = 1
             img:drawPixel( px, py, ct )
          end
        end
      end
    end

    -- rule P: pairs with diagonal 2-connects
  
    for cy=ry1,ry2 do
      for cx=rx1,rx2 do
        local px = cx-rx1 + selection.bounds.x - cel.bounds.x
        local py = cy-ry1 + selection.bounds.y - cel.bounds.y
		
        if check(cx,cy) and ff(px,py) then
          if ff(px+1,py) and nn(px-1,py) and nn(px+2,py) then
            if ff(px,py-1) and ff(px+1,py+1) and nn(px+2,py-1) and nn(px-1,py+1) and nn(px+1,py-1) and nn(px,py+1) then
              any = 1
              if ang(px,py,px+1,py,0,-1,1,1) > ang(px+1,py,px,py,-1,-1,0,1) then
                img:drawPixel( px+1, py, ct )
              else
                img:drawPixel( px, py, ct )
              end
            end

            if ff(px+1,py-1) and ff(px,py+1) and nn(px-1,py-1) and nn(px+2,py+1) and nn(px,py-1) and nn(px+1,py+1) then
              any = 1
              if ang(px,py,px+1,py,0,1,1,-1) > ang(px+1,py,px,py,-1,1,0,-1) then
                img:drawPixel( px+1, py, ct )
              else
                img:drawPixel( px, py, ct )
              end
            end
          end
    
          if ff(px,py+1) and nn(px,py-1) and nn(px,py+2) then
            if ff(px-1,py) and ff(px+1,py+1) and nn(px-1,py+2) and nn(px+1,py-1) and nn(px-1,py+1) and nn(px+1,py) then
              any = 1
              if ang(px,py,px,py+1,-1,0,1,1) > ang(px,py+1,px,py,-1,-1,1,0) then
                img:drawPixel( px, py+1, ct )
              else
                img:drawPixel( px, py, ct )
              end
            end

            if ff(px-1,py+1) and ff(px+1,py) and nn(px-1,py-1) and nn(px+1,py+2) and nn(px-1,py) and nn(px+1,py+1) then
              any = 1
              if ang(px,py,px,py+1,1,0,-1,1) > ang(px,py+1,px,py,1,-1,-1,0) then
                img:drawPixel( px, py+1, ct )
              else
                img:drawPixel( px, py, ct )
              end
            end
          end
        end
      end
    end

    -- rule C: lone corners
    
    for cy=ry1,ry2 do
      for cx=rx1,rx2 do
        local px = cx-rx1 + selection.bounds.x - cel.bounds.x
        local py = cy-ry1 + selection.bounds.y - cel.bounds.y
		
        if check(cx,cy) and ff(px,py) then
          if ff(px,py-1) and ff(px+1,py) and nn(px-1,py) and nn(px-1,py+1) and nn(px,py+1) then
            any = 1
            img:drawPixel( px, py, ct )
          end

          if ff(px,py-1) and ff(px-1,py) and nn(px+1,py) and nn(px+1,py+1) and nn(px,py+1) then
            any = 1
            img:drawPixel( px, py, ct )
          end

          if ff(px,py+1) and ff(px+1,py) and nn(px-1,py) and nn(px-1,py-1) and nn(px,py-1) then
            any = 1
            img:drawPixel( px, py, ct )
          end

          if ff(px,py+1) and ff(px-1,py) and nn(px+1,py) and nn(px+1,py-1) and nn(px,py-1) then
            any = 1
            img:drawPixel( px, py, ct )
          end
        end
      end
    end
    
    ttt = ttt - 1
    if ttt <= 0 then break end
  end
  
  cel.image = img
end

app.transaction( function() Engy_Jagger() end )

2 Likes