hoodwink.d enhanced
RSS
2.0
XHTML
1.0

RedHanded

Sparklines for Minimalists #

by why in inspect

If you’ve read Joe Gregorio’s Sparklines in data: URIs in Python, then you know that he’s presented us with an incredibly compelling use for data: URIs. Namely, this is not an external file: But we don’t have Python’s image library, so what are we to do?

What if I offered you a handful of code that could generate inline sparkgraphs without library code? Presenting Bumpspark.

 def bumpspark( results )
     white, red, grey = [0xFF,0xFF,0xFF], [0,0,0xFF], [0x99,0x99,0x99]
     ibmp = results.inject([]) do |ary, r|
         ary << [white]*15 << [white]*15
         ary.last[r/9,4] = [(r > 50 and red or grey)]*4
         ary
     end.transpose.map do |px|
         px.flatten!.pack("C#{px.length}x#{px.length%4}")
     end.join
     ["BM", ibmp.length + 54, 0, 0, 54, 40, 
       results.length * 2, 15, 1, 24, 0, 0, 0, 0, 0, 0].
       pack("A2ISSIIiiSSIIiiII") + ibmp
 end

It’s got one known bug: graphs built from 16-element arrays get staticky. Free inky duck drawing to first patcher. (Keep reading.)

Great coats! jzp has dropped a ping version in the comments! MenTaL has a bump one with compression! Peewee graphics libs within!

said on 28 Apr 2005 at 13:55

Neat. I’ll have to play with this later if I get a chance.

Staticy images when you plot 16-element arrays is an interesting bug though.

Since I don’t have time to run through the code right now I’ll just pass along my gut feeling: I think you’re looking at byte alignment issues (perhaps due to rounding issues, with 16 being a likely boundary case). May also want to double-check the alignment requirements for rows imposed by the BMP format.

said on 28 Apr 2005 at 14:00

Incidentally, the data: URIs don’t work in IE. So if you’re getting a broken image…

said on 28 Apr 2005 at 14:22

Hrm, I had a few minutes so I tried it and I couldn’t reproduce the bug. Could you provide a sample results array for which it fails?

said on 28 Apr 2005 at 14:25

The bitmap output size makes me cry. PNG ’s are so much smaller.

I thought about writing a PNG out-put-er – but I don’t want to deal with CRC ’s.

said on 28 Apr 2005 at 14:31

You do rock though.

said on 28 Apr 2005 at 15:19

Hmm, the resulting bitmaps are a bit in the large size, I suppose.

This should help a little bit (as well as make the code portable to non-little-endian architectures):


 def bumpspark( results )
     white, red, grey = 0, 16, 32
     ibmp = results.inject([]) do |ary, r|
         ary << [white]*15
         ary.last[r/9,4] = [(r > 50 and red or grey)]*4
         ary
     end.transpose.map do |px|
         px.pack("C#{px.length}x#{px.length&1}")
     end.join
     ["BM", ibmp.length + 66, 0, 0, 66, 40,
       results.length * 2, 15, 1, 4, 0, 0, 0, 0, 3, 0,
       0xFFFFFF, 0xFF0000, 0x999999 ].
       pack("A2Vv2V4v2V9") + ibmp
 end
said on 28 Apr 2005 at 15:19

Incidentally, the blaahg preview eats my plus signs.

said on 28 Apr 2005 at 15:41

Innocent bystander I am, clappings hands red.

said on 28 Apr 2005 at 16:00

You could get around the IE problem by making it an HTML image and making the url point to a ruby script that returned a gif or png mime type and the actual byte content of the image.

said on 28 Apr 2005 at 16:02

The following hack on the original encodes the URLs in base64, and creates compressed PNGs for extra space saving.

Like MenTaLguY, my plus signs seem to go away in the preview.


require 'base64'
require 'cgi'
require 'zlib'
def build_png_chunk(type,data)
    to_check = type + data
    return [data.length].pack("N") + to_check + [Zlib.crc32(to_check)].pack("N")
end
def build_png(image_rows)
    header = [137, 80, 78, 71, 13, 10, 26, 10].pack("C*")
    raw_data = image_rows.map { |row| [0] + row }.flatten.pack("C*")
    ihdr_data = [   image_rows.first.length,image_rows.length,
                    8,2,0,0,0].pack("NNCCCCC")
    ihdr = build_png_chunk("IHDR", ihdr_data)
  idat = build_png_chunk("IDAT", Zlib::Deflate.deflate(raw_data))
    iend = build_png_chunk("IEND", "")

    return header + ihdr + idat + iend
end
def bumpspark( results )
     white, red, grey = [0xFF,0xFF,0xFF], [0,0,0xFF], [0x99,0x99,0x99]
     rows = results.inject([]) do |ary, r|
       ary << [white]*15 << [white]*15
         ary.last[r/9,4] = [(r > 50 and red or grey)]*4
         ary
     end.transpose
     return build_png(rows)
end
said on 28 Apr 2005 at 16:25

Hmmn. Seem to have forgotten to include the URL encoding bit in the above post. Not sure if the CGI.escape ing is really necessary. The URL should be quoted with CGI.escapeHTML anyway before getting put inside double quotes in an img tag.


def build_data_url(type,data)
    data = Base64.encode64(data).delete("\n")
    return "data:#{type};base64,#{CGI.escape(data)}" 
end
url = build_data_url("image/png",bumpspark(results))
said on 28 Apr 2005 at 16:50

Carl, that’s wonderfully devious.

jzp, that’s excellent. I had forgotten I could have had zlib at my fingertips. While you’re at it, why not make an indexed PNG ?

said on 28 Apr 2005 at 17:18

As a matter of interest, I tried a version that produced RLE4 -compressed bitmaps (the sparklines are well-suited to that, since they skip every other row and thus always let us combine every two pixels).

PNG consistently wins over that at half the size.

said on 28 Apr 2005 at 18:07

SVG ?

said on 28 Apr 2005 at 18:30

MenTaLguY is now the overlord for RedHanded comments. I think I’m going to add a second layer of comments (to the right hand of every comment) where MenTaLguY can annotate. I’m brutally serious.

Krikey, jzp! Send me your address. It’s true that you get a duck. (MenTaL, send me yours too. You get a clock.)

Next up: we gotta make a 1k graphics lib for Ruby out of this.

said on 28 Apr 2005 at 19:03

Can we get the .pngs to have transparent backgrounds instead of white?

said on 28 Apr 2005 at 21:11

Yes, a transparent background is quite easy; this is untested, but I believe you need only modify build_png thusly:


def build_png(image_rows)
    header = [137, 80, 78, 71, 13, 10, 26, 10].pack("C*")
    raw_data = image_rows.map { |row| [0] + row }.flatten.pack("C*")
    ihdr_data = [   image_rows.first.length,image_rows.length,
                    8,2,0,0,0].pack("NNCCCCC")
    ihdr = build_png_chunk("IHDR", ihdr_data)
  trns = build_png_chunk("tRNS", ([ 0xFF ]*6).pack("C6"))
  idat = build_png_chunk("IDAT", Zlib::Deflate.deflate(raw_data))
    iend = build_png_chunk("IEND", "")

    return header + ihdr + trns + idat + iend
end

The added tRNS chunk indicates that white ( 0xFFFF, 0xFFFF, 0xFFFF ) is to be interpreted as transparent.

said on 29 Apr 2005 at 00:58

Concerning jzp’s pung stuff: The sparklines are coming out upside down, unless I do a transpose.reverse. The transparency fix looks right, but it’s not right. Tinker tinker.

said on 29 Apr 2005 at 03:45

Re: transparency: it’s also possible to use RGBA pixel values instead of RGB and have per-pixel alpha. Although, I don’t know whether any browsers actually render these properly.

Re: upside down: Oops! I should have tested more…

Re: indexed PNG : Haven’t tried it yet. I’m hoping that the compression of RGB pixels does a fairly good job. However, I suppose 4-bit indexes could reduce the data nicely. Makes packing a bit more complicated though!

I did try implementing the Up() filter which does a line-by-line delta (good since there’s lots of vertical redundancy in these sparklines), but the compressed data actually got bigger in my little test case (!), so I abandoned it.

said on 29 Apr 2005 at 04:36

Cant you do the same things with inline XBitmaps in HTML ?

said on 29 Apr 2005 at 04:45

As a red Tufte disciple, I find myself prostate to this sparkline development.

But, “honey, where’s my super test/unit suit?”

said on 29 Apr 2005 at 06:06

Requesting the continuous plot in Joe Gregorio’s updates.

said on 29 Apr 2005 at 07:29

This is good stuff. I think it would be great for adding barcodes to pages (though most readers emit light, so won’t read off the screen). But reading code off here in the narrow column for comments is tricky for me, even when I make the font too small for comfort. _Why, please could you copy the comments to your Bumpspark page, for easier access? Thank you.

said on 29 Apr 2005 at 08:48

hgs: The latest working ping spark is up on the Bumpspark page now. I haven’t been able to get MenTaL’s compressed bump to work yet. It’s close, but just not ready yet.

said on 29 Apr 2005 at 10:07

Thank you, that’s much easier to see. I may have something to contribute later…

said on 29 Apr 2005 at 11:22

Mine (the one I posted at least) isn’t really compressed; it just creates a 4bpp indexed BMP rather than a 24bpp truecolor one.

Out of curiousity, what problems are you having with with that version? I didn’t test it exhaustively, but I thought I checked all the corner cases.

said on 29 Apr 2005 at 11:32

Whoa, never mind. I see what you mean.

Argh… I think I uploaded the wrong version. And I’d just deleted my scratch files since I thought I’d preserved the final version for posterity here….

said on 29 Apr 2005 at 12:02

I love the pingspark! So much information in so few bytes.

red should be [0xFF,0,0] not [0,0,0xFF] in pingspark though.

said on 29 Apr 2005 at 12:11

Yeah, stride issues. I remember now. It’s the x#{px.length&1} bit.

I don’t remember what the correct solution is. The alignment rules were a bit non-obvious.

said on 29 Apr 2005 at 12:22

If I’m correct Base64 encoded data does not need to be cgi escaped for data urls. It’s at least working well for me without the extra escaping.

Base 64 is just: 26 lower case letters 26 upper case letters 10 digets 1 ”+” 1 ”/”

64 characters.

said on 29 Apr 2005 at 12:24

Ah, I’m stumped. Even simple byte-alignment doesn’t work. Check this (using a 4bpp bumpspark with configurable byte alignment for rows): width/alignment matrices

said on 29 Apr 2005 at 15:46

Sparklines in use.

http://www.braino.org/blog/images/sparklines.gif

said on 29 Apr 2005 at 22:14

Oh. Duh. Look at the 4-byte alignment column.

I suspect that might start working if an extra byte of padding were added to the end of the odd-byte-width ones.

Let me try that…

said on 29 Apr 2005 at 22:15

(I have a feeling I never really had it quite working before, I was just “lucky” in picking my test cases)

said on 29 Apr 2005 at 22:37

Got it!


 def bumpspark( results )
     white, red, grey = 0, 16, 32
     padding = 3 - ( results.length - 1 ) % 4
     ibmp = results.inject([]) do |ary, r|
         ary << [white]*15
         ary.last[r/9,4] = [(r > 50 and red or grey)]*4
         ary
     end.transpose.map do |px|
         px.pack("C#{px.length}x#{padding}")
     end.join
     ["BM", ibmp.length + 66, 0, 0, 66, 40,
       results.length * 2, 15, 1, 4, 0, 0, 0, 0, 3, 0,
       0xFFFFFF, 0xFF0000, 0x999999 ].
       pack("A2Vv2V4v2V9") + ibmp
 end
said on 30 Apr 2005 at 06:23

Regarding data not workin in IE: I think you can instead use a javascript:’data, it\’s going here’ style URLs to work around that. For images and so you might also have to overwrite the content-type which is possible when using instead of .

said on 03 May 2005 at 07:56
Not bumpspark but nice too:

def fract( iter ) # use a small value like 15 as iter and wait a few seconds
    ycoord = (0..127).map{|i|(i/63.0)}
    xcoord = (0..255).map{|i|(i/63.5)-2.6}
    half = ycoord.map do |y|
      xcoord.map do |x|
        zr = zi = 0.0
        cr,ci = x,y
        i = 0
        while i < iter and (zr*zr + zi*zi)<8
          zr,zi = zr*zr-zi*zi+cr,2*zr*zi+ci 
          i+=1
        end
        if i == iter
          [0,0,0]
        else
          va = (zi.abs*127).to_i^(vb=zr.abs*127).to_i
          [[0,va,vb],[va,vb,0],[vb,0,va],[0,vb,va]][i&3]
        end
      end
    end
    build_png(half.reverse+half)
end
said on 10 May 2005 at 15:03

alas, this inline data stuff makes Privoxy (my filtering proxy) die, wit a quickness.

said on 12 May 2005 at 06:26

anyone else not able to see the data uri on this page in Firefox’s view source? On mine it just appears as white and I have to highlight it to see it. Seems to only be limited to the image on this page though. (filed bug 293875)

said on 29 Jul 2005 at 13:34

For those looking for a less clever but more featureful RMagick implmentation, try my Sparkline Library for Ruby

And, it comes with a helper so you can use it with Rails in a cross-browser-friendly way!

said on 28 Aug 2005 at 17:31

TiddlyWiki can build sparklines on the client side. Some helpers for doing the same thing in Rails would probably go a long way. Saves a lot of bandwidth, too, when you only have to pump the numbers down the pipe. :-)

said on 29 Aug 2005 at 13:28

jjkllkjklljkljlkjlkj

said on 10 Dec 2005 at 15:26

Does anybody know why some combinations don’t work? For example, this sequence of numbers gives me a blank image:

[15, 13, 61, 35, 79, 9, 59, 79, 73, 50, 82, 88, 14, 80, 81, 41, 43, 33, 3, 97, 75, 4, 42, 77, 46, 1]

Comments are closed for this entry.