toBuffer optimization

[kangax]

I ran into performance issues with toBuffer. Here's an example of drawing (rather) large image onto canvas, then calling toBuffer. toBuffer is what's taking the longest here (drawImage, for example, is insignificant). I'm getting ~7-7.5 sec on toBuffer call. This is on node v0.10.13, node-canvas v1.0.3, and Mac OS X (10.7.5).

Is there a way to speed things up? Anything at all? I also tried async toBuffer but it doesn't really help.

var fs = require('fs'),
    Canvas = require('canvas'),
    Image = Canvas.Image;

// 3.9MB, 4686x4668
fs.readFile(__dirname + '/big_img.jpg', function(err, bigImg) {
  if (err) throw err;

  img = new Image;
  img.src = bigImg;

  var canvas = new Canvas(img.width, img.height);
  var ctx = canvas.getContext('2d');

  ctx.drawImage(img, 0, 0);

  console.time('toBuffer');
  canvas.toBuffer();
  console.timeEnd('toBuffer'); // 7196ms, 7435ms, 7237ms
});

Attaching image used in a test.

[tj]

yikes that does seem pretty slow, I can't remember if we have .toBuffer('image/jpeg'), I don't think so but you might want to try the jpeg stream thing instead

[kangax]

You're right JPEG stream is almost 10 times faster — I'm getting ~900ms consistently even for 100% quality. I'll see if I can use JPEG instead of PNG, but would still be curious to see if we can optimize PNG somehow.

Offtopic: there's pngStream and createJPEGStream. Inconsistency! :)

[rvagg]

I'll have a look at this to see if I can understand it; just cause it's interesting and because we have some new Buffer stuff added into the next NAN release.

[rvagg]

oh, and I can see an additional memcpy that I left in with my NAN PR that may be contributing here. @kangax can you confirm you're using the latest node-canvas?

[rvagg]

Ouch, not what I expected. See flame graph.

This memcpy is dominating the stack, it's actually called ~11,000 times for @kangax's example image.

From what I can glean, this is a libpng issue where it's writing the image in many small chunks.

[rvagg]

By default, the data comes out of libpng in these chunks: 4b, 8b, 8192b. There's a memcpy for each of these chunks and it goes on for the whole image. I don't believe you can control this behaviour, it'd be nice if you could increase the buffer or have more control over the buffer allocation and usage but I can't see a way that you can.

[kangax]

@rvagg Interesting. Thanks for investigating. What about compression level? Is there a way to change it? Would it affect performance at all or is the bottleneck purely in data output?

[rvagg]

I don't believe you can control the compression level via Cairo without reimplementing the write_png() function. http://cgit.freedesktop.org/cairo/tree/src/cairo-png.c#n170 the png_struct *png; is the object you need to set png_set_compression_level() on: http://www.libpng.org/pub/png/libpng-1.2.5-manual.html#section-5.8 but you can't access it.

In fact, just to confirm that this is a libpng/Cairo problem, I did this in Canvas.cc:

    //status = cairo_surface_write_to_png_stream(canvas->surface(), toBuffer, &closure);
    status = cairo_surface_write_to_png(canvas->surface(), "/tmp/tmp.png");

which bypasses the whole memcpy-to-buffer stuff completely, the toBuffer callback isn't invoked at all, Cairo simply calls its write_png() and does an fwrite() for each chunk that libpng returns.

Timing the execution shows that it takes roughly the same amount of time as the original.

[kkoopa]

What about rewriting it to use a buffered dynamic memory implementation instead of just doing malloc each time? Each buffer would be backed by a dynamic array that doubled in size each time it didn't have space. It would be possible to give a starting size, or it should default to... 10, like Java's ArrayList.

[rvagg]

Well it sort of is at the moment, it starts off at 1024 bytes and then does a realloc each time it overflows and rounds up to the closest multiple of 1024. I had some success on a slower (SmartOS) VM by bumping this up 8192, speeding it up by nearly a factor of two, but it had no impact on speed on my much faster Linux machine.
Unfortunately you can't control the buffer that libpng uses internally to write the data to, it'd be nice to be able to intercept that and use your own buffer mechanism but I don't think that's possible, even with a local reimplementation of Cairo's write_png().

[kkoopa]

If a normal page size is 4k, it would make slightly more sense to do things in increments of that, rather than in 1k chunks. However, if the required size is large, it will be much faster to do it by exponential than linear growth, as it will lead to O(log n) mallocs rather than O(n) mallocs.

4 MB in 1K chunks
Linear: 4 MB / 1 kB = 4096 mallocs
Exponential: log 2^22 - log 2^10 + 1 = 13 mallocs

4 MB in 4K chunks
Linear: 4 MB / 4 kB = 1024 mallocs
Exponential: log 2^22 - log 2^12 + 1 = 11 mallocs

Anyway, LodePNG http://lodev.org/lodepng/ is a highly customizable PNG library. Could libpng be replaced? Might be possible to write a libpng facade on top, if one does not exist already.

[kkoopa]

I did some checking of my own, the solution seems rather straightforward. Starting with a bit from cairo_png.c

/**
 * SECTION:cairo-png
 * @Title: PNG Support
 * @Short_Description: Reading and writing PNG images
 * @See_Also: #cairo_surface_t
 *
 * The PNG functions allow reading PNG images into image surfaces, and writing
 * any surface to a PNG file.
 *
 * It is a toy API. It only offers very simple support for reading and
 * writing PNG files, which is sufficient for testing and
 * demonstration purposes. Applications which need more control over
 * the generated PNG file should access the pixel data directly, using
 * cairo_image_surface_get_data() or a backend-specific access
 * function, and process it with another library, e.g. gdk-pixbuf or
 * libpng.
 **/

The reason it works in this bad fashion is because it does not use buffering with libpng.

static void
png_simple_output_flush_fn (png_structp png_ptr)
{
}

So, the solution is to use cairo_image_surface_get_data() and feed it through libpng with buffering.

By the way, libpng does support custom memory management. If you want to use your own memory allocation routines, define PNG_USER_MEM_SUPPORTED and use png_create_read_struct_2() instead of png_create_read_struct():

png_structp png_ptr = png_create_read_struct_2
       (PNG_LIBPNG_VER_STRING, (png_voidp)user_error_ptr,
        user_error_fn, user_warning_fn, (png_voidp)
        user_mem_ptr, user_malloc_fn, user_free_fn);

[rvagg]

Nice find! It sounds like a lot of work, are you offering?

[kkoopa]

Shouldn't be too much. I'll cook something up later today..

[kkoopa]

PNG writing code has been extracted: kkoopa/node-canvas@9981b0f
Everything is still very slow, as there is no custom memory management or flushing behavior set up.