Handling image data as binary for canvas, also looking into what's the Tainted canvas, how to XHR load array buffer and streaming binary data with fetch.
Since we're working with binary data, lets take a look on Typed arrays and a nice feature of them, called Subarray vs splice. Working with Typed Arrays can be tricky, lets see how to Concatenate TypedArrays and Set data in TypedArrays
With a recent project the usage of Web Crypto API came up, so I've checked out how Crypto libraries handle performance.
Some minor goodies: a quick https server for your localhost using docker & nginx, embedding google fonts and testing your code against all sort of HTTP status code responses out there
This month there was a project where I needed to fetch and process the binary data of an image. We can assume this is going to be easy, as the browser can load images in image tags. As it turned out, accessing the image as binary data is not a trivial task.
Get image data as binary
In the past, this was Flash territory, but now we have tools like Canvas, ArrayBuffers, TypedArrays, Blobs, URL API and the FileReader API
First, I just had to experiment with canvases, because putting an image on a canvas is so easy. The three ways of getting the image from a canvas:
toDataURL(mimeType)
, which basically serialize the image to base64getImageData(x,y,w,h)
, which has to be ran on the context, and provides the data in a TypedArraytoBlob(callback, mimeType, quality)
, an async method, resulting in a Blob
All together in an example:
gist:994d775d74d5b253e97dcc13795588fb#canvas_to_bytearray.js
Most examples stop here, announce that it’s done. But if you want to scale things up, this images will probably come from exotic URLs of CDN providers, which are far away from your site on your domain, or your Origin. At this point, trying to get the image data from the Canvas results in a dark and mysterious error message!
Tainted canvas
The Tainted Canvas error message in the consoleThis means, that the image can be used and displayed in canvas (just like in the IMG tag) but access to its data is locked, for security reasons.
To cure the tainted canvas, you have to make sure, you trust and negotiate with the source where you received the image, simply by using CORS. Supported cross-origin images can be used just as same-origin ones, and you can pull the data.
gist:994d775d74d5b253e97dcc13795588fb#cors_image.js
A better way
If you’ve managed to serve these assets using CORS, there is a more elegant way to get the binary data of images.
Simply load them using XHR, but set the responseType
to “arraybuffer”.
gist:994d775d74d5b253e97dcc13795588fb#xhr_to_arraybuffer.js
You have to load the full image to make this work. You cannot do much with an ArrayBuffer at hand, it’s not even an iterable object, but you can convert it into more usable data formats. (I’ll skip the DataView interface for now).
gist:994d775d74d5b253e97dcc13795588fb#bytearray_to_blob.js
Streaming binary data using Fetch
Finally, we can try and parse binary data on-the-fly while it’s being downloaded. The relatively new Fetch API makes this possible, with the help its components, the Request, Response API and the Body mixin.
Turns out the body
in the arriving response
instance, is a ReadableStream! If you are familiar with streams in Node.js,
you either rejoice or start to look at the screen suspiciously, but here it looks pretty usable.
gist:994d775d74d5b253e97dcc13795588fb#fetch_stream.js
(BTW, loremflickr.com is the only placeholder image service so far, that has images with CORS headers.)
When a chunk has arrived, you can pass its data to further processing, and update a progress bar for example, so it’s pretty useful UI/UX wise too! Check out the whole example in this gist.
Conclusion
Which one is better? That depends...
- use the canvas method if you need an image, and plan to handle the data in a visual context, like cropping, altering
- use the others if you need raw binary data, whatever the mime type is
Loading just any binary data to canvas through images is only possible if you serve the binary as an image Content-Type
.
Since we’re meddling with binary data, it’s important to take a closer look at TypedArrays
TypedArrays!
TypedArrays are array-like objects, representing binary data. It's not a global property, there is no window.TypedArray
but a number
of global objects that has the same API. Check out TypedArrays on MDN for all of them!
Some API methods are familiar from the simple Array, but they have a few special ones that are really interesting and useful.
subarray vs slice
gist:994d775d74d5b253e97dcc13795588fb#subarray_vs_slice.js
Both of these methods return a part of the original TypedArray. It seems they do the same, even has the same signature, but there is a major difference in how they provide the data!
- slice creates a new array, copying data over from the original one
- subarray provides an array, where the values are references to the original ones
Subarray has a huge performance win, but be aware that altering the values in such a subarray mutates the original source too! If you keep that in mind, it's a great tool to split up binary data into smaller chunks and read/decode them for example
Concatenating TypedArrays
First, let it be clear, there is no .concat()
method on TypedArrays. Binary data structures have fixed length, so you cannot concatenate one to another,
neither can push
or unshift
data to them.
But you can create new ones from other fixed-length binary arrays, using their values in the .of() method - this way they are concatenated, sort of.
gist:994d775d74d5b253e97dcc13795588fb#typedarray_concat.js
Another method is using a Blob to glue the TypedArrays together. This may be useful if you're streaming down some larger binary data with a known MIME type, and you need to access it later as a blob url.
gist:994d775d74d5b253e97dcc13795588fb#blob_concat.js
Set data in TypedArrays
TypedArray.set takes an array of values and an offset, and copies these values to the target array starting at the offset index. Much cleaner than iterating between indexes and writing them one-by-one, and much faster than splice would be - which is unavailable in TypedArrays.
gist:994d775d74d5b253e97dcc13795588fb#typedarray_set_vs_slice.js
Keep in mind that binary data is always fixed size, so you cannot use .set()
to make the TypedArray bigger!
Web Crypto API
This month I had the opportunity to work with the Web Crypto API, which was really a black box for me until now. Before anyone rushes to use it for encryption or decryption, it’s important to emphasize that it works under HTTPS only - which is understandable. We have to get accustomed to the practice of browser vendors, where they enable new web APIs strictly under HTTPS only - and this happens to make the web a better place.
In the simplest example, we encode a string with a password, using some cryptographic algorithm.
In the example, we’ll import a key from a password. Here we’ll use the AES-GCM algorithm to encode data, which needs a key at least 128 bit long, and an initialization vector of random data, 16 Byte long - both of them should be a TypedArray, so we have to prepare them first
gist:994d775d74d5b253e97dcc13795588fb#crypto_initvector.js
Now a key has to be created. With the Web Crypto API, the key is not just a piece of string or a password, it has to be an object
instance implementing the CryptoKey
interface. This key object can be generated, derived from a master key or imported,
from a password string for example. Create them using one of the factory methods on the SubtleCrypto object.
The factory will return a promise, which receives the key when it resolves.
gist:994d775d74d5b253e97dcc13795588fb#crypto_importkey.js
We have the key! We can encrypt our super secret string! Just pass the initialization vector, chosen algorithm,
the key and the data to the .encrypt()
method. The result arrives in an ArrayBuffer.
gist:994d775d74d5b253e97dcc13795588fb#crypto_encrypt.js
Take a look at these examples using all the cryptographic algorithms.
Libraries
If you don’t have Web Crypto support in the target browsers of your web app, or want to encrypt/decrypt on HTTP for some reason, there are several JavaScript implementations of the most used cryptographic algorithms. The most performant may be the Asmcrypto.js from the Tresorit team, check it out!
Quick local HTTPS server
Recently I needed HTTPS on my local machine several times and found out that configuring & running apache on OSX can be tedious. So I’ve created a small nginx setup in docker, that can serve some static files and assets for playing with APIs or prototyping.
Enjoy: necccc/quick-local-https
Embedding Google Fonts
It's still debated if embedding your web fonts in css a good idea or bad idea - caching versus extra requests, but if you ever need to embed your selected fonts from Google Fonts, this tool might come handy.
HTTP Response Statuscode test service
Ever wondered what would your web app, or service do, if it suddenly receives some http status code other than 200 or 403 or 5xx? Test and prepare for those with the help of this nice tool, which can respond with almost any http status code that is commonly used: httpstat.us