Thanks to greg.org reader Fred for sending along a link to a memo computer graphics pioneer Alvy Ray Smith wrote in 1995, soon after his company Altamira [the one he founded after Lucasfilm and Pixar] had been assimilated by Microsoft.
The Title: “A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square!” [pdf via alvyray.com, a year later, he added, “(And A Voxel Is Not A Little Cube)”] Two guesses what it’s about.
Alvy’s rant sounds exactly like what I’d expect from circa 1995 Microsoft: brilliant, self-assured, and presumptuously prescriptive. Which is not, alas, the same thing as being entirely right or even aware of its own limitations.
Because it gets a little rambling, and because I am basically arguing pixels with the guy who worked on the Project Genesis sequence in Wrath of Khan, I have moved it all after the jump.
He’s absolutely right in one sense: a pixel is not square, any more than an atom is made of tiny marbles. They’re models, metaphors, abstractions, which are used to simplify and enable something. A pixel is a data point, a point sample.
And Smith argues that it’s sloppy and harmful to conflate the real thing–the point sample–with the inherited, inaccurate, geometry-based model used to describe it–the rectilinear grid of squares. Because it screws up all your reconstruction filters, the calculations and processes for assembling the highest quality image from your pixels, which in turn screws up all the things you should be doing with digital images but can’t, because they’re trapped in this useless rectilinear grid.
OK, except that there’s a reason people–including, apparently, plenty of people involved in imaging at Microsoft–think of square pixels. Because they were square and they still are:
Figure 3 [above] is the same image reconstructed with one of the poorest reconstruction filters– a box filter. The only thing poorer is no reconstruction at all–resulting in the abominable “jaggies” of the early days of computer graphics–and we will not even further consider this possibility. The Figure 3 reconstruction too is a valid image computation, even though it is lacking in quality. This lowest-quality case is the only one that suggests the little square model.
That’s what Smith loves to call it, the little square model. Which is not just a model, it’s a historical fact. When Kirsch et al scanned the first image, they literally laid a grid mask over a photo and programmed the scanner to take a binary, b/w reading of each square. But Smith knows this, of course.
His point turns out to be, not that pixels aren’t squares, but that square pixels suck.
What Smith’s criteria for quality are, though, remains unarticulated. Since he frequently points out how he shouldn’t have to point out painfully obvious things, but he will, because people are so slow, I’ll assume that if he even thought assumptions about quality were in question, he would have had to explain those, too.
Maybe it’s something like elegance in mathematics. Smith writes at length about mapping, and reconstruction filters, and the Sampling Theorem, a foundation of all computer imaging, wherein a reduced number of samples [pixels] accurately reproduce the whole. So now we’re talking about accuracy, which is still subjective, but more measurable than quality.
Or maybe it’s something even more self-evident, like Progress. Better, stronger, faster. Realer. The little square model is the best they could do in the past, but now we can do better. As long as you don’t let the outdated model hold you back. One result, for better or worse, of this progress view of the little square model is the rise of the neo-retro pixel aesthetic. Square pixels are just another obsolete [sic] mode of expression to choose from, like letterpress, painting, or film photography.
Which I’m sure is fine for creative types, different strokes for different folks and all. But how frustrating it must be to be Alvy Ray Smith. He was kicking against square pixels in 1995, while evangelizing his own clearly [clearly!] superior paradigm for digital imaging, and yet 15 years later, not only has the square pixel become an art form, it still persists. In Photoshop, in photo-everything. In the browser. I’m sure we’ll figure it out soon enough.