HiDPI Images or “Fun With Trigonometry”

This week I was in the process of ordering a metal print of one of my turtle photos from my vacation. The instigator of this is photographer Mark from Twin Lakes Images, the artist formerly known as Switcher Mark. He’s a big believer in printing photographs and he convinced me this would be fun.

Printing the photo meant making a lot of decisions, including not just the medium (I wanted to try metallic because it looked cool in a gallery I visited recently) but also the size and proportion of the print.

While this photo was taken on a GoPro and was a 10MP image, the resolution was a bit low for the size print I wanted to do. In Affinity Photo in the Export Persona (they call the different modules personas for some reason) they have options for 1x, 2x, and 3x. Mark and I inferred that the latter two buttons would up-sample the image. Up-sampling an image means to add pixels to an image.

I suggested that this was fake, cheating, misleading or at the very best silly, because the pixels don’t exist so the software has to make them up. I told Mark that the software would have to interpolate to figure out where those extra pixels were supposed to be. (At the end of this story I’ll tell you that I figured out it’s not interpolation but we can’t get there yet!)Little did Mark know that his offer to help me get an image printed would turn into a trigonometry lesson. I asked him if he took Trigonometry in school and he said he did not, he took a Debate class. It won’t come as a shock to you to learn that in addition to being a photographer, he’s also a judge.

I explained to him that back in MY day, we didn’t have these fancy calculators so we had to do math the hard way. In the snow. Uphill both ways.

Trigonometry gets a bad name but I thought it was great fun. It will come as no shock to you to learn that in addition to being a podcaster, I am also a mechanical engineer.

Let me give you a basic trig lesson and then I’ll explain why interpolation was so important.

If you have a right triangle, meaning one angle is 90 degrees, and you know the lengths of two of the three sides, you can actually figure out the interior angles using the sine function.

The sine of an angle is simply the length of the opposite side from the angle divided by the length of the hypotenuse (that’s the slanty one). Easy peasy, just remember sine is the opposite over the hypotenuse, which is only hard if you try to spell hypotenuse.

This starts to make more sense if you draw it out. First draw a circle with a radius of one. Draw a line from the center to the edge of the circle at 30 degrees from horizontal. Drop a line down from where it hits to the horizontal line. Now we have a right triangle with a 30 angle and the hypotenuse is 1. We can simply measure the height of the line we dropped and we’ll find out that it is .5. From this we know that sin(30) = .5.

Luckily, with today’s calculators you don’t have to draw circles and lines and measure, you just type in the numbers and the calculator spit it out for you. But what about when I was in high school back in the dark ages?

Well back in MY day, they had tables of the values for sine. We had books made of paper that enclosed these tables. The one I still have is called the CRC Math Tables, published in 1971 by the Chemical Rubber Company. There’s a screenshot from my CRC Math Tables book in the shownotes but let me give you a simplified example of how we used them.

The CRC can’t tell you the sine of every single angle because there’s an infinite number of them, right? So they give you some and you have to calculate the one you want. This is where interpolation comes in.

Let’s say we need to know the sine of 31.8 degrees. We can look in the CRC math tables and see find the sine of 31 and the sine of 32. Now remember how we started with a circle to figure out the sin(30), right? Well what if we were to approximate a circle by making it out of 360 little lines instead of a perfect circle? If we assumed that between 31 and 32 degrees, we had a straight line instead of an arc, we could approximate sin(31.8) as being 80% of the way along that line between 31 and 32.

I did the math on this and put it in the shownotes, but let’s walk through the logic if not all the numbers. I said that 31.8 is 80% of the way between 31 and 32 degrees. We get that by calculating 31.8 – 31 divided by 32-31, It turns out that you can substitute in the sines of all those angles and it has to also be 0.8. So sin(31.8) – sin(31) divided by sin(32) – sin(31). That’s a tough one to hear in your head to follow, I know.

In the equation I just described, we know the value for everything except sin(31.8) so the arithmetic is to solve for sin(31.8). When I did that, I got that the sin(31.8) is .5269. I gave this example because the CRC math tables do give us the value for sin(31.8) and instead of 0.5269 as we just calculated, it’s actually 0.5270 according to the CRC Math Tables. Remember we said that assuming it’s a straight line between the two points was an approximation? You can see that in the fourth decimal place we are inaccurate, but that’s pretty darn close!

Believe it or not, that’s how we calculated the trigonometric functions when I was in high school. Except it was one tiny bit harder than that. When I took trig, there were no calculators for the common person. We did the arithmetic part of these calculations using a slide rule. To four significant digits! Now do you see why Mark took Debate instead of Trigonometry?

So what were we talking about? Oh right, photography! By this time in the conversation Mark had died of boredom but I was still thinking about the original problem. When we up-sampled the photo using Affinity Photo’s 2X feature, was it actually doing interpolation? Was it looking at the pixels on either side of it and figuring out something in-between? If it was, then there could be a perceived improvement to the photo. Edges that might look jagged when zoomed up close, might be smoother after this up-sampling.

I drew up a diagram in Excel to think this out. I created a 10×10 grid, and made a diagonal line of green and one of blue right next to it. Then I drew a grid of 20×20 representing the 2x example from Affinity, and populated the grid again. Now we have a smoother-looking line. This is essentially what anti-aliasing does in text, which is how we ended up with smoother looking text than the old days.

Now is that actually a better photo? In my opinion, it depends on the algorithm to create the pixels. If poorly done, it could create a terrible-looking image with all kinds of fake looking edges and artifacts. Or if done brilliantly, it could create an image which when blown up looked much crisper than the original. If they were using interpolation, did they use linear interpolation like in our trig example, or did they using a polynomial interpolation? How much math were they going they do?

I ran the 2x export on Affinity Photo on an image, zoomed way up…and I couldn’t tell the difference! So I started searching around for more wisdom on what this 2x and 3x thing might be. I started to find a pattern online that when people talked about 2x, they also said HiDPI and retina screen in the same sentence. I watched the video tutorial from Serif on the subject and they said the same kinds of words.

So what does this mean? Let’s go back to our simplistic example of our original 10×10 grid with diagonal green and blue jagged lines. What if instead of interpolating, they’re actually just putting 4 pixels where every original pixel was. Instead of one green pixel in the bottom left, we have 4 green pixels. The image, entitled 2X would then become a 20×20 pixel image.

This sounds dumb, but remember people use HiDPI and Retina in this context. Imagine that you have two laptops, say a 13″ MacBook Air which has a normal resolution screen and a 13″ MacBook Pro that has a new Retina screen, which is just a fancy marketing term from Apple that means higher resolution with lots of dots per inch, also known as HiDPI. For the sake of simplicity, let’s imagine that the retina screen has twice as many dots per inch as the normal screen.

If we took our original 10×10 pixel image with the diagonal line and opened it on both screens, the image on the retina display would be half the size of the one on the normal screen. Does that make sense? Let me say it another way. If the normal screen was 10 pixels per inch, our 10×10 pixel image would be 1 inch across. If our retina screen was 20 pixels per inch, the 10×10 pixel image would only be 1/2 inch across. With me?

So, back to our 2x export. If they give us 4 pixels for each 1 in the image, then that image, then our 20×20 pixel image would look 1 inch across on the retina screen!

You might be thinking this would make things look dumb on the normal screens but I think I know the use case for this. When you surf to a website, the site can tell what kind of computer you’re coming from, it knows what browser you’re using and it even knows the resolution of your display. This means that webmasters can store multiple versions of the same image and if you come with a low res screen, serve you the normal version but if you come with a higher res screen, it can serve you the 2x (or 3x) version.

I know this was a long way around to get there but I had fun talking about interpolation and the old days and I’m delighted that I put the pieces together to understand what’s really going on in Affinity Photo when they offer the 2x and 3x options on export. It’s not making your image any better looking, it’s just changing how big it looks on screen when you’re served it from the web.

Maybe I should have a Disclaimer on my podfeet.com warning people that I’m an engineer.