What I Learned Doing Speed Tests of the Drobo 5N2 vs. 5N

This article started out as a simple review of the Drobo 5N2 network attached storage (aka NAS). But as you might have noticed by now, these things take on a life of their own. I’m going to start with a bit of a walk down memory lane to explain how I got into using Drobos in the first place.

I’ll explain how I ended up with their newest 5-bay NAS and explain that my quest was to prove that the device is twice as fast as the previous model, the Drobo 5N. But the quest becomes the real story, because of what I learned about testing storage speeds. It’s a long story but it has unexpected twists so I think you’ll enjoy learning along with me.

I Said Drobos Were Stupid … Till I Bought One

Back in March of 2011 on NosillaCast #304, I had Katie Floyd of Mac Power Users on the show to talk about her new toy, a Drobo FS. In that episode, I specifically talked about why Drobos are dumb. My main problem with them was how I thought people were buying them for the wrong reasons.

A month later, I bought Steve a Drobo FS for his birthday. I explained in NosillaCast #310 how it wasn’t my fault, that Steve asked for it and he pretty much gets whatever he asks for. I said back then that I still thought it was silly but he actually did have a good use case for it. He needed a place to put his giant, uncompressed video projects to get them off of his internal drive, and we wanted them protected by redundancy. This wasn’t a true backup but it was better than having the data on a single drive.

The FS worked fine and in fact is still running today, but it wasn’t very fast. Three years later in September of 2014, I bought the Drobo 5N. In the post entitled How to Upgrade a Drobo The Hardest Way Possible I went through an elaborate justification of why we needed it. I explained that I needed a place to put tax documents and other files that I didn’t want on my laptop and definitely didn’t want in the cloud.

The Drobo 5N has been a workhorse and with the mSATA SSD Accelerator card, it’s really quite fast. The Drobo Dashboard software used to be very slow and a bit janky but vast improvements have been implemented lately and now it’s quite delightful to use to manage the devices.

Backing Up Drobo to Drobo Works, But It Has Issues

By September of 2014, I had figured out how to back up the new Drobo 5N to the Drobo FS (Backing up a Drobo – The Adventure Continues and my final instructions How to Back Up from One Drobo to Another). We finally had true backups of our data, albeit all on site. But it has pretty much become the gift that keeps on giving.

The backup from the 5N to the FS is accomplished through Chronosync on our Mac mini, and it’s been a big bag of hurt. I don’t blame Drobo or Chronosync but the failures are constant.

Sometimes, the Mac mini decides to change networks which breaks the connection to the Drobos who stay on the right network. Sometimes, Chronosync gets its panties in a bunch about a specific file and refuses all other transfers after that. We have an awesome house sitter who sleeps in the room where the Drobos live, so Steve will turn them off before we go on a vacation. But the poor Mac mini is still running Chronosync and crying, “where oh where have the Drobos gone?” Sometimes a stiff breeze comes through and the whole Drobo-to-Mac-mini-Chronosync structure collapses in a heap. I should have kept all of the email notifications I’ve gotten over the years that start with “ChronoSync Scheduler: Failed Sync”. I counted and I got 24 of those emails in May alone, and it wasn’t a bad month!

Drobo Offers Me a 5N2 to Test

You might be wondering by now why I’ve been taking you down memory lane on my Drobo adventures. I was approached by Drobo to test out their latest network attached storage device, the Drobo 5N2. While the Drobo 5N we’re using today really meets our needs, the Drobo 5N2 (according to the marketing messaging) explained that it’s twice as fast.

To be specific on that speed, the product page on the 5N2 explains that this new version supports Duel Ethernet Ports that can be bonded together either for speed or failover. In terms of speed, it says, ” These can be bonded together to provide approximately twice the read/write performance you get via a single port.” We’ll come back to that a bit later. I decided this would be fun to test since they offered to send me one.

In the interest of full disclosure, the 5N2 costs $499 before you add any drives. As I contemplated how I would test the 5N2, I thought about what drives I should put in it. I could tear the drives out of the FS, or the 5N that I already had, but then how would I do a speed comparison between any two models?

I wrote to the rep from Drobo and asked whether they would be wanting it back and they said that I would be allowed to keep it. I wanted you to know I got it for free, but to test it would not be without cost. Since I would be able to keep it, now I was fully motivated to populate it with new drives.

I bought five 4TB Hitachi 7200RPM 4TB drives and a Samsung 250GB 860 EVO Series mSATA SSD to go in the Accelerator bay. The hard drives were around $95 each and the SSD was $109, so I was out over $500 before tax for this “free” device.

I thought that was as important as telling you I got the Drobo for free because it says that I had enough faith that this would be a great device to put out my own matching funds.

Setup

First time setup of a Drobo is really easy. They do have an online guide but the pictures and animated GIFs are generic to any Drobo. They do offer steps to migrate disks from one Drobo to another but I wanted to start from scratch anyway.

The first step is to install the mSATA SSD. I keep repeating mSATA because these drives are nothing like a standard SSD. If you flip over the Drobo 5N2 (or the 5N), there’s a little drive bay door on the bottom, which is what they call the Accelerator Bay. Pop in the drive just like you would insert memory into a computer and close the pod bay door. I would be giving you specifics on the speed increase of adding the accelerator card, but they aren’t actually available from Drobo. We do it on faith pretty much that the caching is worth the money.

The front of a Drobo is a door that’s attached magnetically. Pull that off and you can see the 5 empty drive bays. Believe it or not, you just take your bare drives and shove them in there. They tell you to put the label up, but I double checked those instructions by peering inside to see which way the connector was facing.

Now you plug it into your network and power and turn it on. The lights on the front of the Drobo do a little dance and then turn amber. Normally you would next download and install the Drobo Dashboard software, but I already had it installed. I brought it up and giggled with delight that I could now see 3 Drobos!

When was the last time you bought a new electronic toy and there wasn’t a firmware upgrade to do before you got to play with it? The Drobo was no exception to this new way of life, and I was happy about that. Upgrades are generally good things!

One of the advantages of the 5N2 is that it has a tool called DroboDR, where DR stands for disaster recovery. The idea is that you get two Drobos and back one up to the other automatically so if you have a failure on one you just swap in the second one. I was hoping it would be the answer to my misbehaving Mac mini/Chronosync/Drobo dance I do today, but that was not to be. Turns out to use DroboDR, you have to have two Drobo 5N2s, which is pretty disappointing. I will be telling the tale of my adventure to get a backup going between the 5N2 and the 5N, but that’s a story for another day.

Port Aggregation

I’d like to quote the web page that explains the speed of the Drobo 5N2:

The 5N2 has dual Gigabit Ethernet ports. These can be bonded together to provide approximately twice the read/write performance you get via a single port. Or they can be used to provide a connection to two different subnets. Furthermore, they can be used in an “active/passive” mode so that if one port fails the other takes over.

I did a lot of reading to understand what this means, and how it works. The first thing to know is that there are about 128 synonyms for this technology. It’s called port bonding, port aggregation, bonded Ethernet, link aggregation, teaming port, and port trunking and network interface bonding. I’ll try to stick to port bonding but I may throw in some synonyms to see if you’re paying attention. The other thing to keep in mind is the speed increase only affects those who are reading and/or writing to the NAS from two different computers at the same time. If you access the device one at a time, you’re only limited by the one Ethernet connection.

Aided by my trusty research assistant Steven Goetz, he explained that the great news is that my Netgear R8500 router also supports port bonding. Sure enough, I found two Ethernet ports on my router that were made for this and I’d never figured that out. I really should raise his salary. I fussed around in the router’s settings until I found Ethernet Port Aggregation. After plugging in the two Ethernet cables from the Drobo 5N2, I enabled it on the Netgear. Take a spec note here – the Netgear specifically says to Enable LACP-IEEE 802.3ad. I applied and restarted my router. And it came back saying it was Enabled, but Inactive.

Ok, let’s take a look at the Drobo Dashboard settings from the 5N2. Under network settings, I found one called Network Interface Bonding. (Aren’t you glad I taught you the synonyms now?) With bonding unchecked, it showed 2 interfaces, and down below it showed the two IP addresses of the Drobo. I checked the box to bond the interfaces, and then it showed only one IP. Yay! We’re cooking with gas now!

I switched back to the router settings and it still said that port aggregation status was Inactive. I rebooted every device I could think of and I simply could not get aggregation to be Active. I finally resorted to writing to my technical contact at Drobo. He got back to me right away and explained that their port aggregation solution was not LACP 802.3ad. They’re going to migrate to that later this year, but right now simply disabling it on the router and enabling it on the Drobo will make it all work.

Time to Start Testing!

Finally, it was time to run some tests to see if this port aggregation thing was really all that and a bag of chips. On the Netgear router, they explained that you had to be careful which ports to hardwire Ethernet for your two computers to truly test the speed. They said,

To get the fastest performance with port aggregation, for wired connections use Ethernet port 3 for the first computer, and use Ethernet port 4, 5, or 6 for the second computer. This will allow for speeds up to 2 Gbps with port aggregation. The maximum speed between 4, 5, and 6 is limited to 1 Gbps.

I plugged my 2016 MacBook Pro into one port and Steve’s 2017 iMac into the other and we were ready to go. I created a 1.3 GB zipped file for us each to transfer to the new Drobo 5N2. I figured I’d test the 5N2 with and without port bonding and also test the 5N which doesn’t have port bonding.

I needed a way to accurately measure the speed to sync a file over, so I decided to use rsync. This is a linux command you can use via the Terminal. With a couple of flags set, I would be able to get human-readable stats while the file transfer was in progress and a final report after each transfer.

rsync -a --progress --stats --human-readable path_to_source path_to_dest

Steve and I started running tests by me yelling, “3 … 2 … 1 … GO!” Over and over and over again. We ran dozens of tests, and I compiled them all into my beloved Excel. However, our results were very odd. Let’s do a little math to set the stage.

We were using Gigabit Ethernet, which is 1000 Mb/sec. There are 8 bits in a Byte, so 1000 Mb/sec is 125 MB/s. In theory (and we know theory is always wrong) we should get 125 MB/s transfer from one Mac to either Drobo as long as only one of us was writing or reading.

In our tests though, we are seeing less than 50 MB/s, so not even half that speed. Now I know that about 128 different things can bring that speed down (as I read in an article on Tom’s Hardware guide) but we shouldn’t be at less than half speed.

The other odd thing was that when Steve and I were both writing to the Drobo 5N, Steve’s much more powerful iMac would drop down to 26 MB/s.

Now with port bonding, we should both be getting more than 100 MB/s, but oddly there was no speed increase at all with port bonding. The two of our read speeds combined went from 87 MB/s without bonding on the 5N to 97 MB/s on the 5N2 with port bonding.

Now it was time to bring in the big guns. They connected me with Rod Harrison, the CTO of Drobo, with whom we’d spoken several times at CES and NAB. I sent my results to Rod with my test equipment description, a diagram of my network setup, my test procedures, and my results. Can you tell I’m an engineer?

Rod’s initial reaction was that the individual speeds were way too low, again that we should be above 100 MB/s. He had me run log files from the Drobos as I ran more tests and he analyzed them. He explained, “An imbalance on the transmit side like that is indicative of the bonding sending out of the same port, the receive side was in balance” and he suggested restarting one of the Macs to fix it.

Rod had his team run similar tests and found that they were able to replicate some of the performance drop. They found a problem that caused a performance problem only with Mac clients, not with Windows. It was fresh news and they’re working on a fix for it and they expect to have out sometime soon. But then he figured out a much bigger problem in my testing.

After a few more tests, Rod suddenly realized why my speeds were overall so much slower than expected. He apologized for not noticing sooner, but he explained that rsync is notoriously slow! I did some hunting online for verification and found a super nerdy article at lwn.net/… that walked through a bunch of tests and geeky explanation. I would summarize their findings like this. rsync uses three processes: one process does writes, one does reads, the two stop what they’re doing and lean on their shovels and chat about it between them from time to time, and then the third process is the supervisor who stands around talking to them about the game last night.

Luckily Rod didn’t just say “don’t use rsync”. He suggested a tool called AJA System Test Lite, a free app in the Mac App Store designed for testing network attached storage speeds for people using them to write and read video. I’d used AJA’s local test years ago for testing an OWC SSD, so I was familiar with them.

Steve and I both installed AJA System Test Lite and got back to testing. AJA System test has two dials for write and read speeds. If you tap the little graph symbol at the bottom you can see a graph of the speed over time when the test is done. The left side lets you choose the size of the file copied back and forth, the codec, and of course the target disk. We left everything at the defaults.

Sure enough, just as Rod had predicted, we got great speeds with the new testing method. By myself, I could read and write to the 5N2 at around 92MB/s and Steve with his Mac could read and write at around 106MB/s. If we compare that to our speeds running one at a time to the older 5N, it was actually twice as fast. But we really want to know if port bonding allows two users to go twice as fast if they access the Drobo at the same time.

Now that we had an accurate way to test, we ran three scenarios:

• The older 5N which doesn’t have port bonding
• The newer 5N2 with port bonding disabled
• The newer 5N2 with port bonding enabled

We ran the tests from both computers at the same time and then added our speeds together to come up with a combined speed comparison.

If we compare the 5N2 without port bonding enabled, it was 37% faster in writes than the 5N and 11% faster in reads with two clients pushing data at the same time. But with port bonding turned on, the 5N2 was 53% faster in writes and 50% faster in reads than the 5N.

50% faster is a huge improvement and definitely makes the 5N2 with port bonding a great advancement over the 5N. Remember that the value of port bonding is when you have more than one computer accessing data from the same Drobo at the same time. It doesn’t hurt you at all if only one computer is accessing data, but it isn’t an advantage either.

I still had a question back to Drobo though, because the 5N2 with port bonding being 50% faster than the 5N without it doesn’t match up with their claim that the 5N2 is twice as fast. Marketing first answered me by saying that the 2X speed increase is if you put SSDs into the 5N2, which isn’t a fair comparison at all.

The CTO came back with a better answer though. He said that with jumbo frames turned on (and good spinning hard drives), they have measured the speed increase to be 2X. Well gosh darn it, what the heck are “jumbo frames”?

Drobo has clear articles explaining how to enable jumbo frames under the heading of modifying MTU settings. but what’s an MTU? In a little popup they explain jumbo frames as follows:

Jumbo frames maintain data transfer rates of more than 1,500 bytes per frame and can carry up to 9,000 bytes per frame. Enabling jumbo frames minimizes transaction loads and can dramatically improve network performance by raising effective bandwidth.

And on defining MTU they say:

Maximum Transfer Units – the maximum number of bytes that can be transferred at a time, in one frame.

Ok…if that’s a good thing to do, why aren’t jumbo frames always enabled? Off to the Googles. Wikipedia wasn’t much help to me, but Netgear actually had a great post entitled Guidance on the use of jumbo frames. They explained why you’d want to use jumbo frames (network performance) but also explained that you should be very careful when using them. They said that you can actually slow your network down or even break it if the MTU of one device is different from the MTU of another. They advise that you should only enable jumbo frames if you can isolate the NAS on a separate LAN or VLAN.

At this point, I decided I was done testing and I’d take their word for it on the whole jumbo frame/MTU idea.

Bottom Line

I really enjoyed learning so much about how to accurately test the speed of different NAS devices and was pleased with the end results of the 5N2. It’s an expensive device, but as you can see I’ve had three of them now (investing half of the cost with my own money) so I’m a believer in the devices.

My next step is figuring out a way to have the 5N2 automatically sync to the 5N. I’ve got a solution working but I think we’ll save that for next time!