A few weeks ago I talked about the long adventure I’d been through with Lawrence from CrashPlan trying to help me adopt my off-site backup from my 2013 MacBook Pro to my 2016 MacBook Pro. I explained that he’d worked really hard but never could get a backup running for me.
I tested out Backblaze, and found that in just 18 hours it had successfully backed up my new Mac. I gave Lawrence the link to the blog post about what happened, and you would think the story would end there, but you would be wrong.
Lawrence kept working on my backups. He eventually handed me off to a guy named Brandon. Brandon went to great lengths to make sure I understood that everything Lawrence had done was right, and that he had not made any mistakes. But Brandon did one thing completely new. He asked to talk on the phone and to use screen sharing. I had thought partway through my long relationship with Lawrence that a phone call would have sped up understanding, but like any good little nerdlet I try really hard never to talk on the phone.
My curiosity was peaked and I agreed to talk to Brandon. As much as I like to drag out a story, I’m going to give you a spoiler. In a couple of hours of working on my backups, and a couple of days to allow some maintenance to run on CrashPlan’s end, I have working backups at CrashPlan. I’m sure some of you are saying to yourself that you’d have abandoned this before this long, but what I learned from Brandon is really interesting.
Let me explain what Brandon and I did to tailor my backup first, and then I’ll read you his explanation of why we had to do this. Then I’ll explain why you really do care about this if you’re a Backblaze customer.
In the CrashPlan software, Brandon had me modify what files could be excluded from my backups. I had chosen to back up pretty much every single doggone file on my Mac. I thought that was what I was supposed to do. He started in the System Library and he had me uncheck it at the top level. You know how I always say to clean your closet you have to remove everything and only put back what you need? That’s what Brandon believes too so it’s how we started working on my System Library.
He emphasized that I could put back anything I thought I needed. He was super knowledgeable about which files were just cache files, or app-generated files that would be reproduced when those apps got reinstalled after a catastrophe. In the System Library the only thing I had created myself were some filters to create smaller-sized PDFs with good quality images (I have a tutorial on that of course).
We then followed the same process in my user Library. In there we found a few more things of interest. In particular, many applications store files inside your Library/Application Support folder. I remember ages ago after I’d done a clean install and discovered that my database of all my DVDs was gone. I store them in the awesome DVDedia, and it turns out they store the database inside that Application Support folder. Luckily I had a local backup that had the file.
We started going line by line through the Application Support folder. Audio Hijack stores your session files as plists in there, Clarify stores your custom-made templates in there, Feeder keeps my podcast feed files there, Hazel, MarsEdit, Nicecast, SuperDuper!, Telegram, TextExpander, Transmit, and Auphonic all keep your tailored settings in there. Sure I could reproduce them, but Brandon said I should back them up.
I just realized while writing this up that I have another question for Brandon. If I add a new application and it stores files in the Application Support folder, I’m not sure whether CrashPlan will back it up by default or whether I have to specifically add it back in. That’s going to be hard to remember! Maybe I can just back up that whole folder.
It sounds like I’ve listed a TON of stuff to keep but there is so much more in there I didn’t need. For example, the MobileSync folder is in your user Library. That’s where Apple stores the backups of your iOS devices. Surely there’s no reason that I would need to back those files up to an offsite backup service.
We also took a stroll through the user Library/Containers directory and I chose to keep a few directories in there. I did a little bit of research on the Containers directory and it seems that some companies keep their databases in there. For example, I found that Things and Omnifocus both keep their databases in Containers. Definitely good to look in there.
I promise to get back to Brandon’s explanation of why this tedious task was necessary but I want to bring Backblaze into the story first. I got an email from Lewis Butler pointing out that Backblaze doesn’t back everything up. He didn’t elaborate, so I did some investigation and I found some interesting bits of information you might find useful in making your own offsite backup decisions.
Let me read the explanation of excluded files from the Backblaze website:
Backblaze does not want to waste your bandwidth or Backblaze datacenter disk space. Thus, we do not backup your operating system, application files, empty folders/directories or temporary internet files that are transient and would not be useful in the future. You can see these exclusions by clicking on “Settings…” in the Backblaze Control Panel and selecting the Exclusions tab.
Lewis Butler later pointed out there are some directories that if you’re an uber geek (like many of us) might care about. Areas like /etc, /sbin and more are in the exclusions list. I’m not sure why CrashPlan does back up your applications, but Backblaze specifically prohibits that too. You cannot modify this exclusion list.
Backblaze does have a list of file types that they don’t back up, but you can actually override those. For example, if you like to save disk image files, either ones you’ve made yourself or application installers, you have to remove .dmg from the file type exclusion list. Cool that you can remove the exclusion, but if you don’t look at that list closely you could get a very sad surprise. I know people make encrypted disk images on their Macs to store sensitive information so that could be an unwelcome surprise if you didn’t pay attention while setting up Backblaze.
One of the great things about working with Brandon at CrashPlan was that he was more than willing to explain things to me. He’s pretty verbose in fact. Instead of parsing all of the conversations we’d had myself, I asked him if he’d explain what he did and why. It’s a little long but it’s really interesting so I’ll read you the whole thing. Brandon from CrashPlan wrote:
Though we are unlimited in terms of archive space, we’re not built to back up everything that people keep on their machines. Specifically, CrashPlan wasn’t designed to back up system/application/temp/cache/log/etc files. As such, when those file types are included in one’s file selection, due to the rapidity at which they change (usually), they start to fill up an archive—not in size as much as sheer number.
For you, there were literally millions of these files floating around in your archive dating back to ’13 (many worthless in terms of restore purposes). Again, archive size not really being an issue, CrashPlan had to keep track of all those files.
The kicker for you had to do with CrashPlan also having to track the versions of all those (unnecessary) files. This effectively swelled your archive’s index files (ie. the files that keep track of all of this stuff) to many times their normal size. Think about a 3-4 GB text file that needs to be transmitted between our server and your client and traversed for the purposes of de-duplication and retention purposes.
Needless to say, though things can still work for you (sorta), it does create somewhat of a bottleneck so that normal things like archive maintenance can’t take place/be effective as they should. The fix was simple, scale back your versioning a bit, remove all the system/application/temp/cache/log/etc files from your archive then push it through archive maintenance to pull that data out of your archive.
You were fixed when this process completed. The rest was just the ‘initial’ backup on your new machine.
I thought Brandon gave a great explanation. With literally millions of these files no one cares about, the process of comparing every single one to what I have now just to figure out what to back up was taking so long that the backup simply never could start. This was exacerbated by the fact that CrashPlan checks newer, smaller files first, and newer, larger files second. That would include all of these silly changing cache files!
Conclusions
Let’s see if we can wrap this up with some conclusions.
- I give CrashPlan HUGE props for staying with me on this. They dedicated a huge amount of time to me, and even kept going after I sort of slammed them in my blog post. They were head down committed to getting this fixed and were never distracted by other events.
- CrashPlan backs up everything you ask it to back up, even stuff that doesn’t make sense to back up. Backblaze on the flip side restricts what you can back up. I’m not sure which path is a better way to go. Perhaps CrashPlan could be a little more verbose in their backup tool. If it had been suggested to me that perhaps I should reconsider backing up my entire system level Library, I probably wouldn’t have done that in the first place. Then again, I’m a geek and can understand the repercussions of those decisions and most normal people wouldn’t be equipped for that.
- Backblaze has way way faster backups. From my experience the process is unrestricted by anything other than your own upload speeds. CrashPlan in contrast will allow 10-30GB a day maximum. For most people that’s probably enough and for incremental backups it’s even enough for me, but that first backup at CrashPlan took 3 weeks where it took Backblaze a mere 18 hours.
The main conclusion I’ve come to is that both CrashPlan and Backblaze are great services with pros and cons. The good thing is that we have competition and have two excellent choices. And I want to close by thanking both Lawrence and Brandon for their commitment and help.
Thanks, Allison,
Your detailed recollection of your experiences with both services is much appreciated. I am a long time Crashplan subscriber, but, other than recovering the occasional file here or there, I’ve never had to recover from a major disaster.
In digesting your post, I’ve come to the conclusion that you are in a relatively “happy space” with either/both services. My problem is, without the detailed handholding you received…..many/most of us are in the “lost in space” place you were in.
I’d like to have one of these services (or their myriad competitors) offer a handholding set up. Given the critical nature of what it is that they do, it would go a long way toward instilling confidence that “what you need” is there when you need it. It might require an additional set up charge (or optional set up charge), but I for one, would be willing to pay it.
Cheers
Kenny In NJ
Good point, Kenny. Remember for 3 years I was having constant CrashPlan offsite backups and it worked flawlessly, and that was with no hand holding at all. It was also backing up more than I needed which I guess is a good thing.
If you do need/want hand holding on setting up backups, and you’re willing to pay a small fee, I think that’s where a member of the ACN (Apple Consultants Network) can come in handy. I know that Pat Dengler does this for clients with Backblaze all the time. Her email is [email protected]. She’s in California but a setup like this could certainly be done via Skype or Log Me In or any other screen sharing application. If requested, she’ll even run software to monitor your backups for you (but you have to agree to her having access to your data). Can’t recommend her highly enough.
Thanks, Allison. Thats a good plan. Thanks for the referral.