Recovering data after a NAS failure
Article By : Brian Dipert
Although this engineer has long been a RAID advocate, one of his devices bit the dust in a way that RAID wouldn’t have circumvented.
As regular readers may already recall, I’ve long been an advocate of network-attached storage (NAS) for home and office use alike, both as a common (and no-brainer) destination for backups of computers and other devices on the LAN and as a nexus for music libraries and other files shared over the LAN. And as those same readers may also recall, I’ve long been an advocate of RAID-configuring the hard drives in those NAS devices, at minimum in a two-drive RAID 1 mirrored setup that supports recovery from failure of any one HDD, and ideally in a three-or-more RAID 5 (or X-RAID, etc. proprietary alternative) arrangement that also supports performance-boosting striped access.
Well, one of my NAS devices just bit the dust, in a way that redundant array of independent disks (RAID) wouldn’t have circumvented. And in retrospect, I had early warning of its demise, which I’d completely forgotten about. My NETGEAR (formerly Infrant) ReadyNAS NV+ had been running pretty much 24/7 since early 2008, per my email and blog post archive “notes,” solidly so with the exception of a few quirks. It and its ReadyNAS Duo sibling, both “v1” units (SPARC-based vs Arm-based v2 successors), are powered through a beefy UPS in case of transient premises power loss. We had one such power outage a couple of weeks ago, and after five minutes or so without electricity, I decided to “gracefully” shut down both NAS models in a controlled manner via their web interfaces (my router and intermediary switches are also UPS-backed, therefore were still running at the time) versus letting the UPS batteries drain (which at minimum would have resulted in a lengthy RAID verification upon reboot, and worst case in data corruption).
Xcel Energy got the power restored an hour or so later, and I went downstairs to the furnace room to turn the storage devices back on. I hit the power button on the ReadyNAS NV+ and saw the front panel LEDs reassuringly flicker, then immediately turned my attention to the ReadyNAS Duo next to it. After I got the Duo started up, I looked back at the NV+ and noticed that its front panel lights were now extinguished again and I heard no usual sounds of HDDs being accessed or of an active cooling fan spinning. Uh-oh.
Turns out, mine was seemingly a fairly-common problem. The NAS would continue to run as long as it remained powered up, but the initial incremental surge current of a power-on sequence would (I’m surmising) clobber its power supply. Apparently, according to the NETGEAR user forum archives, the company for a time got its hands on a particularly bad batch of power supplies, which it later replaced under extended warranty at no charge to its customers, but that program had long ago wrapped up by the time mine met its demise.
About that foreshadowing I mentioned a few paragraphs back: in 2014, a ReadyNAS NV (the precursor to the NV+) had failed in exactly the same way, as I documented at the time in a post that I’d frankly completely forgotten about until I stumbled across it in the process of writing this one. Why had my earlier situation faded from memory? My advanced age, perhaps, but in this case likely due primarily to the fact that the failure hadn’t affected me so significantly. The ReadyNAS NV was in California, I was by that time full-time in Colorado (preparing to marry and merge households), and an updated version of its stored data set was already on the ReadyNAS NV+. So instead of trying to resurrect it, after pulling the HDDs out of it, I just recycled it.
This time I wasn’t so lucky. Hard drive redundancy was of no help if none of the hard drives would even power up. And of course I hadn’t been regularly backing up the NAS to other archival storage, either direct-connected via USB or elsewhere on the network. My voluminous music library could painstakingly be recreated, after doing a lot of CD re-rips and MP3 etc. re-downloads (along with, inevitably, some re-purchases). But there was lots of other data on the NAS, stretching back decades, that was irreplaceable. I had to get the ReadyNAS NV+ running again, even if only long enough to copy the contents elsewhere.
Turns out the solution wasn’t quite as painful as I’d feared. Others in my shoes had already figured out how to hack a standard ATX power supply, for example. Or you can do what I did and acquire an aftermarket replacement that’s claimed pinout- and form factor-compatible. Apparently, according to the NETGEAR user forum archives, these things originally cost a couple of hundred dollars, believe it or not; fortunately, I found one for “only” $69.49 on Ebay. It wasn’t a perfect replacement; the measurements were a bit off, and several of the mounting holes didn’t exactly line up with their chassis counterparts, but I made it work. And, after saying a short prayer and pressing the power button, I was relieved to see the ReadyNAS NV+ boot sequence begin once again.
While this isn’t a formal teardown, I thought I’d share some before-and-after photos of the original power supply and its replacement. Here are some overviews of the original after I pulled it out of the NAS:
And here are the same-perspective shots of its successor:
One thing you’ll likely notice right away is that the new PSU includes its own fan, whereas the original was fan-less (said another way, it relied on the main NAS system fan for active airflow). I have no idea whether component overheating led to the original PSU’s failure, therefore whether the new one will last any longer. And a look inside the original isn’t any more enlightening, though it’s still educational.
Here I go, disobeying orders again:
I obviously have no idea whether or not the solder flux remnants and other staining in this last photo were there from the beginning or are left-over evidence of subsequent failure.
I was surprised to see no obviously swollen capacitors or any of the other usual suspects. Thoughts, readers?
After quickly copying over the ReadyNAS NV+ contents to the Thunderbolt drive attached to my mac Mini, I powered both it and the ReadyNAS Duo off and have retired them both to the storage closet. While the ReadyNAS Duo was still running fine (and leverage a more easily replaceable external PSU, to boot), both were getting long in the tooth. My various browsers had long complained about their self-signed certificates, for example. They only supported v1.0 of the SMB protocol, which Microsoft had begun blocking in Windows 10 Home Edition (although a workaround was available) and was also deprecated from Chrome OS. And Mozilla Firefox, my primary browser of choice, was also starting to balk at their archaic TLS 1.0 support, not to mention the fact that neither NAS supported per-HDD capacities beyond 2 TBytes.
What did I replace them with? That’s the subject of my next post. Until then, I as-always welcome your thoughts in the comments!
This article was originally published on EDN.
—Brian Dipert is Editor-in-Chief of the Edge AI and Vision Alliance, and a Senior Analyst at BDTI and Editor-in-Chief of InsideDSP, the company’s online newsletter.
Related articles:
- Beating the heat: NAS’s CPU requires a critical mass of software in order to compete
- Accelerating consumers’ NAS adoptions: assessing your product options
- A slick upgrade to industry-standard RAID
- Tech expired and exchanged: The only constant is change
- Product quirk almost takes down NAS
