One way to overcome a lot of the challenges and concerns with storing large amounts of unstructured data is to move it to a public cloud provider. As for the challenges of legacy NAS filers, most of those concerns would no longer be applicable. Essentially, you would be paying the cloud provider to worry about all the infrastructure concerns like capacity, performance, uptime, H/W support, and so on.
Your company would still have to deal with the day-to-day operational management of these data sets in the cloud, but more and more cloud-ready solutions and applications that can make this task much more manageable exist now than ever before. Even some of the compliance, retention, security, and data privacy requirements could be addressed once you are in the cloud, as well. The marketplace is actually flush with data analytics and cloud security and compliance solutions specifically designed for cloud-based workloads.
So why not go all-in on public cloud for your unstructured data? Cost is one obvious answer. The shift from CapEx spending to OpEx spending that comes with embracing the public cloud model is very attractive to most organizations initially, but at the end of the day, the company is still spending the money one way or the other. For some use cases, the cloud makes perfect sense—for example, to spin up a new application and grow its underlying cloud infrastructure as needed via the “pay-as-you-go” model.
But as it concerns large unstructured data sets that will need to be continuously accessed, analyzed, or even pulled back on-premises from time to time, the economics of cloud starts to become less and less viable for most organizations. Most cloud providers make it very enticing economically to save data up to their cloud but charge additional fees to actually access that data. This fact has made it fairly expensive to use the public cloud to store unstructured data sets if a company truly wants to leverage that data for analysis.
New and unique skill sets must also be acquired by IT staff in order to truly optimize a cloud infrastructure. For the younger organization that was “born in the cloud,” this is less of a challenge, but for the more established businesses that have already invested in on-premises data centers, adding cloud infrastructure requires new skill sets in order to maintain and integrate both on-premises and off-premises environments.
I do not intend to turn this into a cloud-bashing blog post, so let me take a moment to be perfectly clear on one point: Cloud is the future and is not going away. It absolutely has its benefits, and any IT decision-maker that fails to embrace the public cloud in some form or another is eventually going to find themselves behind the 8-ball. That being said…
To address the unique challenges of housing today’s large unstructured data sets, what really is needed is to have the best of both worlds.
Customers looking to design storage repositories to house large unstructured data sets need the scalability, accessibility, and application compatibility of the cloud—but via an on-premises solution that you have tighter control over, albeit with the ability to burst or tier to the public cloud as needed. The answer is to move away from legacy NAS filers and file systems toward a more modern solution that is cloud-ready and built from the ground up to handle today’s modern unstructured data challenges. The key component that allows the modern file storage solution to accomplish this is the ability to provide for what is known as object storage.