company location

Get In Touch

Why not join us on Twitter or Facebook? We only post about useful things like new releases, and unlike some others we promise we won't spam you with meassages!

           MLtek Software on Facebook        MLtek Software on Facebook

 

             File Server Archiving Software - Archive Manager

Archive Manager from MLtek
 

 

Is archiving your file server to the cloud a good idea?

Home | About Us | Products | Support | Contact Us | Library

Why would anyone consider archiving old files from thier file server?

We normally get asked at least once a week by a potential customer if Archive Manager can archive to the cloud. The answer is always a resounding yes. However, we advise people to think this through very carefully before they embark on such a project as costs can quickly spiral.

Given the frequency with which we get asked this question we decided to write this article to help people in the decision making process.

The decision to deploy a file server archiving solution brings multiple advantages, including the following:

  • If you migrate all your old files to second line storage you free up space on your expensive high performance first line storage thereby increasing it's lifespan before you outgrow it.
  • Backups are faster and you need to back up less data. Archived data hardly changes unless an archiving action is performed and additional files are archived.
  • Restores are quicker. In an emergency you can restore just the frequently used files first and everyone gets back up and running sooner.
  • Compliance. Many countries now have legislation that places obligations on companies to only keep data for as long as it is needed. A good archiving solution will help ensure you comply with the relevant legislation.

There are other drivers but these seem to be the main reasons our customers decide to deploy a file server archiving solution like Archive Manager.

Could archiving to the cloud be the answer?

At first glance the idea of offloading all of your old and unused data to someone else's infrastructure seems like a great idea, and maybe it is. You don't have to spend time sourcing additional hardware, putting together project teams, updating infrastructure and supporting documentation, deploying physical kit and all of the other activities that go along with such an activity. In this example we'll use an 'average' file system. Obviously everyone's file system is different, but a lot of our customers file systems seem to fall in the range of 20TB to 50TB of total storage.

Now I know there is obviously selection bias in those numbers, after all this represents OUR customers. Our customers tend to represent the upper end of the spectrum of corporate file systems, the largest file system is 3.4 PetaBytes and the average tends to be around 20TB to 50TB of total storage. As that is the range we know we'll go with 30TB which is pretty much smack bang in the middle.

What do you need to start archiving to the cloud?

The first thing you'll need is somewhere to archive too! In this example you'll need around 20TB of storage to make a serious dent in the live file system. Pricing was obtained early in April 2017 and as the providers below are always launching new products you should probably check for yourself just in case they have recently released new VM sizes or storage options.

Each entry represents the best option we could find to obtain 20TB of storage with each provider.

Amazon AWS's offering get's you around 20TB of storage for $2,200 per month.

VM Name vCPU RAM Instance Storage Price Per Hour
d2.4xlarge 16 122GB 12 x 2TB HDD $3.062 p/hour

How about Microsoft Azure? Better.. a similar amount of storage for $718.40 per month.

VM Name vCPU RAM Instance Storage Price Per Month
A3 4 7GB 22x 1TBS30 HDD $718.40 p/m

And lastly, Google Cloud. About 20TB for $998.42 a month

VM Name vCPU RAM Instance Storage Price Per Month
n1-standard-4 4 15GB 22x 1TBS30 HDD $998.42 p/m

With the figures above in mind the best option on the table looks to currently be Microsoft Azure. There may well be other charges involved like bandwidth usage charges, VPN charges and others. Even if one took the Azure option you could be looking at nearly $1000 a month with the extra charges involved.

Customers tend to quickly realise that for under $10,000 as a one off cost they can deploy a glorified NAS device with >20TB of storage onsite. The conversation then almost invariably turns to a request for a recommendation on a suitable device that is tried and tested alongside our software.

So far only one customer has gone ahead with a cloud archiving project, and that was because a) their file server was about to run out of space and b) they didn't have any space in thier server rack for a new device so they would have needed to open up another rack at nearly $1300 a month. They were planning on decommissioning some kit in the near future, so when they have created some space they will be deploying a NAS device and migrating all their archived files back in house.

But if I really REALLY want to archive to the cloud, how would I do it?

The process of setting up cloud based file archiving is actually really simple! Let's take a look:

  1. Sign up to the hosting provider of choice and create a new VM with the appropriate specification (don't forget the extra drives!).
  2. Create a site to site VPN from your datacenter to your cloud platform.
  3. Create additional VPN's from your users sites if required so they can access the cloud server.
  4. Set your cloud servers DNS to point to one of your Active Directory Domain Controllers.
  5. Join your cloud server to your AD.
  6. Configure the disks that are attached to your cloud server as one volume (at last a use for software RAID!).
  7. Create a new share on your cloud server called 'Archive'.
  8. Download and install Archive Manager on your Cloud server.
  9. Create a job to archive all of the old content from your live file system to your cloud server.
  10. Watch the used space on your live file server plummet.

The process is really simple and can be carried out in a just a few hours (initial data sync aside). The problem isn't the technology, it's the pricing.

Driven by the number of enquiries we receive, we are actively working on a solution to this and are carrying out a fesability study on setting up a dedicated hosting service. Initially it would be based in the UK, but if it does go ahead and if it is popular enough we will expand it to other countries. We are hoping to hit much lower price points than any of the big players for storage provided specifically for archival purposes when used alongside our software.

The idea is to give customers their own segregated private VM with the potential for 100's of TB's of storage to be attached.

If you have any questions or you would like to express an interest in our potential hosting service please don't hesitate to contact us via sales@mltek.net, we would love to hear from you!

Mark Laverty
MLtek