Endy:Data storage

1. The problem, briefly stated

We need an easy, secure and efficient way to store all our files:

• individual user files (backup)
• shared project files (centralized storage and backup)
• old user and project files (centralized storage and backup)

2. Current specifications our backup system

Most people use Bionet to backup their files.

Bionet

• Storage space (as of 2007-01-16):
  • total: 110GB
  • used: 86GB
  • available: 24GB
• Where is it physically located?

  Building 68, 3rd floor

• Are Bionet files regularly copied and stored somewhere else?

  Yes, to the MIT datacenter in W91.

• Problems with Bionet:
  • Not enough space to backup all our files e.g. microscope images
  • Isn't Bionet not going to be backed up anymore?

    This is currently (2007-01-23) unknown, will depend on the allocated budget.

3. Ideal specifications of our future backup system

(lab and individual data storage, sharing and backup needs - please list what would you like have available)

• Capacity: we want to be able to store all files in a single location
• Easy: automatic backup
• Secure: the backup system shouldn't be located in building 68, in case of a fire
• Efficient backing up or retrieving files should be speedy
• Affordable

Types of data

• Individual user data
  • active:
    • stored on: Bionet (easy to access, backed up), some on lab computers
    • size: ?
  • inactive:
    • stored on: Bionet (easy to access, backed up), some on lab computers, including shmoo (~10GB?)
    • size: ?
• Project data
  • active:
    • stored on: Bionet (easy to access, backed up)
    • size: ?
  • inactive:
    • stored on: Bionet (easy to access, backed up)
    • size: ?
• Microscope data
  • stored on: lab computer
  • size: 170GB

4. Potential solutions

The win.mit.edu Domain

• Pros and cons

MIT TSM Backup Service

• Monthly service charge: $7.50 per month per computer
• Storage limit: 300GB
  • a soft limit, some users go over
  • an approximate figure because it includes both "active" and "inactive" files but this is offset by data compression
• TSM software is required to use the service and is available for Windows, Mac and Linux (free to MIT community per site license)
• Backups are stored on one of the TSM backup servers in buildings W91 and E40 (not redundant)
• Types of backup:
  • Scheduled: everything by default but can be configured to exclude directories
  • Manual: nothing by default, need to specify which directories to backup
• Inactive files (old versions of current files and deleted files) are kept for 30 days using incremental storage (only changes are stored)
• Need a separate account for each computer to be backed up
• Performance will vary, depending on time of the day, network condition and machine itself)
• 5,000 users, 250,000 files restored per quarter
• 128-bit encryption available
• coming soon (summer of 2007 at the earliest):
  • free service (for personal use): 10-20GB
  • enhanced service (for DLC use): 1TB and up, offsite mirroring, will be expensive, etc

Misc

• FAQ
• Software client manuals
• Security notes
• TSM administration interface

Network Attached Storage

A Tale of Two Terabyte NAS Boxes

Buffalo Technology

• Buffalo TeraStation Home
  • Example disk configuration: 4 x 250GB IDE (750GB in RAID5)
  • Protocols: FTP, SMB
  • USB 2.0 port for external hard drive (backup or additional storage)
  • Review by PC Magazine
    • Bottom line: Flexible and reliable storage for everyone on your network. Print sharing is a plus, as is expandable USB disk storage.
    • Pros: Offers RAID level data protection; easy-to-configure shared and private storage for all workgroup members; print sharing is a plus.
    • Cons: Large footprint. No logging or reporting features.
  • Review by ExtremeTech
  • TeraStation wiki
• Buffalo TeraStation Pro
  • Released in March 2006
  • S-ATA drives

Infrant ReadyNAS

• ReadyNAS NV
• ReadyNAS NV+
• Infrant ReadyNAS NV+ and 1100: Small steps forward - review
  • comes with a 5+5-user license for EMC's Retrospect for Windows and Macintosh client backup software
  • The NV+ is a slight improvement over the NV, with most of the value coming in the Retrospect backup client bundle
  • Since both the NV and NV+ use the same processor and have the same memory, the performance difference I saw is more due to better drives in the NV+ and newer firmware than anything else
  • with the lowest price at time of review at $831 for a driveless NV+ and $517 for an NV, you might be better served by using the $300 to buy drives
• Infrant ReadyNAS NV Review
  • X-RAID (Expandable RAID) allows to add capacity without deleting existing data, automatically adjusts RAID level and formatted capacity to match the available drives
• ReadyNAS NV - user review
• ReadyNAS NV - AnandTech review
• ReadyNAS NV - PracticallyNetworked.com review