Endy:Data storage

From OpenWetWare

(Difference between revisions)
Jump to: navigation, search
([http://itinfo.mit.edu/article.php?id=6628 MIT TSM Backup Service])
(Types of data)
Line 34: Line 34:
*Individual user data
*Individual user data
**active:
**active:
-
***stored on: Bionet, lab computers or both (easy to access, backed up)
+
***stored on: Bionet (easy to access, backed up), some on lab computers
***size: ?
***size: ?
**inactive:
**inactive:
-
***stored on: lab computers (hard to access, not backed up)
+
***stored on: Bionet (easy to access, backed up), some on lab computers, including shmoo (~10GB?)
***size: ?
***size: ?
*Project data
*Project data

Revision as of 13:53, 29 January 2007

Contents

1. The problem, briefly stated

We need an easy, secure and efficient way to store all our files:

  • individual user files (backup)
  • shared project files (centralized storage and backup)
  • old user and project files (centralized storage and backup)

2. Current specifications our backup system

Most people use Bionet to backup their files.

Bionet

  • Storage space (as of 2007-01-16):
    • total: 110GB
    • used: 86GB
    • available: 24GB
  • Where is it physically located?
    Building 68, 3rd floor
  • Are Bionet files regularly copied and stored somewhere else?
    Yes, to the MIT datacenter in W91.
  • Problems with Bionet:
    • Not enough space to backup all our files e.g. microscope images
    • Isn't Bionet not going to be backed up anymore?
      This is currently (2007-01-23) unknown, will depend on the allocated budget.

3. Ideal specifications of our future backup system

(lab and individual data storage, sharing and backup needs - please list what would you like have available)

  • Capacity: we want to be able to store all files in a single location
  • Easy: automatic backup
  • Secure: the backup system shouldn't be located in building 68, in case of a fire
  • Efficient backing up or retrieving files should be speedy
  • Affordable

Types of data

  • Individual user data
    • active:
      • stored on: Bionet (easy to access, backed up), some on lab computers
      • size: ?
    • inactive:
      • stored on: Bionet (easy to access, backed up), some on lab computers, including shmoo (~10GB?)
      • size: ?
  • Project data
    • active:
      • stored on: Bionet (easy to access, backed up)
      • size: ?
    • inactive:
      • stored on: Bionet (easy to access, backed up)
      • size: ?
  • Microscope data
    • stored on: lab computer
    • size: 170GB

4. Potential solutions

The win.mit.edu Domain

MIT TSM Backup Service

  • Monthly service charge: $7.50 per month per computer
  • Storage limit: 300GB
    • a soft limit, some users go over
    • an approximate figure because it includes both "active" and "inactive" files but this is offset by data compression
  • TSM software is required to use the service and is available for Windows, Mac and Linux (free to MIT community per site license)
  • Backups are stored on one of the TSM backup servers in buildings W91 and E40 (not redundant)
  • Types of backup:
    • Scheduled: everything by default but can be configured to exclude directories
    • Manual: nothing by default, need to specify which directories to backup
  • Inactive files (old versions of current files and deleted files) are kept for 30 days using incremental storage (only changes are stored)
  • Need a separate account for each computer to be backed up
  • 5,000 users, 250,000 files restored per quarter
  • coming soon (summer of 2007 at the earliest):
    • free service:
    • enhanced service:

Network Attached Storage

A Tale of Two Terabyte NAS Boxes

Buffalo Technology

  • Buffalo TeraStation Home
    • Example disk configuration: 4 x 250GB IDE (750GB in RAID5)
    • Protocols: FTP, SMB
    • USB 2.0 port for external hard drive (backup or additional storage)
    • Review by PC Magazine
      • Bottom line: Flexible and reliable storage for everyone on your network. Print sharing is a plus, as is expandable USB disk storage.
      • Pros: Offers RAID level data protection; easy-to-configure shared and private storage for all workgroup members; print sharing is a plus.
      • Cons: Large footprint. No logging or reporting features.
    • Review by ExtremeTech
    • TeraStation wiki
  • Buffalo TeraStation Pro
    • Released in March 2006
    • S-ATA drives

Infrant ReadyNAS

Personal tools