Guide to Photo Storage - Page 2


The Hardware

 Finally getting down to the nuts and bolts. We can set up some general guidelines here, that based on your current capacity needs and the expansion you will need later will help you select an enclosure and the drives to suit the need. From there you can choose from a couple really good solutions that will work spectacularly.

Firewire - USB

 For smaller capacity needs, where you can attach just a couple hard drives and get along fine, I advise using either Firewire or eSATA. Performance from Firewire is fine for a couple drives. As you increase the number of drives, eSATA begins to really show massive performance benefits. Firewire drives share in the bandwidth, the more drives, the less each drive gets. eSATA gives close to full bandwidth to each drive. So it can be a better choice for higher performance access, especially as you increase the number of attached drives.

 For iMacs, Mini and Laptops without an expansion slot, Firewire is THE choice. As long as you stick with Firewire Enclosures that can maintain a satisfactory environment for drive longevity then Firewire will work for you. No brick powered Firewire enclosure with little or no cooling will do though - stick with a Firewire Burly with its superb capability to maintain your drive and data health.

 USB just doesn't cut it. USB was never intended, nor designed, to attach and communicate with the high speed data transfers of hard drives. We always recommend that if you have a choice, use Firewire or eSATA. USB isn't efficient enough for this and should be left to keyboards, mice and card readers.

eSATA and Port Multiplication

 eSATA is the number one technology today when it comes to external storage. And the real power in eSATA is in Port Multiplier Enclosures. Port Multiplier (PM) technology is available for most every computer with an expansion port by use of Port Multiplier capable SATA host cards. Simple to use, expandable and fast, PM Enclosures are the way to go for most every photo database requirement. It has low cost benefits as well - with cost per GigaByte of complete storage system lower than any other attachement method.

 Port Multiplication is a technology invented by Silicon Image. It uses a port expander board inside the drive enclosure that acts as an extension of the host card in your PCI slot. (a few PC motherboards now include PM capable eSATA ports. Hopefully Apple follows suit in the future) That expansion board allows up to 5 hard drives to be attached - just as if they each had their own data cable - but all connected via one external data cable and occupying a single port on the host card. An amazing technology that is even more amazing in the simplicity of its use.

  Port Multiplier enclosures are the right choice for most photo storage requirements. Far and away the most versatile because of the ease of expansion. Fully hotswap capable and easily configured to run in either individually mounted (JBOD) drives or in Software RAID0 for speed and large capacity. Drives can be configured in any fashion imaginable - including combinations of RAID and JBOD.



Choosing JBOD or RAID

 There are two very simple configurations that we use and recommend for your storage. They both take the same hardware - this is simply two ways of setting it up for ease of use and performance.

Method 1 - JBOD

 The first method uses single JBOD drives with a matching identical drive for a backup. This system has the benefit of being able to roll off into archives the older drive pairs making room for new pairs and new data. This setup can be easily cofigured based on either labeling these as 'Jobs' disks or as a 'date range' of disks. I like to physically label the outside of the drive trays to keep track of them.

 The simplicity here is the benefit. Increasing capacity is just adding another pair of drives. Adding another enclosure doubles the available drive slots. With todays superb drives reaching a full TeraByte, and larger tomorrow, using a simple JBOD arrangement makes for nearly unlimited future expansion.

  Performance is somewhat fixed. This setup is fastest with an empty drive, in the 100 MB/sec range for todays big SATA drives. But as the drives fill, they also slow. If you are working with large image files, many layers and channels, compressed TIFFs, or high History settings - you may need more power. So onward to RAID, method 2.


Method 2 - RAID

 For performance hungry users (at MacGurus we are all seriously infected with speed addictions) a solution using drives configured into RAID0 arrays is perfect. As you add more drives to a RAID0 you not only gain tremendous speed increase but also get much larger volume size. Software RAID created with Apple’s Disk Utility, SoftRAID LLC or Windows Disk Manager have all proven themselves as very robust troublefree solutions. Today’s desktop hard drives have a unmatched reliability record. Between the two you can count on a RAID system to be quite reliable. Like any solution the backup gives needed peace of mind.

 As shown in the diagram at right, the first two drives are configured into a single RAID0 volume. Using 1 TeraByte drives this would give you a 2 TeraByte volume at very close to twice the speed of a JBOD drive. The third and fourth drives are configured into an identical RAID0 volume and designated as the backup. For scalability you can build either RAID using as many hard drives as you like. The more drives, the larger the capacity and the faster the performance. All drives should be the same size. Mixing drive sizes in a RAID is not advised as the RAID will only use the capacity of each drive equal to the size of the smallest drive.

 Here's some numbers to give you an idea of the performance advantages of using RAID0 arrays. My latest testing was with the new Seagate 7200.11 Barracuda 1 TB drives. Individually mounted in JBOD these drives were capable of just barely under 100 MB/sec (MegaBytes) read and write speeds. With just a pair of them in a Disk Utility RAID0 we brought those numbers to around 194 MB/sec read and write. These are tremendous performance numbers that will go a long ways towards improving your overall computer speed when manipulating, loading and saving your photos.

 Scalability is enormous. The simplest RAID0 with a backup is a 4 drive enclosure - 2 drive RAID0 database drive to an indentical 2 drive RAID0 for the backup. This is shown in the picture at right. To achieve larger capacities and greater speed, move up to a pair of 4 or 5 drive enclosures. Each of these enclosures would contain one of the RAIDs. A 4 drive RAID backed up to an identical 4 drive RAID. This setup is BIG, Fast and very cost effective. The improvement to your work flow from the added drive speed will blow you away.

Starting out small and expanding the RAID later

 You can start out with a 4 drive Burly - expanding it may be easiest by doubling it with another 4 drive enclosure. To add a second identical 4 drive enclosure and double your capacity, format the entire new unit into a single 4 drive RAID0. Then copy the data from the original 2 drive database RAID. Reformat the original 4 bay into a single 4 drive RAID0 and designate it for your backup.

 You can also just add a second 4 bay, with any drive size available at the time, and build a second completely separate storage volume with backup via the same structure as the first array.

Ordering the Right Stuff

 First thing, pick an enclosure. The one I like most is a Burly 5 Bay PM with standard trays and the default 2 meter cables. This enclosure is quiet, has big slow turning fans which move some air but don't make a lot of high pitched noise. With 2 meter cables you can keep it far enough away so you almost won't hear it. The Burly is made to run 24 hours a day letting you run automated nightly backups. 24/7 operation also lets OSX do its nightly maintenance on the system and log files. We set all computers to never sleep and drives are never allowed to sleep either.

 There are numerous uses for a 5th bay. Time Machine backups of the internal drives is probably one of the best. Also can be used as the hotswap drive for offsite archiving of those 'can't lose' photo databases. Even if you don't have a use for it now, it will come in handy.

Drives

 You can get the enclosure already populated and tested with Seagate or Hitachi hard drives. Or you can order with whatever combination of drives you like from our hard drive page. We will install and test them. Or you can get your own hard drives. If using your own drives please stick with desktop drives (Seagate and Hitachi prefered and tested and proven). Do not get Enterprise or Server class drives as the feature sets on these are designed for servers, not desktop computer or PM enclosures.

  And most important - avoid drives marketed as OEM! OEM drives can come wth any custom firmware on them imaginable. That firmware can cause literally anything to happen from spontaneous dismounting of your drives to unstable RAID arrays and very early drive failures. Seemingly extrordinary low priced sales on OEM drives is common and they are the most frequent cause of drive problems on desktop computers today! The price is not nearly low enough for the trouble these drives cause.

Host card

 You will only want to consider the PM capable 4 port host cards. The chipsets on the 4 port cards is capable of far more speed than any of the 2 port cards. If less speed is acceptable then the 2 ports cards may work for you. Like us, most users will want the speed! All these cards are Leopard compatible.

- Simple rules for which card to choose.

Computer Model	Host Card
- MacPro - G5 "Dual Core" with PC2-4200 RAM *	Choose - Lycom LYCeSATA-4e PCIe 4 Port SATA Host card
- PCI-X G5s with 8 RAM Slots and PC3200 RAM *	Choose - Lycom LYCeSATA-4X
- PCI G5s  - those G5s with 4 RAM Slots * - All G4s	Choose - Sonnet Tempo X4P
- MacBookPro	Choose - Lycom Express34 SATA Host Card
- PowerBook G4 with Cardbus Slot	Choose - Lycom Cardbus SATA Card

* Note: You can quickly check your model G5 by using Apple System Profiler and looking in the Memory Section under Hardware. The PC2-4200 speed reading will tell you you have a PCIe Dual Core G5. PC3200 under Speed and 8 total Slots (numbered 0 through 7) indicate you have a PCI-X G5. 4 RAM slot (numbered 0 through 3 in Profiler) indicate you have a PCI G5.

Conclusions

 Figure out how much capacity you require. Select an enclosure and the right host card. Installation is very simple, requiring a philips screwdriver for the host card and for any drives requiring installation in trays. It is recommended that you zero drives before putting them into production. This is the best way to test them. Check out our Guide to Preparing New Drives.

  Format into your choice of JBOD or RAID. Start building your database. For designing a system for Digital Asset Management there is a superb guide written by Peter Krogh called The DAM Book that we highly recommend. It will help you design an indexing system for your large photo database. Essential as these databases get larger and larger.

 Please feel free to contact us with any questions or concerns you may have. We are always looking to share what knowledge we have. We also are open to corrections and thoughts as to how to better this advise.

The Gurus Certify This Guide Rules