massive ripper

[zuk]

I'm thinking of building a massive ripper with 10 CD drives to offer a service to rip your entire CD collection at once, like http://www.ripdigital.com/.

I figure two boxes = 20 drives would keep me busy enough just swapping in CDs, ripping to wav files. Then I'd take the boxen home and let them encode to mp3 and look up the CDDB info, etc.

Anyhow, from my calculations 40x DAE will produce ~6MB/s when on the outer track. 10 CDs would be 60MB/s then. And all that data has to go back to a hard drive or two, so 120MB/s throughput would be ideal.

A PCI 2.1 bus can handle 264 MB/s, which would be adequate. I guess I would need an EIDE/ATA card for each set of 4 drives. 12 drives-10 CD & 2 hard drives would mean 2 cards + motherboard ATA support.

When done encoding I'll burn DVDs with all the music on them.

So did I do my math correctly? I know very little about ATA cards and such, so any links or help there is really appreciated. I plan on running linux or freebsd on these boxes.

Thanks!

Monty

[burninfool]

Ripping and encoding is CPU intensive(80-100%) so ripping more than one CD at a time will be useless unless you have a P4 HT or dual Xenons.You could rip in low priority mode but still ripping 2 or 3 CD's at a time is pushing it.
I don't want to discourage you from making money but if you give them a DVDR why couldn't your customers do it themselves...all they need is a DVD-ROM.

[CowboySlim]

Oh-oh! Sounds like Hoxlund's room on wheels!

Whereyat, Hox? Will that work?

Slim

Atari 400 ------------- WiFi Bluetooth -------------- 80 column punched card reader

[zuk]

Ripping and encoding is CPU intensive(80-100%) so ripping more than one CD at a time will be useless unless you have a P4 HT or dual Xenons.You could rip in low priority mode but still ripping 2 or 3 CD's at a time is pushing it.
I don't want to discourage you from making money but if you give them a DVDR why couldn't your customers do it themselves...all they need is a DVD-ROM.

The idea is to go to someone's house, rip (but not encode) 200+ CDs/hour, take the machines back and then encode them at my leisure. So the first priority is massive I/O, which will mainly be limited by my hands shoveling CDs around, as long as I have the machines that can outpace me.

The whole point is to find the guy with 1000 CDs, a new iPod and frustration at how long it is going to take to transfer all his CDs to iTunes. My customers are probably not the kind of people that hang out here...

[dodecahedron]

you will also be limited if you put 4 optical drives to each ATA card. two drives on the same channel will limit your ripping speed.

besides, many people are having problems with running optical drives off ATA cards. they ususally put the hard drives on the ATA card and leave the optical drives on the motherboard's built-in controllers.
you may find it diffitcult to find ATA controller cards that will work well with your drives.

[zuk]

you will also be limited if you put 4 optical drives to each ATA card. two drives on the same channel will limit your ripping speed.

besides, many people are having problems with running optical drives off ATA cards. they ususally put the hard drives on the ATA card and leave the optical drives on the motherboard's built-in controllers.
you may find it diffitcult to find ATA controller cards that will work well with your drives.

Perhaps I should look into putting the drives into a big firewire enclosure then? Firewire 800 should still give me enough speed with a couple of firewire boards.

[dodecahedron]

sounds good. probably better than IDE but definitely more expensive.

[blakerwry]

I'm thinking of building a massive ripper with 10 CD drives to offer a service to rip your entire CD collection at once, like http://www.ripdigital.com/.

I figure two boxes = 20 drives would keep me busy enough just swapping in CDs, ripping to wav files. Then I'd take the boxen home and let them encode to mp3 and look up the CDDB info, etc.

Anyhow, from my calculations 40x DAE will produce ~6MB/s when on the outer track. 10 CDs would be 60MB/s then. And all that data has to go back to a hard drive or two, so 120MB/s throughput would be ideal.

A PCI 2.1 bus can handle 264 MB/s, which would be adequate. I guess I would need an EIDE/ATA card for each set of 4 drives. 12 drives-10 CD & 2 hard drives would mean 2 cards + motherboard ATA support.

When done encoding I'll burn DVDs with all the music on them.

So did I do my math correctly? I know very little about ATA cards and such, so any links or help there is really appreciated. I plan on running linux or freebsd on these boxes.

Thanks!

Monty

Monty, the PCI bus is limited to ~124MB/sec theoretical data rate... if you can get 90MB/sec out of it you're doing well. Luckily you dont have to use the PCI bus if you choose carefully.


Also, uncompressed audio takes a huge amount of space. each CD is likely ~500MB. For a medium collection of 100 discs that'll be about 50GB. But for a large collection that might end up being closer to 500GB. Something to think about...


For your needs RAID 0 would be perfect because you wont need to retain the data long and you do need alot of sequential speed.


I would suggest a board with Native S-ATA like the Asrock k7s8xe+.

It provides 2 S-ATA channels that support RAID 0, plus 2 P-ATA channels for the typical 4 P-ATA devices. These are all run off the southbridge which has a 1GB/sec link to the northbridge (more than any other PC board available).


I would put two fast S-ATA disks in RAID 0 for ripping+OS (Hitachi 7k250 disks come to mind) and put 4 optical disks on the 2 built in P-ATA ports.

You could put additional readers on a PCI card if you think you'll need them (I think SIIG controllers are supposed to work well with ATAPI devices)




Other choices for native S-ATA include the intel ICH6 (comes on 865/875 NB boards), or the VIA 8237 SB (comes with kt600) boards... not sure what is available yet for opteron besides the SiS based boards which are good.


btw, linux support for S-ATA is pretty dim, most drivers are in beta stages or worse. WinXP would be my best bet here.

[zuk]

Monty, the PCI bus is limited to ~124MB/sec theoretical data rate... if you can get 90MB/sec out of it you're doing well. Luckily you dont have to use the PCI bus if you choose carefully.


Also, uncompressed audio takes a huge amount of space. each CD is likely ~500MB. For a medium collection of 100 discs that'll be about 50GB. But for a large collection that might end up being closer to 500GB. Something to think about...


For your needs RAID 0 would be perfect because you wont need to retain the data long and you do need alot of sequential speed.


I would suggest a board with Native S-ATA like the Asrock k7s8xe+.

It provides 2 S-ATA channels that support RAID 0, plus 2 P-ATA channels for the typical 4 P-ATA devices. These are all run off the southbridge which has a 1GB/sec link to the northbridge (more than any other PC board available).


I would put two fast S-ATA disks in RAID 0 for ripping+OS (Hitachi 7k250 disks come to mind) and put 4 optical disks on the 2 built in P-ATA ports.

You could put additional readers on a PCI card if you think you'll need them (I think SIIG controllers are supposed to work well with ATAPI devices)
...
btw, linux support for S-ATA is pretty dim, most drivers are in beta stages or worse. WinXP would be my best bet here.

Thanks for the excellent suggestions! What about putting extra readers externally on the USB 2.0 ports? How does that data flow through the motherboard?

I'll try and see what I can find about SIIG controllers and optical drives. Any hints on where to ask appreciated.

[wicked1]

Lookinto firewire 800 interfaces instead of IDE. I have a couple myself.I can burn to 5 different burners at once without a hitch ar around 20% CPU power.

[wicked1]

caloptic.com is where I got mine. The internal firewire 800 kits.Work great.

[blakerwry]

USB is pretty interrupt happy (that is it takes alot of CPU power compared to other technologies). On any modern board the USB should be implemented through the southbridge. Most southbridges now have 4 USB controllers (8 ports). Which means technically you're not going to run out of bus bandwidth, however you might run out of CPU time/interrupt time.

I agree with the comments on firewire. It is a better technology for higher speed devices like optical and magnetic storage. Mainly because it requires less interrupts (CPU time again). But I have no 1st hand experience with external optical drives.

Personally, I think the four 5.25" bays that a normal tower is going to provide you are enough(that's an average of what? ~30-40 seconds between disk switches?). But if you want more, you could certainly tag a couple firewire and/or USB 2.0 devices on.

Because of price, i think I'm more inclined to go all IDE and get 4-6 readers in a single box.

[wicked1]

USB is pretty interrupt happy (that is it takes alot of CPU power compared to other technologies). On any modern board the USB should be implemented through the southbridge. Most southbridges now have 4 USB controllers (8 ports). Which means technically you're not going to run out of bus bandwidth, however you might run out of CPU time/interrupt time.

I agree with the comments on firewire. It is a better technology for higher speed devices like optical and magnetic storage. Mainly because it requires less interrupts (CPU time again). But I have no 1st hand experience with external optical drives.

Personally, I think the four 5.25" bays that a normal tower is going to provide you are enough(that's an average of what? ~30-40 seconds between disk switches?). But if you want more, you could certainly tag a couple firewire and/or USB 2.0 devices on.

Because of price, i think I'm more inclined to go all IDE and get 4-6 readers in a single box.

you can get dual INTERNAL firewire kites for 60 bux that convert 2 drives into firewire.They dont have to be external.Mine are all internal.I'd like to see you use 4-6 readers on an IDE bus at the same time. Its NOT gonna happen. Ive tried many different solutions to this. Such as running multiple controller cards etc. None worked except firewire. Screw USB

[zuk]

Lookinto firewire 800 interfaces instead of IDE. I have a couple myself.I can burn to 5 different burners at once without a hitch ar around 20% CPU power.
...
you can get dual INTERNAL firewire kites for 60 bux that convert 2 drives into firewire.They dont have to be external.Mine are all internal.I'd like to see you use 4-6 readers on an IDE bus at the same time. Its NOT gonna happen. Ive tried many different solutions to this. Such as running multiple controller cards etc. None worked except firewire. Screw USB 8)

And how are you doing the firewire into the motherboard? Built-in or on a PCI card?
That's great that you can burn to all of them at once.

[wicked1]

built in motherboard for now. I bought the 800 firewire kits and ran outta cash so I hooked them to my onboard 400 firewire controller. Firewire 800 backwards compatible.Works great.Havent had a reason to buy the 800 controller yet.The guys at caloptic said they do 30+ drive systems with the firewire setup no problem. Nero isnt the greatest software for doing a bunch of burns at once yet.Recordnow works better for alot of different drives at once.

[zuk]

Well I bit the bullet and got a 6 ext. bay case, 5 Liteon LTR-52327S CDRs, One Pioneer DVR-106, a couple of Samsung SP1614N 160G drives, and four caloptic dual firewire adapters.

I've never built a computer before and I've got the drives in the case and the adapters in but the power supply isn't feeding power to the drives. I think somehow the switch on the front gets connected through a motherboard to a relay in the power supply. I just gotta figure out how to bypass it.

It'll be a couple of weeks before I have any timings, but when I have them I will post here.