Best performance setup?

[rdgrimes]

Sorry Cowboy, your test is invalid. You have to use the same 2 drives (identical drives preferred) and do the same transfer in each configuration. That's right, you actually have to move the cables. Different drives can have very different read and write speeds, and the transfer will only occur as fast as the slowest link in the chain. Suggest you start with some disc benchmarks to determine if the drives have the same write and read speeds. They have to be the same, or you at least have to have the same drive reading and the same drive writing in each scenario.

oops, cfitz and I posted at the same time.

Oh, and BTW, my new Raptor RAID-0 will read or write that 900MB file in about 10 seconds. If only I had a second set......

[CowboySlim]

Rd,

Yes, I forgot to mention that aspect of the test although I had thought of it previously. Actually, the drives are different brands, but they are both 40GB, 7200RPM, 2MB, UATA. I wouldn't expect them to be identical in transfer rate to the 5th digit; however, that alone shouldn't explain the difference. Frankly, I don't know which is the "fastest". I can repeat the intrachannel test on the primary channel.

[rdgrimes]

they are both 40GB, 7200RPM, 2MB, UATA. I wouldn't expect them to be identical in transfer rate to the 5th digit; however, that alone shouldn't explain the difference

That's a HUGE assumption. If you use the same one for reading and the same one for writing in each scenario, it won't matter.

[CowboySlim]

The copy from B to B, on the other hand, is a random access transfer. Drive B's head has to seek to the start of the files, read some data, then seek to the empy space, write some, then seek back to the files, etc. All this seeking really slows down the transfer.

cfitz

cfitz, I wondered when you were going to get here and help us out.

Firstly, I did consider moving the drive cables. However, in my belief that the data flows from one partition via the motherboard, regardless if the two partitions are on the same drive, different drives but same channel, or different drives and different channels, I thought that it would tell me something. Well, I tried to get by too easily. I'll move the cables tomorrow night.

Secondly, upon reflection of the two to one factor, I don't think that it will be anywhere near that. The nature of the IDE transfer is no different today than five years ago. However, what is different is the amount of RAM that we use. I'm running 525Mb. For a 936Mb transfer, that appears to be only two fills. Five years ago, when 64Mb was typical, that transfer would be 15 RAM fills. That is, unless I'm completely mistaken that intrachannel, interdrive IDE transfer is discontinous as compared to interchannel IDE transfer which is much more continous.

[Inertia]

So gather 'round the campfire and maybe we can separate some mythology and anecdotal observations from good ol' fashioned book larnin'.

We are speaking here of empirical observations, not reliance on anecdotes. Good ol' fashioned book larnin' is only as good as the information in the book, and knowledgable use of the the information.

It has to do with the open systems architecture as defined in the IEEE Specs/Standards. Once these have been voted on and become official, everybody has to design hardware and software that is in compliance with them or their designs just won't work. It's just that simple - that's all there is to it. It doesn't matter if the motherboard with the IDE channels/connectors and the ATA/ATAPI devices are 5 years old or new this year. The standards specify one way data transfer only. (Now, this is different from SCSI, which does have two way data transfer.) What else can I say?

You could say that you are not relying on your interpretation of technical standards, but have run a valid controlled test that produces results that agree with your theory. That has not yet been done as reminded by cfitz and rdgrimes.

I take no offense that disagreement was expressed with my postings in this issue. However, I am gravely dissapointed in the attitude expressed concerning the veracity of Robert Bruce Thompson's book (my referenced source). That disdainful attitude does not convey an overwhelming sense of intellectualism.

There was no specific criticism of the veracity of the book which you used. It is not uncommon for material in books (or the internet) to be inaccurate, partially true, or true but inaccurately interpreted. If information from a book is taken to be absolute proof that something can't work a certain way when actual experience indicates otherwise, this is intellectual inquiry? My dictionary defines "intellectual" as someone who uses his mind creatively. This would include not always taking "book larnin" at face value to advance a theory whatever the source, but confirming the hypothesis through intellectual curiosity.

[Inertia]

Secondly, upon reflection of the two to one factor, I don't think that it will be anywhere near that. The nature of the IDE transfer is no different today than five years ago. However, what is different is the amount of RAM that we use. I'm running 525Mb. For a 936Mb transfer, that appears to be only two fills. Five years ago, when 64Mb was typical, that transfer would be 15 RAM fills. That is, unless I'm completely mistaken that intrachannel, interdrive IDE transfer is discontinous as compared to interchannel IDE transfer which is much more continous.

There have been six updates to IDE transfers since ATA-1 was discontinued in 1994. The last five years have seen updates from ATA-4 to ATA-7 for much faster and reliable data transfers. These advances have included improvements such as support for integral ATAPI and queuing commands (ATA-4) for enhanced multitasking, similar to that provided in SCSI-2.

RAM memory is used only for data caching, not for continuous data transfers from storage devices. "RAM-fills" have no meaning in continuous storage data transfers. Caching is most useful in reducing seek times for frequently used data or predictive assumptions (read-ahead) for data. Increased RAM per se does not increase continuous storage data transfer speeds, although it can ameliorate other system bottlenecks and improve overall system utility.

In contrast, the DOS era RAMDisk was a trick to convert system RAM to be allocated and recognized as a hard drive. This trick made the RAMDisk an extremely fast, practically real-time storage device (especially when compared to the older and slower hard drives). Unfortunately, unless the data on the RAMDisk was saved to a non-volatile medium, the volatile RAM data would disappear with a reboot.

[minix]

These advances have included improvements such as support for integral ATAPI and queuing commands (ATA-4) for enhanced multitasking, similar to that provided in SCSI-2.

OK.
According to StorageReview, "special command queuing and overlapping protocols were defined" in ATA-4.
They were defined, but the question is: were they implemented in any drive?


In Serial ATA it seems this has been implemented only a few months ago:
http://www.harddiskinfo.com/Sections/Ne ... newsID=554


For the moment, I still rely on this:
"In contrast, IDE/ATA transactions to one device "block" the channel and the other device cannot be accessed. Putting two devices on two different channels allows simultaneous access, but severely restricts expandability."

Anyway, I also have the experience with two drives working perfectly well in the same IDE channel. It usually works well, nowadays.

[CowboySlim]

For the moment, I still rely on this:
"In contrast, IDE/ATA transactions to one device "block" the channel and the other device cannot be accessed. Putting two devices on two different channels allows simultaneous access, but severely restricts expandability."

Well, this source provides nothing but corroboration for the source that I've cited above. However, I would be interested in some elaboration concerning the restricted expandability.

In the meantime I'll be running the "valid" tests shortly.

[minix]

Well, this source provides nothing but corroboration for the source that I've cited above.

Yes, but that is no longer true if new drives are using the "command queuing/disconnect" protocols.
I'd say this hasn't been been generally implemented, maybe in hard disks like that Serial ATA Seagate, but I don't have any proof.

However, I would be interested in some elaboration concerning the restricted expandability.

well, I'd say that if you have 2 IDE controllers and you can only put 1 drive per controller, you can only have 2 drives instead of 4.

[Inertia]

According to StorageReview, "special command queuing and overlapping protocols were defined" in ATA-4.
They were defined, but the question is: were they implemented in any drive?

ATA-4 command queuing was not broadly accepted by hard drive manufacturers for technical reasons. Pacific Digital has support for ATA-4 command queuing in an ATA RAID controller that will work with any ATA drive from ATA 4 (UDMA 33). It supposedly provides SCSI-like performance with lower cost ATA drives. See Pacific Digital DS4-100 Technical Specs. and Pacific Digital DS4-100 Retail.

The ATA-6 IBM Deskstar 180GXP uses "tag 'n seek" command queuing.

For the moment, I still rely on this:
"In contrast, IDE/ATA transactions to one device "block" the channel and the other device cannot be accessed. Putting two devices on two different channels allows simultaneous access, but severely restricts expandability."

Anyway, I also have the experience with two drives working perfectly well in the same IDE channel. It usually works well, nowadays.

The question was not whether command queuing was implemented in ATA drives, although it is an interesting topic. The question is whether a data transfer can be made efficiently between two ATA(PI) devices as master and slave on the same channel. No one is arguing that this is a preferred configuration, only whether it may work very satisfactorily for some system configurations. I don't know if you have noticed that your own successful experience appears to contradict your quote in italics.

Perhaps some of the confusion may arise over the use of the term "block" or interpretation of the misleading "ATA forbids simultaneous I/O on an interface, which means that only one device can be active at a time. If one device is reading or writing, the other device cannot read or write until the other device yields the channel. The implication of this is that if you have two devices that need to perform simultaneous I/O, e.g., a CD writer that you use to duplicate CDs from a CD-ROM drive, you should place those two devices on separate interfaces."

Consider this hypothesis: If an IDE channel were "blocked" after a read command for the duration of the reading cycle, it would not be possible to write to a device on the same channel. A read command may take only milliseconds, after which the write command is sent to the other device. This is not a simultaneous command transfer and is technically slower than with command queuing or channel separation. After the initial slight command delay, a large data transfer might proceed efficiently without interruption. The command latency would be more objectionable for many small data transfers, as a slight read/write command delay would be introduced at each copy cycle. This is why, for example, if two hard drives are the only storage devices it is preferable to put them on separate channels.

[CowboySlim]

Yes, but that is no longer true if new drives are using the "command queuing/disconnect" protocols.

Interesting answer, but somewhat speculative. In doing so, maybe they can't maintain backward compatibility with ATA-3 and earlier motherboards?

Well, anyway, here are the results from my "valid" tests (once again from defragged NTFS partitions to freshly formatted NTFS partitions in each case):
1. Several hundred *.jpg files, 936Mb total (repeated several times):
Intrachannel: 110sec
Interchannel: 105sec
2. One image file, 1,317Mb total:
Intrachannel: 65sec
Interchannel: 60sec

Conclusion: Interchannel transfer is only marginally faster, certainly a lot less than I expected. So if Intimidator were doing intrachannel backups with compressed images, etc., the effect probably would not be noticeable.

One other observation: Well, Buckaroos, it sure was hard to move that jinglebob from 3-4 to 5-6 with my Bar-B-Q tongs.

[minix]

I don't know if you have noticed that your own successful experience appears to contradict your quote in italics.

Not exactly.
The differences that CowboySlim got, are what I expect in a good system.
Two drives in different channels will be marginally faster, nothing noticeable. It's not necessary unless there are problems (these problems are more likely if you have the drives in the same controller).


I'm not sure what you mean with your final explanation.
If you mean that the command to write won't start until all the data is read, then well, yes, theorically a transfer might be read1-write1, read2-write2, read3-write3, etc. In this case, yes, there's nothing to be gained. But there are a lot of buffers around...

[Inertia]

I'm not sure what you mean with your final explanation.

My last statement was just a theory of what might be involved if there are marginally slower copy times on the same channel compared to separate channels. The idea is that command processing might be less efficient on the same channel, operating in consecutive cycles rather than simultaneous cycles. In any event, under most circumstances the effect (if noticed) would be minimal.

Back to the main question: Can data transfers between two devices on the same channel be done efficiently. Evidence indicates that the answer is yes for most newer computers. There is no significant throughput constraint between intrachannel transfers. I think this thread has gotten off track because the quoted statements regarding "blocks" and constraints were taken to mean intrachannel transfers. The warnings about not being able to use one device until the other is released is meaningless for same channel transfers, which by definition use both devices simultaneously . I contend that these cautions about ATA behavior are concerned with multitasking interchannel transfers.

For example, consider tests just run using a primary channel with a 12x Plextor as master and DVD-ROM as slave, and a secondary channel with another 12x Plextor as master and a 52x Liteon as slave. Four 7200 RPM hard drives are on a mainboard IDE controller. Tests were run with Stomp RecordNow Max copying 700 MB AVI files. All times include initialization and finalization.

12x Plextor alone: 6:54
52x Liteon alone: 2:33

Copying two different files simultaneously from separate hard drives on the same channel to two burners on the same channel:

12x Plextor: 8:37
52x Liteon: 8:10

Copying three different files simultaneously from three separate hard drives, two drives on the same channel and the third from a separate networked computer. The first two burners below are on the same channel and the third on a separate channel.

12x Plextor: 8:29
52x Liteon: 8:10
12x Plextor copying from network: 7:33

Interestingly, in the simultaneous separate source tests above, CPU utilization for RecordNow Max was from 8-10%. Next, I ran some tests copying the same (one) 700 MB file simulaneously to multiple burners.

Plextor, Liteon, Plextor: copied to all three burners simultaneously: 8:41
Plextor, Plextor on separate channels: 6:54 (same as single Plextor).
Plextor, Liteon on the same channel: 8:29

When copying one file with multiple burners, CPU utilization was 3%. In summary, it appears that when data is copied to two burners on the same channel, only one burner at a time can be accessed and the other burner on that channel will not be simultaneously available. This is due to to the constraints of the ATA protocol when not using command queuing.

I don't consider these tests to be definitive, because the 52x Liteon burner is obviously a mismatch for a 12x Plextor in dual (triple) burning. Even so, the RecordNow Max program handled it well and I tend to think that similar results would have been obtained even if another Plextor were substituted for the Liteon. It was easy to tell when burners on the same channel were operating together, as the lights indicated that buffer underrun protection was being used.

If anyone else would care to run some tests to corroborate my results or find a different result, please do so in order to help clear up this subject.