Quote:
Originally Posted by niky
A-B testing is flawed simply because you can get hiccups either way.
|
Its used in science quite a bit. Go read some Nasa papers.
In this case the tests were run more like an A B C test.
A no clay
B with clay
C with dimples
Quote:
A cooled car is different from a cold car. The residual heat in the radiator and transmission is still there hours later. The huge time interval between runs is also a big factor when you consider the effects of external air pressure and temperature on the way an EFI engine runs and rolling resistance.
|
Those factors aren't going to get you an 11% change. The fact that the A and B tests were so similar shows a high level of consistency in their test methods. A 5% change would be debatable but an 11% change can't be ignored.
Quote:
|
This is why I feel they should have done the test with fiberglass panels or inserts, quick-swapped in just minutes between the runs. That would enable you to do three back-to-back runs over the course of the morning.
|
But then your introducing error because you have to create to sets of molds. and you can't be sure they are the same except for just the dimples. Pop on and Pop off panels will vibrate thus creating drag.
Quote:
-
The fact that there was a result is interesting. The fact that there was no verification of the result by re-testing is frustrating. You can have a lot of errors and miss a lot of things working to a deadline like the Mythbusters do. And more than once they've had to revisit a "Myth" and change their verdict because of some flaw in their methodology.
|
Keep posting on their fan page then and get them to do it. I however haven't seen one decent argument that will nullify an 11% gain. In reality the gain may be lower but its not going to be negative