How to make good test documentation from of A-B-A test

[Vekke]

Hi,

I have plans to make A-B-A test to FleXcaps following the instructions on this topic:
http://ecomodder.com/forum/showthrea...ery-11445.html

- Test will be made at dark, when there is less to none traffic and weather changes are not that big
- On motorway
- Cruise control speed 86 km/h
- On level straight road test length about 2-5 km
- Choose non windy day from forecast reports
- Make about 20 tests so 10 A and 10 B
- If there is lots of variation between runs, even more

I dont know will this fit that topic or should we have own topic on test documentation. Mods can decide. I will first put this as own topic.

Anyway I want the documentation to be as "bulletproof" as possible. What I will do:

- Measure windspeed between every run
- Measure temperature between every run
- Put chapter for notes if there has been something that may have effected the results. Car that you can see in front or back of you etc.
- Will make a test video to show the test procedure and conditions. One take on each A and B versions for full test length. Show how the changes has been made. Shoot some nice drive by footage. Other video footage that will affect the results will be done separetely from the tests, like drive pass the truck filming the same time...What else?.
- Report as precisely the equipment used
- Show the raw data on my webpages so dont hide them from public.

Make some A4 for all these procerdures which can used in all tests and will be available for everyone.

After tests make some nice charts and plots from the data, for commercial use. Always have link to the actual test data and procedures used.

Am I missing something out?

[MetroMPG]

Sounds good to me.

I think one of the biggest potential problems to avoid is aerodynamic interference of other vehicles. Particularly vehicles ahead of you on your side of the roadway.

I'm lucky because the road I have used in the past for my experiments is very lightly used by other cars. It's possible (easy) for me to do runs with literally no other cars in sight in either direction.

[saand]

Vekke, sounds likea good method.
I tried to quantify some of my mods and failed, after 3 attempts i gave up. The main problem i had was mentioned by metrompg. Any other cars on the road even off in the far distance ahead caused variable results. Unfortunately i couldn't find a suitably long stretch of road which had no traffic at some time in the day even late at night.

Also for a modification that has a low percentage improvement (below 1%) it becomes very difficult to measure as many variables will effect measurements more than 1%.

I recommend to save yourself some time
- park a car where you plan to do your test, see how many cars go by to gauge if its worth testing on that road.
- Mark clearly on the road where you will begin measuring from and where you will stop measuring from. Can be hard in the dark to locate a generic road feature when your trying to set speed control and measure whatever else your after
- Choose a suitably long stretch of road which will show enough deviation for the expected percentage of improvement. For example for a 0.5% change expected you probably need 10km to see a clear improvement over the noise.
- Check your results as you go, if your first 2 results dont align rethink your testing method, no use doing 10 repetitions if you are just going to get useless data.

Well good luck and i look forward to seeing your data

[Vekke]

I was able to get the test for this night. Starts after 22 local time. 16 capsels. I hope that results will be about three percentage difference.

Front capsels will be stainless steel versions, but they cover the front wheel nuts so it will represent my version of the capsel in this test.

Yes we will wait until the road is silent.
We will paint lines on the roadside with black paint. There will be road lightning.
First we will start warming up the car and see how steady the results will be. If there is variations like close to 3 percent its not going to be accurate enough. Have to wait better time or place.

Täsmäsää Vantaa - Foreca.fi

after 22 weather should be steadily at -3 celsius degrees. There should be litlle wind about 6m/s. Roads will be frozen and there is no snow which is biggest problem in finland this time of year.

I will have a video camera and reguar camera and my brother to take pictures. Now I just need to make the A4 form.

[Vekke]

How does this look?
ABA test sheet.pdf - Windows Live
Still some time to make adjustments...

[Frank Lee]

Back in college I ran some coast down tests. I didn't encounter any other vehicles while testing, but little breezes and gusts made the results unreliable anyway.

[RobertSmalls]

Originally Posted by Vekke

- Make about 20 tests so 10 A and 10 B
- If there is lots of variation between runs, even more

In order to be scientific about this, you need to use statistics to determine the required sample size. If possible, bring a spreadsheet with you during your testing. As you increase the number of entries, (hopefully) the standard deviation will decrease and you'll be able to say with greater confidence how accurate your numbers are.

I took the 6 A and 4 B runs tabulated here: Reflections on side mirrors: testing drag vs. MPG - MetroMPG.com
and ran it through Excel. For A runs, the mean is 55.2, with a standard deviation σ=.28. Therefore, you can say with 95% confidence (2σ) that his mpg was between 54.7 and 55.8, or with 99% confidence (3σ) that it was between 54.4 and 56.1 - you're >99% sure it was less than the 56.5 mpg that he reported in configuration B.

If you spend too much time out there, e.g. with 20 runs, other parameters like ambient temperature, driver attention, and the weight of fuel in your tank will start to drift, potentially skewing your data.

[Vekke]

Good points there. So if the results start clearly be the same in every run and there is a clear difference there is no need to do 20 runs. If they wary little then more runs is better. Variations can be from wind for example. I know there is little wind coming but hopefully it will be somewhat steady. Now I will have to go to made preparations ready. Tests will begin about after 4 hours from here.

[Frank Lee]

Hmmmm... well I'll be the first to admit I'm not the greatest statistician, but I distinctly recall, after completing my coast down tests, finding in research about such testing that it can take hundreds of runs to achieve a statistically significant mean.

[euromodder]

Checking tyre pressure after each run changes the measured value.

Set tyre pressure before starting a series of runs, and leave it at that.
Check again after completing a series of runs.

When doing another series of runs, get the tyres up to the same pressure using the same equipment and meter.