How to make good test documentation from of A-B-A test
 

Hi,

I have plans to make A-B-A test to FleXcaps following the instructions on this topic:
http://ecomodder.com/forum/showthrea...ery-11445.html

- Test will be made at dark, when there is less to none traffic and weather changes are not that big
- On motorway
- Cruise control speed 86 km/h
- On level straight road test length about 2-5 km
- Choose non windy day from forecast reports
- Make about 20 tests so 10 A and 10 B
- If there is lots of variation between runs, even more

I dont know will this fit that topic or should we have own topic on test documentation. Mods can decide. I will first put this as own topic.

Anyway I want the documentation to be as "bulletproof" as possible. What I will do:

- Measure windspeed between every run
- Measure temperature between every run
- Put chapter for notes if there has been something that may have effected the results. Car that you can see in front or back of you etc.
- Will make a test video to show the test procedure and conditions. One take on each A and B versions for full test length. Show how the changes has been made. Shoot some nice drive by footage. Other video footage that will affect the results will be done separetely from the tests, like drive pass the truck filming the same time...What else?.
- Report as precisely the equipment used
- Show the raw data on my webpages so dont hide them from public.

Make some A4 for all these procerdures which can used in all tests and will be available for everyone.

After tests make some nice charts and plots from the data, for commercial use. Always have link to the actual test data and procedures used.

Am I missing something out?

Sounds good to me.

I think one of the biggest potential problems to avoid is aerodynamic interference of other vehicles. Particularly vehicles ahead of you on your side of the roadway.

I'm lucky because the road I have used in the past for my experiments is very lightly used by other cars. It's possible (easy) for me to do runs with literally no other cars in sight in either direction.

Vekke, sounds likea good method.
I tried to quantify some of my mods and failed, after 3 attempts i gave up. The main problem i had was mentioned by metrompg. Any other cars on the road even off in the far distance ahead caused variable results. Unfortunately i couldn't find a suitably long stretch of road which had no traffic at some time in the day even late at night.

Also for a modification that has a low percentage improvement (below 1%) it becomes very difficult to measure as many variables will effect measurements more than 1%.

I recommend to save yourself some time
- park a car where you plan to do your test, see how many cars go by to gauge if its worth testing on that road.
- Mark clearly on the road where you will begin measuring from and where you will stop measuring from. Can be hard in the dark to locate a generic road feature when your trying to set speed control and measure whatever else your after
- Choose a suitably long stretch of road which will show enough deviation for the expected percentage of improvement. For example for a 0.5% change expected you probably need 10km to see a clear improvement over the noise.
- Check your results as you go, if your first 2 results dont align rethink your testing method, no use doing 10 repetitions if you are just going to get useless data.

Well good luck and i look forward to seeing your data

I was able to get the test for this night. Starts after 22 local time. 16 capsels. I hope that results will be about three percentage difference.

Front capsels will be stainless steel versions, but they cover the front wheel nuts so it will represent my version of the capsel in this test.

Yes we will wait until the road is silent.
We will paint lines on the roadside with black paint. There will be road lightning.
First we will start warming up the car and see how steady the results will be. If there is variations like close to 3 percent its not going to be accurate enough. Have to wait better time or place.

Täsmäsää Vantaa - Foreca.fi

after 22 weather should be steadily at -3 celsius degrees. There should be litlle wind about 6m/s. Roads will be frozen and there is no snow which is biggest problem in finland this time of year.

I will have a video camera and reguar camera and my brother to take pictures. Now I just need to make the A4 form.

How does this look?
ABA test sheet.pdf - Windows Live
Still some time to make adjustments...

Back in college I ran some coast down tests. I didn't encounter any other vehicles while testing, but little breezes and gusts made the results unreliable anyway.

In order to be scientific about this, you need to use statistics to determine the required sample size. If possible, bring a spreadsheet with you during your testing. As you increase the number of entries, (hopefully) the standard deviation will decrease and you'll be able to say with greater confidence how accurate your numbers are.

I took the 6 A and 4 B runs tabulated here: Reflections on side mirrors: testing drag vs. MPG - MetroMPG.com
and ran it through Excel. For A runs, the mean is 55.2, with a standard deviation σ=.28. Therefore, you can say with 95% confidence (2σ) that his mpg was between 54.7 and 55.8, or with 99% confidence (3σ) that it was between 54.4 and 56.1 - you're >99% sure it was less than the 56.5 mpg that he reported in configuration B.

If you spend too much time out there, e.g. with 20 runs, other parameters like ambient temperature, driver attention, and the weight of fuel in your tank will start to drift, potentially skewing your data.

Good points there. So if the results start clearly be the same in every run and there is a clear difference there is no need to do 20 runs. If they wary little then more runs is better. Variations can be from wind for example. I know there is little wind coming but hopefully it will be somewhat steady. Now I will have to go to made preparations ready. Tests will begin about after 4 hours from here.

Hmmmm... well I'll be the first to admit I'm not the greatest statistician, but I distinctly recall, after completing my coast down tests, finding in research about such testing that it can take hundreds of runs to achieve a statistically significant mean.

Checking tyre pressure after each run changes the measured value.

Set tyre pressure before starting a series of runs, and leave it at that.
Check again after completing a series of runs.

When doing another series of runs, get the tyres up to the same pressure using the same equipment and meter.

Sad news.

We were not able to perform the test yesterday. We broke two tools when trying to remove the first of 6 stainless steel hubcaps which were installed to the truck.

22.5 inch Achter wieldop RVS 1 stuks [12011] - €85.00 : FSB-SHOP!, Truck@Vans interieurs en onderdelen

They were this type that there is bracket under bolts where you screw that hubcap with two different size flanged bolts. They were just so rusted that with our tools no chance. Put some rust off threre and the guy will try to remove those during next week.

Make sure the car's running gear and engine are at equilibrium before running the first test. This means fully warmed up. A 5 km drive will not be enough.

Instead of evening runs I suggest early morning runs. The atmosphere is at its most stable before sunrise and there is frequently no wind. Traffic is at its lowest in early morning too.

Make everything about the car the same so you can get comparable results across several days. For example, full fuel tank, no extra weight, tire pressures.

if possible, perform a calibration run or two before and after each test series where the car is set to zero state (i.e. no mods). This lets you know if things have changed during the series, and if each series is comparable to runs done at other times. These calibration runs also let you correct for factors beyond your control, such as seasons and temperatures change.

You can see estimation when there will be least traffic. Usually in Finnish roads it start to get "silent" at 22.00. From 24.00 to 05.00 is best time and the lowest point is near 03.00:

Liikennetiedot Pääkaupunkiseutu - Foreca.fi

Go over those arrows to see estimations and real numbers.

A-B-A Testing variables
 

Interesting thread. A-B-A tests have been reported here and elsewhere as a great way to calculate the fuel efficiencies of aerodynamic modifications to vehicles.

In this thread it is discussed things like atmospheric conditions (winds, temperature, etc.), even the aerodynamic effect of other vehicles upon the test vehicle, actively engaged in a test, may have an effect on the results of the test.

I have often wondered if the very cold temperatures and brisk cross wind may have effected this A-B-A test, which was also determined in just one day. From what I have read in this thread, my thinking is that Fubeca's test may be very flawed indeed, which he has actually agreed to. Not to belittle Fubeca for the hard work and fine job he did on his aerocap, he just needed better atmospheric conditions on the day he ran the A-B-A test below.

http://ecomodder.com/forum/showthrea...ead-11611.html


Bondo

This is why A-B-A-B testing is better than A-B-A. If there is a trend, you will notice it in both trials.
Extrapolating this: It would be best to do one run with, one run without mods. Then again and again and again. But this of course takes up too much time, so A-B-A-B is the best compromise. If you have time left over, then continue to do another A-B as soon as possible.

Hi,

Performed first tests and learned thing trought the hard way. Plan was to do A-B-A measurements in 3km long and level straight. We used only 14 FleXcaps because front capsels were so rosted that the bolt heads were ruined and we decided to leave them on.

Longer distance was about 11 long kilometers in one direction. Total length of one north to south run is about 28 kilometers.

Mittaustulokset 5-6.2.2011.jpeg - Windows Live

1. We drived the first run to north and got a result 38.7L/100km. Mistake number one was that we drived the car about 20 km before the first run.
2. Same leg to south 30,5 l that number (27.1) is for longer distance.
3. North still capsels off 39.5l. Those + plusses mean that car went by, one plus one car...
4. South 30,3l and (27.0)

When we first drived two times the straight and got north results varied 2% we decided to measure effect on longer distance, because we got better accuracy for that to south. First results for south were. 27.1 and 27.0 so accuracy was amazingly good between runs.

5. North same long distance as to south. 43,1 and 42,5. Measured results to two points on the road.
6. 29.9l (26,7l) As you can see from these figures they get smaller. That is caused the fact that car warm up during the runs bearing, oils etc. Now we had already driven the car before the test for 50 km before the first run.

Now it took about 20 minutes to install the capsels beacause we took some video material also. Car was runnin this time but temperature was -2 celsius so the car calmed little.

Now the first difference is that we did not warm the car for 20 km before first run capsels on so the car is "cold"
7. result was 44.2 liters which is 3,8% more than the one run (42,5l) we had for that long run to north capsels off.
8. 30,6 and to that longer distance 26,8 liters. That short leg car is still guite cold and gives the highest result so far but to the longer measurement it is already best 26,8.
9. Now here we had a human error and driver forget to put down one of the axels on the truck. We decided to measure same leg as in first two north runs. 38,6 liters which is the lowest by small margin.
10. Here police patrol stopped us checked our papers. They were wondering what the hell we drive aroung motorway in the middle of the night. It took 20 minuts and again car became "cold". Still we were able to get 31,4 and 27,3 liters. Now the warm up effect is 4.1 liters or 13% difference. This police incident pretty much ruined our last measurements because we measurements are not really comparable after this.
11. 43,1l
12. 27,1 liters. I have also wind data and try to find out if this can be caused by the wind. During measurements wind speed rose little.
13. We took 20 minutes coffee breack and too Capsels again off. 45,6 liters. Now this runs only difference measuremenet 7 is wind speed so this is good comparison. Difference is 3.1 %.
14. 32,7l and 28,1. Again best comparison to mearument 8 and difference is 4,6%. Difference between these two runs which are tests best comparisons is total 3,85% less fuel.
15. 44.0
16. 31,1l and 28,0 liters
17. 43,8 liters
18. 28,5 liters.

Average of all A off north 43,975 (4 runs)
average of all B on north 43,65 ( only 2 runs)
Difference 0.74 % and remenber those figures are not good comparison to each other...

A1 off south 26,933l (3 runs)
B on south 27,07 l (3 runs)
A2 off south 28,2l (3runs)
Better comparison is B to A2 and difference is 4 %. Difference between A and B south all runs is 1.8%

Difference between all A and B runs is 1.3% which is still good result. We have to do second test on that same place. Place worked out well but our testing werent perfect. Biggest factors were, mistake on the first measurement, and the police. Also we should have changed the capsels between every run to avoid that wind and also warming up effect. Next time we will install capsels in 5 minuts and take them out in 5 minuts so all runs are comparable to each other in better manner.

More data will come as soon as I am able to process it first. Also video footage.

Wow! That's a lot of work, Vekke. It's too bad that you had problems during the test runs, hopefully the next try will be better.

BTW Do have any idea how much fuel was used during the test? Doing that many 20km runs @ 30-40 l/100km must have used up quite a bit.

Average consumption with emty trailer is circa 35 l/100 km. We drove about 50 km to test place and test driving for 250 km and back that 50 kilometers so total 350 km that means about 122 liters which is in money 156€. Whole test took over 6 hours to finish it.

I could drive with my seat with that money for over 2400 km easily at 5l/100 km. When you are driving empty truck you see bigger gains than with full load, but you can make rough estimation what is the effect of those FleXcaps with full load. Now we just need new test.

Now imagine that some of the trucks do run empty on our roads and try to imagine why? One of the best ways to save fuel is to get your gargo space full all the time and minimize empty driving.