EcoModder Forum How to properly test modifications (AKA: don't believe every MPG claim you read)

Register Now
 Remember

 Submit Tools LinkBack Thread Tools
01-13-2012, 10:43 AM   #71 (permalink)
EcoModding Apprentice

Join Date: Oct 2008
Location: Chile
Posts: 222

Mercedes 89 D - '89 Mercedes 300 E
90 day: 33.86 mpg (US)

Skodie - '09 Skoda Octavia TDI PD
90 day: 38.84 mpg (US)

1993 Mercedes 300D Turbo - '93 Mercedes Benz 300D Turbo W124
90 day: 26.19 mpg (US)

Crossie - '16 Subaru XV Crosstreak
90 day: 9.61 mpg (US)

Crossie - '16 Subaru XV Crosstreak
90 day: 33.34 mpg (US)
Thanks: 15
Thanked 9 Times in 7 Posts
You asked if I studied with professor Hawkins?

Quote:
 Originally Posted by jakobnev Your professor was (Dammit, that was supposed to be linked but not embedded!)
No, my professor wasn´t the same as yours. However, I like him.

My professor in Physics was Oscar García, Ph.D., Scientist and International Referee.

However, I wasn´t that good student. Don´t command Entropy.

__________________
Mercedes 300 D turbo 1993

Last edited by oldbeaver; 01-13-2012 at 10:47 AM.. Reason: Improving

 Today Popular topics Other popular topics in this forum...
 05-09-2012, 02:44 PM #72 (permalink) EcoModding Lurker   Join Date: May 2012 Location: Delaware, Ohio, USA Posts: 24 Bluemobile - '11 Honda CR-Z EX Thanks: 0 Thanked 21 Times in 4 Posts to MetroMPG: I've read and re-read your post which starts this thread. I have no training in statistics, and although I can do the "A-B-A" testing, I don't know how to tell whether differences are significant. Can you describe "statistical significance" in layman's terms? For instance, if I do 5 two-way "A" runs with an average of 59.8 MPG, and standard deviation of 1.4, then 5 "B" runs of 60.6 MPG and s.d. of 1.4, does this mean anything?
 06-16-2012, 03:58 AM #73 (permalink) Master EcoModder   Join Date: Feb 2010 Location: Elmira, NY Posts: 1,729 Thanks: 305 Thanked 333 Times in 278 Posts Yes, it does - sort of. The term 'significant' by convention means that 5% of the values sampled fall outside of an acceptable range of deviation (1.95 x s.d.) in a normal distribution. Whether the 0.8 mpg is due to a modification or the randomness of the samples is another question. Since the difference of the two averages is barely half of the s.d. of each sample, most samples fall within a range common to both A and B.
 06-16-2012, 10:22 AM #74 (permalink) EcoModding Lurker   Join Date: May 2012 Location: Delaware, Ohio, USA Posts: 24 Bluemobile - '11 Honda CR-Z EX Thanks: 0 Thanked 21 Times in 4 Posts Thanks for your reply. It sounds as though 5 runs is not enough to determine whether some mpg figures are "outliers", or whether the set of mpg figures just vary a lot. I'll try a set of 10 runs next time, and post results here.
 06-16-2012, 06:52 PM #75 (permalink) Master EcoModder   Join Date: Feb 2010 Location: Elmira, NY Posts: 1,729 Thanks: 305 Thanked 333 Times in 278 Posts Confidence level in testing is somewhat personal. The question is how certain do we want to be that our tests give are truly representative of what is actually going on. As a quality inspector I would be comfortable with five samples and a standard deviation of 1.4 to accept a batch of product from a proven process. If I was going prove to a major customer that a process line was operating within tolerance 99.9% of the time I would need to take 40 samples or more. What we are doing is called 'hypothesis testing' in the statistics books. We are testing the claim that a certain product or technique will provide a detectable (and economically feasible) improvement in fuel economy. We may decide that we want to be 99% sure that on average there is a 5% improvement in mpg. Given that test, there are means to calculate the number of trials to run. Most libraries will have books on statistics or quality and you can pick one that matches your reading level.
 06-16-2012, 07:14 PM #76 (permalink) Master EcoModder   Join Date: Feb 2010 Location: Elmira, NY Posts: 1,729 Thanks: 305 Thanked 333 Times in 278 Posts There is a great deal of debate over Intelligent Design, Evolution, faith, and science with most of it being as acrimonious as it narrow minded on all sides. The reality of the situation is that Science is a means to describe what we can observe. Faith, in the Biblical sense, is confidence based on relationship with God. As in science, we gain confidence in a relationship by repeated trials and tests. If we are honest with ourselves we can learn a great deal. This I have learned: a) God is very scientific, b) the easiest person to outsmart is yourself.
 06-19-2012, 09:42 AM #77 (permalink) EcoModding Lurker   Join Date: May 2012 Location: Delaware, Ohio, USA Posts: 24 Bluemobile - '11 Honda CR-Z EX Thanks: 0 Thanked 21 Times in 4 Posts Grant-53... I found a copy of "Statistics for Dummies" at the library. Thought I'd start at the VERY beginning!
 10-13-2012, 03:35 AM #78 (permalink) Master EcoModder   Join Date: Oct 2012 Location: Carson City, Nevada Posts: 612 Jimmy - '00 GMC Jimmy SLT 90 day: 21.18 mpg (US) The White Gnat - '99 Suzuki Swift Team Suzuki 90 day: 51.87 mpg (US) Thanks: 240 Thanked 109 Times in 87 Posts Guys, I find this very interesting and appreciate the knowledge available in this thread. I've had my questions about "when do you have results that mean anything?", and this discussion gives me some much appreciated guidance. Thank you!! Bill
 10-15-2012, 02:34 PM #79 (permalink) EcoModding Apprentice   Join Date: Apr 2012 Location: kansas city, mo Posts: 214 Pantry Boy Supreme (pbs) - '04 Volvo S80 T6 Premier Thanks: 28 Thanked 46 Times in 26 Posts So, maybe its just me, but i hate statistics, too "general". But i am a lazy human, that will eventually get around to my own testing ...eventually. If you were to say 100=(100%) people gathered for a "study", revealing that 90% = 90 people had mental disorders, 100% of the time. You still whined up at 90% = 90 people. Thats great, but there's nothing that Specifically determines this result. Just a overall Broad Conclusion... IE: What type of disorders do they classify, and or test for individually?! if at all for the overall result. One test might result with flying colors, while the same "participant" failing the next test miserably. ^^^Example only^^^ Now for this type of testing, I'd "prefer" to have individual "Lab" testing for certain things (funds pending) to eliminate variables like mentioned before.. But Backyard Testing, I would much rather include a little more detail into my Results if I am certainly going to be testing in them. While Caost down test (Aero), or Instant feedback instrumentation good for (MPG) runs, are great, they include all the Variables, and come out with a more or less consistant result if done right enough times for a satisfactory conclusion (for that specific time frame). I would run the test as "normal", multiple times, but just testing specifically at a certain Weather/temperature, Atitude, traffic vs. no traffic, "individual variable" basis instead. Temps are one of the biggest variables geographically(anywhere), next would be Altitude. (X)Temp x (X)Altitude = (X)mpg, OR (X)Aero efficiency Hence forth I think that if I so happen to live geographically(not at nor near sea level) for both of these variables then I would just be conducting this test multiple times. Then at a later time, when the temp has changed, rerun the test at the same altitude to see how much(if any) change has accured. Later on if you have the time or ability, Run a similar test at a different altitude/temp combo, etc. Continue this testing pattern for the other variables where possible. While Each graph wont be exactly the same, result wise, you will be able to see the corrolation between the variables specific to your vehicle This would make for a battering amount of Tests and Combinations, Time and Effort used would be astronamical. But once its all said and done you have all variables on their own, that way you could maybe provide yourself info for YOUR "Optimum" curcumstance to test at for an Overall Result based on your previous research. The Ideal curcumstances would be to unload the car, inflate tires(specific PSI), Cruise control, Instruments calibrated, 0 wind, 0 traffic, "real world" 70-80 avg tepms, humidity, Sea Level, etc, etc... But not all those factors align perfectly for everyone... __________________ __________________ "The test of the machine is the satisfaction it gives you. There isn't any other test. If the machine produces tranquility it's right. If it disturbs you it's wrong until either the machine or your mind is changed." Robert M Pirsig. Last edited by RiderofBikes; 10-15-2012 at 02:54 PM..
 10-15-2012, 04:22 PM #80 (permalink) Master EcoModder   Join Date: Oct 2012 Location: Carson City, Nevada Posts: 612 Jimmy - '00 GMC Jimmy SLT 90 day: 21.18 mpg (US) The White Gnat - '99 Suzuki Swift Team Suzuki 90 day: 51.87 mpg (US) Thanks: 240 Thanked 109 Times in 87 Posts RiderofBikes, During my past life as a land surveyor, I picked up a few concepts in my 27 year career. While maybe not "scientifically rigorous" statistics-wise, we had many methods for ensuring our results were good. We hardly did anything without checking it. If we measured a distance, we'd measure it twice. If we'd turn an angle, we'd turn it twice (or more). Same thing with elevations. The concept of "closing the traverse" is to make sure that when you come back to where you started from, your measurements in your notes ALSO say you came back to that same point. Same thing with "closing a level loop" When I read Metro MPGs article on "proper testing" with the "A-B-A" method, it made perfect sense to me as a surveyor. So while maybe not statistically rigorous, I think three "sets" of data can tell you something IF they're consistant. The first one gives you a number. The second one gives you a comparison. Then the third one confirms (or disputes) that your set is consistant. For example, you make 3 "A" runs (two way, of course) and get 40.2, 40.1 and another 40.1. Actually, that would be 6 runs, since these are all the averages of the "out and back". But I would personally be very satisfied to go away thinking the results were 40.133 or whatever it comes out to. (actually, 40.1 taking "significant figures" into account). However, if the runs were 40.2, 41.8, and 33.4, I'd want to do some more, because the spread was so great. Doing two-way runs is EXTREMELY important because it automatically averages-out so many error factors. Grade, wind, etc. If one-way is bad because it's not level, doing it the other way will compensate because it's not level, but in the opposite direction. Same thing with wind. (within reason, of course - doing it on a hill so steep you have to go up it in low gear and brake on the way back down obviously won't give you good results.) But back to the surveying, my wise old bosses introduced a concept called "does it LOOK right?" A newer member on the crew once tried to stake a new property line out into the street. The boss said, "It doesn't matter how powerful your calculator is, we KNOW that property line doesn't go out into the street". So if a set of three runs "looks good" (tight grouping), then I'm happy with it. But if it's "all over the place", then we need to do more until we figure out where it should be. Anyway, sorry if this is a rant, but for me, I don't (usually) need more than a few runs to be able to determine what I want. For example, yesterday I did 3 out & back runs in the "A" configuration. Two-way averages were: 30.95, 30.95, and 30.80. Overall average of 30.90. I considered this "good", since the maximum spread was only 0.15 mpg. Then I folded the mirrors back to see what the result would be. Two way averages were: 30.95, 31.05, and 31.05. Overall average of 31.02. When you round off for significant figures (ScanGauge only reads to a tenth of a mpg), you get 30.9 vs. 31.0, or one tenth of a mpg gain by folding the mirrors. (I did do a final "A" run with the mirrors back out, that came out at 30.65. Not sure why it was so much less than the first 3, but at least I knew the "B" test going up wasn't because "conditions" went up). Including that with the 3 original "A" tests still only shows the mirror fold as less than 0.2 mpg gain. In my mind, I got the answer I was looking for: "It helps, but precious little". Considering the lack of rearward visibility and potential attention from the cops, I think I'm leaning towards "not worth it". To me, that's all I really wanted to learn from that test. I also learned that same day that adding about 10 more psi of air to my tires seems to be good for 1.3 mpg!!! Now there's something that IS worth it!!!! Last edited by wmjinman; 10-15-2012 at 04:37 PM.. Reason: to correct spelling

 Thread Tools

 Similar Threads Thread Thread Starter Forum Replies Last Post MetroMPG EcoModding Central 438 02-27-2020 12:37 PM Fabio Hybrids 158 01-12-2013 11:59 AM MetroMPG Fossil Fuel Free 30 07-03-2008 12:25 PM

 -- EcoModder Skin -- Mobile / Lightweight skin Contact Us - EcoModder.com - Archive - Recent Posts - Top

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.
Content Relevant URLs by vBSEO 3.5.2
All content copyright EcoModder.com