DOE FAQ Alert

Issue: Volume 2, Number 7
Date: July 2002
From: Mark J. Anderson, Stat-Ease, Inc.

Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed previous DOE FAQ Alerts, go to the links below. Feel free to forward this newsletter to your colleagues. They can subscribe by going to http://www.statease.com/doealertreg.html.

I offer the following link as an appetizer: http://www.hero.ac.uk/culture_and_sport/domestic_science1598.cfm (link no longer active) which reports a study on why soccer teams benefit from being on their home field (note the recent unexpected success of South Korea in the World Cup games held in their country). Evidently the referees consider the value of their lives when contemplating whether to card a home team player. Surprised? Perhaps it may be best to take people out of the equation altogether by building robots that play soccer. Believe it or not, there's an organization called RoboCup that hopes by the year 2050 to develop a team of fully autonomous humanoid robots that can win against the human world soccer champion team. To see pictures from this year's competition (which ran concurrent with World Cup soccer), copy this path into your browser http://news.nationalgeographic.com/news/2002/06/0617_020617_TVrobots.html.

Here's what I cover in the body text of this DOE FAQ Alert (topics that delve into statistical detail are rated "X"):

1. Stat-Teaser alert: See the latest issue of our newsletter featuring "Goldilocks and the Three Chairs"
2. X-FAQ: The dangers of deleting factors
3. X-FAQ: Generating fractional factorial designs - not as simple as it seems
4. Events alert: Stat-Ease is going to New York City to exhibit at ASA's Joint Statistical Meetings (JSM)
5. Workshop alert: See our upcoming schedule of classes - we're visiting Seattle and San Jose soon
6. Book alert: Recommended summer reading - a rousing sea story with 'under-the-gun' analysis of variance in shooting at ships

PS. Quote for the month - Luck versus cause and effect.

1 -Stat-Teaser Alert: See the latest issue of our newsletter featuring "Goldilocks and the Three Chairs"

If you did not get a printed copy, see the latest Stat-Teaser newsletter at http://www.statease.com/newsltr.html. The feature article, "Goldilocks and the Three Chairs," details a simple comparative taste test designed by Stat-Ease consultant Shari Kraber to evaluate chairs for the company's new training room. She also provides a number of helpful hints on how to use Design-Expert® software. Check it out if you've not done so already.

2 - X-FAQ: The dangers of deleting factors

-----Original Question-----

From: A student of MoreSteam.com's Six Sigma class
Re: The Trebuchet simulation*

"On the trebuchet I first did a 1/16 two-level factorial on all 7 items. Then I deleted two factors (time and type of material in missile) and did a foldover for all factors. This analysis seems to indicate that there is no combination of factors that will hit with a 95% confidence level."

*(Discussed in item 3 of Volume 2, Number 1, January 2002 issue of the DOE FAQ Alert. Click http://www.moresteam.com/trebuchet.cfm (no longer available for public use) to run the simulation and try your hand at knocking down a castle.)

Answer:

First of all, let's look at the design you chose and how you followed up. You first studied 7 factors in only 8 runs (mathematically described as 2^(7-4)). This DOE aliases main effects with two-factor interactions (technically known as Resolution III). Thus, some of the factors you thought were significant actually represented aliased two factor interactions. Therefore, in cases like this it becomes very dangerous to delete factors before doing a foldover. Cuthbert Daniel, a pioneer in developing tools for two-level factorials, said: "It is my own practice to recommend great caution in dropping factors. The simplification so produced is illusory."*

I took a bit different approach.
a. Run 7 factors in 16 runs to achieve Resolution IV, thus aliasing main effects only with three-factor interactions. However two-factor interactions (2FI's) remain aliased with other 2FI's.
b. Seeing some 2FI's coming out significant, I did a semi-foldover to de-alias these.

For a description of my strategy of experimentation (admittedly a bit tricky!), see an article I wrote based on a talk for the 2000 Annual Quality Congress sponsored by the American Society of Quality (ASQ). See http://www.statease.com/pubs/breaddoe.pdf for the technical paper. ASQ's "Quality Progress" magazine plans to publish this as an article, but I do not know when.

For students at less than your level of training (Six Sigma Black Belt), I'd recommend that they avoid the cans of worms that both you and I opened on aliases (and subsequent foldovers) by doing at least a 48-run irregular fraction on the 7 factors for the trebuchet. They would then get the right answers, not only for main effects, but also the 2FI's. Statisticians at Stat-Ease have a neat design option up their sleeve: 7 factors in only 30 runs that achieves the same resolution as the 48-run irregular (3/8ths) fraction! This will be presented at the Fall Technical Conference (October 17-18, Valley Forge, PA) and incorporated into our next version of Design-Ease® and Design-Expert software.

*("Sequences of Fractional Replicates in the 2^p-q Series," Journal of the American Statistical Association (ASA), 67, 1962, page 417.)

(Learn more about fractional two-level factorial designs by attending the 3-day computer-intensive workshop "Experiment Design Made Easy." See http://www.statease.com/clas_edme.html for a description. Link from this page to the course outline and schedule. Then, if you like, enroll online.)

3 - X-FAQ: Generating fractional factorial designs is not as simple as it seems

-----Original Comment-----

From: Another student of MoreSteam.com's Six Sigma class
Re: "DOE Simplified"* Practice Exercise 5-1

"This is a 7 factor, 1/4th fraction of a two-level design. For the 32 runs the first 5 factors (A to E) make a full factorial table (see + and - designations). Should not the 6th factor F be (-) or (+) based upon the product of the first 5 factors? And the signs of 7th be the product of first 4 factors? If not, then how are the signs for F and G tabulated?"

*(See http://www.statease.com/doe_simp.html.)

Answer:

It's logical, but incorrect, to assume that since the 5th factor in a 2^5-1 fractional factorial is computed by multiplying the first four factors (E=ABCD as noted in 2nd paragraph on page 90 of "DOE Simplified"), that the 6th factor in a 2^7-2 should equal the product of the first five factors. This could be done, but it would not give the optimal design resolution. The design provided in Appendix 2-4 of the book is generated by F=ABCD and G=ABCE. The end result is a Resolution IV design. If you don't believe that this is best, select 7 factors in 32 runs in Design-Ease or Design-Expert software. Then click the option to "Make generators editable." Change the default selection for F to ABCDE and enter whatever else (some combination of letters ABCD, but not ABCDE again!) you want for generating factor E. Good luck! (Don't worry, the software offers a "Set generators to default" button to re-set things to their optimal settings.)

4 - Events alert: Stat-Ease is going to New York City to exhibit at ASA's Joint Statistical Meetings (JSM)

Stat-Ease is going to New York City to exhibit at ASA's (American Statistical Society) JSM conference on August 11-15. See http://www.amstat.org/meetings/jsm/2002/ for details from ASA.

Click http://www.statease.com/events.html for a listing of where Stat-Ease consultants will be giving talks and doing DOE demos. We hope to see you sometime in the near future!

5 - Workshop alert: Upcoming schedule of classes - visiting Seattle and San Jose soon

See http://www.statease.com/clas_pub.html for schedule and site information on all Stat-Ease workshops open to the public. Note that we will soon be doing sessions of "Experiment Design Made Easy" on the West Coast in:
- Seattle, Washington on July 9-11
- San Jose, California on August 20-22.

To enroll in these or other classes, click the "register online" link on our web site or call Stat-Ease at 1.612.378.9449. If seats remain available, bring along several colleagues and take advantage of quantity discounts in tuition, or consider bringing in an expert from Stat-Ease to teach a private class at your site. Call us to get a quote.

6 - Book alert: Recommended summer reading - a rousing sea story with 'under-the-gun' analysis of variance in shooting at ships

I recently read "Flying Colours," a very enjoyable book written by C.S. Forester in 1938 - one of a series featuring Horatio Hornblower's adventures battling Napoleon on sea and on land. This makes great summertime reading! I especially liked the following passage that describes the hero trying to shoot a flotilla of attacking French boats from a rocking platform on a cutter:

"Hornblower eased the gun round with the handspike until the aim was true, drew himself up and stepped out of the way of the recoil, lanyard in hand. Of necessity he was far more doubtful of the range than the direction, and it was vital to observe the fall of the shot. He took note of the motion of the cutter on the swell, waited for the climax of the roll and jerked the lanyard. The gun roared out and recoiled past him. He sprang sideways to get clear of the smoke. The four seconds of the flight of the shot seemed to stretch out indefinitely and then at last he saw the jet of water leap into brief existence, fully 200 yards short and 100 yards to the right. That was poor shooting!

He sponged out the gun and reloaded it ... and ran the gun out again. It was necessary he realized to get acquainted with the weapon if he wanted to do any fancy shooting with it, so he that he made no alteration in elevation, endeavored to lay the gun exactly as before and jerked the lanyard at as nearly the same instant of the roll as possible. This time it appeared that the elevation was correct for the shot pitched well up to the boat but it was off to the right again, 50 yards off at least. It seemed likely that the gun therefore had a tendency to throw to the right. He trained the gun round a trifle to the left and still without altering the elevation fired again - too far to the left and 200 yards short again!

Hornblower told himself that a variation of 200 yards in the fall of shot from a six pounder at full elevation was only to be expected and he knew it to be true, but that was cold comfort to him. The powder varied from charge to charge, the shot were never truly round, quite apart from the variations in atmospheric conditions and in the temperature of the gun. He set his teeth, aimed and fired again - short and a trifle to the left. It was maddening!"

Imagine what a person could've done back then with the tools of DOE, but think how difficult it would be to perform a proper experiment under battle conditions. Does the hero, Horatio Hornblower, gain control of his gun well enough to blow the French out of the water before they sink him? You must get the book to find out. Check your local library for it, or go to Amazon at http://www.amazon.com and type "Forester" into the search box.

I hope you learned something from this issue. Address your questions and comments to me at:

[email protected]

Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (http://www.statease.com)
Minneapolis, Minnesota USA

PS. Quote for the month - Luck versus cause and effect.

"Shallow men believe in luck. Strong men believe in cause and effect."

- Ralph Waldo Emerson

Trademarks: Design-Ease, Design-Expert and Stat-Ease are registered trademarks of Stat-Ease, Inc.

Acknowledgements to contributors:

- Students of Stat-Ease training and users of Stat-Ease software
- Fellow Stat-Ease consultants Pat Whitcomb and Shari Kraber (see http://www.statease.com/consult.html for resumes)
- Statistical advisor to Stat-Ease: Dr. Gary Oehlert (http://www.statease.com/garyoehl.html)
- Stat-Ease programmers, especially Tryg Helseth (http://www.statease.com/pgmstaff.html)
- Heidi Hansel, Stat-Ease marketing director, and all the remaining staff.

Interested in previous FAQ DOE Alert e-mail newsletters? To view a past issue, choose it below.

#1 - Mar 01, #2 - Apr 01, #3 - May 01, #4 - Jun 01, #5 - Jul 01 , #6 - Aug 01, #7 - Sep 01, #8 - Oct 01, #9 - Nov 01, #10 - Dec 01, #2-1 Jan 02, #2-2 Feb 02, #2-3 Mar 02, #2-4 Apr 02, #2-5 May 02, #2-6 Jun 02, #2-7 Jul 02 (see above)