Stat-Teaser April 2016

The Stat-Teaser Logo

April 2016

In this issue:
1. Version 10 of Design-Expert® Software Released!
2. Free Webinar on What's New and Exciting in v10 of Design-Expert® Software by Pat Whitcomb
3. Register Now for the 6th European DOE User Meeting & Workshop May 18-20, 2016 in Leuven, Belgium!
4. Get Up to Speed on DOE with Our Instructor-Led Workshops
5. Complete Your DOE Package with Stat-Ease Consulting Services
6. Practical Aspects for Designing Statistically Optimal Experiments by Mark Anderson

Version 10 of Design-Expert® Software Released!

“Theory guides, experiment decides.”
–– I. M. Kolthoff
Stat-Ease is pleased to announce the release of Design-Expert® software, version 10 (DX10)! Code-named "Kolthoff" in honor of the renowned Dutch chemist and educator who taught at the University of Minnesota for more than 60 years, version 10 provides many powerful new tools for optimizing your product or process. Design-Expert software for design of experiments (DOE) offers:

  • Two-level factorial screening designs: Identify the vital factors that affect your process or product so that you can make breakthrough improvements.
  • General factorial studies: Discover the best combination of categorical factors, such as source versus type of raw material supply.
  • Response surface methods (RSM): Find the optimal process settings to achieve peak performance.
  • Mixture design techniques: Discover the ideal recipe for your product formulation.
  • Combinations of process factors, mixture components, and categorical factors: Mix your cake (with different ingredients) and bake it too!
  • Design and analysis of split plots: Make your experiment far easier by grouping hard-to-change process factors and/or mixture components.
Some highlights of what's new in DX10 (with customer comments below) include:
1. The Design Wizard provides guidance for less sophisticated users: If you are not sure where to start with your experiment, follow through with a short series of questions to get an answer.

“Great for newbies or just to confirm my own independent selection.” 

“It helps the novice quickly and wisely build a design to suit their needs.”

2. Multiple Graphs view (factor profiler): Seeing effects from input variables side by side shouts out their relative impacts.

“Excellent! Very useful. Love this! The All Factors profiles plot is definitely is a good option.”

3. Optimal split plots for Response Surface and Combined (mixture-process) designs: They make it far easier as a practical matter to experiment when some factors cannot be easily randomized, e.g., first mix your cakes by batches (hard) and then bake them at varying temperatures (hard) for differing times (easy).

“Excellent! Great addition! It is very important that this is now possible.”

4. Automatic Model Selection tool for choosing Criterion and Selection method: Let Design-Expert work out what approach will be best depending on the type of experiment you conducted.

"Love the Autoselect tool!"
“Excellent! AICc is a very useful new feature, it worked better than other software.”

5. Power and sample size calculator for pass-fail (binomial) responses during build: Design an experiment with sufficient runs to be fairly certain that you will not overlook an important discovery.
 “Huge kudos!”

 This is just a sampling of the many great features you'll find in version 10. For a list of all that's new, click here

Free 30-day Trial
Give version 10 a try by downloading the free 30-day trial from our web site.  See for yourself how powerful and easy-to-use DOE software can be!

If you have an older version of Design-Expert software, purchase an upgrade to v10 and save a considerable amount off the cost of a new license.  Purchase an upgrade online or contact for help. 

Annual Network Licenses
If you have an annual network license, the end-user in charge of the license should have already received a free upgrade to v10 (as long as your license is current).  For help finding it, contact

Academic Licenses
For academic inquiries, click here.

For more information on version 10, check out our web site. To purchase a license, click here, and quickly start optimizing your product or process with this powerful, easy-to-use and well-loved DOE software program!

Go back to the top

Free Webinar in April on What's New and Exciting in v10 of Design-Expert® Software

Sign up now for a free webinar on What's New and Exciting in v10 of Design-Expert Software! Pat Whitcomb, Founder of Stat-Ease, Inc., welcomes you to version 10 of Design-Expert software (DX10) for design of experiments (DOE). He hits on the highlights of what’s new and what’s in it for you. For a preview of what’s in store, look here. As a practical matter, the most valuable addition may well be the extension of split-plot design and analysis to response surface and combined designs. Pat provides an enlightening demonstration of this and many other cool new features that provide convincing evidence of why version 10 is a significant step forward from version 9. This webinar will be recorded and posted to our web site after the 25th so that those of you who can't attend the live presentation can watch it at your convenience.

Monday, April 25th at 11:00 am - 12:00 pm CDT*

*To find out what time the meeting is in your city, click here for a link to The World Clock Meeting Planner.

Go back to the top

Register Now for the 6th European DOE User Meeting & Workshop May 18-20, 2016 in Leuven, Belgium!

 Grand Beguinage, Leuven, Belgium
The Grand Béguinage of Leuven

Stat-Ease, Inc. and CQ Consultancy are pleased to announce the 6th European DOE User Meeting and Workshops on May 18-20, 2016 in Leuven, Belgium. It will be held at the Faculty Club, which is part of the historic Grand Béguinage*. The béguinage originated in the early 13th century and is a UNESCO World Heritage location. This bi-annual event is always a favorite with attendees—both informative and fun!  Besides the interesting presentations and learning opportunities, this year's event in Leuven will give you an opportunity to check out the music, architecture and beer that Belgium is famous for!

At the DOE user meeting on the 19th–20th, we will explore the latest design of experiments (DOE) techniques, and demonstrate great new features in just-released Design-Expert® software, version 10! There will be presentations by keynote speakers and DOE experts, as well as practical case study applications by actual Design-Expert software users. In addition, you will have the opportunity to get help from DOE consultants on your own particular applications.

 Our expert trainers are offering two pre-meeting workshops on May 18th:

1. Figuring Out Factorials by CQ Consultancy
2. Optimal Formulation and Processing of Mixtures via Combined Designs by Stat-Ease, Inc. (Prerequisite: Familiarity with factorials.)

Sign up for one of these two workshops and learn practical DOE skills that you can take home and apply. 

Special Event the First Night of the Meeting
On the evening on the 19th, there will be a special event for all attendees. We'll start out with a concert by the choral group, Currende. Directed by Erik Van Nevel, Currende specializes in 16th and 17th music and the great works of Bach and Mozart. Listen to a sample of their beautiful music at the above link.  

After the concert we will head out for the conference dinner and ultimate beer experience at The Capital. In the backyard of the largest beer producer in the world, The Capital offers the largest beer selection in the world. This is an event you won't want to miss! 

Dates & Location
18 – 20 May 2016: DOE User Meeting (Includes dinner and special event on the 19th) and Pre-Meeting Workshops

Faculty Club
Groot Begijnhof 14,
T. +32 (0) 16 32 95 00
F.  +32 (0) 16 32 95 02

For more information and to register for the 6th European DOE User Meeting and Workshops click hereWe hope to see you in Leuven!

*According to the UNESCO World Heritage Convention, "Béguines were women who dedicated their lives to God without retiring from the world. In the 13th century they founded the béguinages , enclosed communities designed to meet their spiritual and material needs. The Flemish béguinages are architectural ensembles composed of houses, churches, ancillary buildings and green spaces, with a layout of either urban or rural origin and built in styles specific to the Flemish cultural region. They are a fascinating reminder of the tradition of the Béguines that developed in north-western Europe in the Middle Ages." []

Go back to the top

Get Up to Speed on DOE with Our Instructor-Led Workshops

Whether you are just starting out or are a practiced experimenter, Stat-Ease has a workshop for you.  Find a list of our upcoming public workshops below.  We also offer a large variety of private on-site workshops, including industry-specific classes. This is a cost-effective and convenient option if you have 5 or more people to train. For more information on private workshops, click here

Experiment Design Made Easy (EDME)
May 3-4: Minneapolis, MN
Jun 20-21: San Jose, CA
Aug 8-9: Edison, NJ
Sep 20-21: Minneapolis, MN
Nov 7-8: San Jose, CA
$1295 ($1095 each, 2 or more)

Response Surface Methods for Process Optimization (RSM)
Jun 22-23: San Jose, CA
Sep 22-23: Minneapolis, MN
$1295 ($1095 each, 2 or more)

Mixture and Combined Designs for Optimal Formulations (MIXC)
Jul 19-21: Minneapolis, MN
Oct 17-19: Edison,  NJ
$1695 ($1495 each, 2 or more)

Workshops are limited to 16. Receive a $200 quantity discount per class when you enroll 2 or more students, or when a single student enrolls in consecutive 2-day workshops. For more information, contact Rachel via e-mail or at at 612.746.2030.

Go back to the top

Complete Your DOE Package with Stat-Ease Consulting Services

Stat-Ease Consulting Group
Back row from left to right: Brooks Henderson, Mark Anderson, Wayne Adams
Front row from left to right: Pat Whitcomb, Shari Kraber, Martin Bezener

Stat-Ease's consulting team recently expanded with the hiring of Martin Bezener, a PhD graduate in Statistics from the University of Minnesota. He makes a great addition to our well-qualified team of design of experiment (DOE) experts. We are pleased to offer our consulting services to support all your experimentation needs. We provide mentoring/coaching to help you set up the right design for the problem, and then guide you through the proper analysis of your data. Our consulting services can be geared to either teach you how to do DOE effectively on your own, or we can be the experts in your back pocket! Services may be purchased for as little as an hour of time, or you can contract us for an entire project. Our standard rate is $275/hour, with a 20% discount available when purchasing consulting (up to 10 hours) bundled together with software or training.  We also offer generous academic discounts.

Our complete package of design of experiment (DOE) services for all your experimentation needs includes:

If you have any questions about our services, contact for help. Leverage our expertise to make your experimentation a success!

Go back to the top

Practical Aspects for Designing Statistically Optimal Experiments

(Editor’s note: This is an outtake of an article by Mark Anderson and Pat Whitcomb that was published in the Journal of Statistical Science, Vol. 2, No. 3, March 2014.)

Due to operational or physical considerations, standard factorial and response surface method (RSM) design of experiments (DOE) often prove to be unsuitable. In such cases a computer-generated statistically-optimal design fills the breech. This article explores vital mathematical properties for evaluating alternative designs with a focus on what is really important for industrial experimenters.

Statistically optimal designs emerged over a half century ago (Kiefer, 1959) to provide the following advantages over classical templates for factorials and RSM:

  • Efficiently filling out an irregularly shaped experimental region such as that shown in Figure 1 (Anderson and Whitcomb, 2005),
  • Minimizing the runs to just what is needed to fit the assumed polynomial model,
  • Accommodating unusual requirements concerning either the number of blocks or the number of runs per block,
  • Handling a combination of factor types, such as continuous, discrete, categorical and mixture.

Figure 1: Example of irregularly-shaped experimental region (a molding process)

Over time a number of criteria labeled alphabetically became favored by industrial experiments for optimal designs (Box and Draper, 2007), of which two will be covered in this article: I-optimal and D-optimal. Our focus will be kept to RSM.

What is a “Good” Experiment Design?
To answer this important question, let’s start with a wish-list for choosing a “suitable” experiment design derived from one provided by George Box (Box, 1982)—the co-inventor of RSM (Box and Wilson, 1951):

    1. Allow the chosen polynomial to be estimated well.
    2. Give sufficient information to allow a test for lack of fit by:
      a. Having more unique design points than model coefficients, and
      b. Providing an estimate of “pure” error, i.e., replicates.
    3. Remain insensitive to outliers, influential values and bias from model misspecification.
    4. Be robust to errors in control of the factor levels.
    5. Permit blocking and sequential experimentation.
    6. Provide a check on variance assumptions, e.g., studentized residuals are normal with a mean of zero and constant variance.
    7. Generate useful information throughout the region of interest, i.e., provide a good distribution of standard error of predictions.
    8. Do not contain an excessively large number of trials.

When applying RSM, industrial experimenters generally choose as step one a quadratic polynomial, which are remarkably versatile for empirical modeling. For this purpose, the central composite design (CCD), also known as the Box-Wilson in honor of the developers, scores well on all the desired attributes. However, standard layouts like the CCD are not a good fit for non-cuboidal regions such as that illustrated in Figure 1. For situations like this or others spelled out in the Introduction, optimal designs are seemingly the panacea. However, as we will discuss further, you had best keep in mind that “designing an experiment should involve balancing multiple objectives, not just focusing on a single characteristic” (Myers, Montgomery and Anderson-Cook, 2009).

Purely Optimal Designs—Comparing I Versus D as the Criterion
Although there are many variations on the theme of optimal design, the following two criteria are the ones primarily used in industrial experimentation:

  • I-optimal (also known as “IV”) to minimize the integral of the prediction variance
  • D-optimal to minimize the volume of the confidence ellipsoid for the coefficients and thus maximize information on the polynomial coefficients.

Rather than getting mired down in the mathematical details for these two criteria and the multitude of algorithms for applying them (readily available from a vast array of references—some already cited here), we will focus on how they differ in actual application to RSM experiments designed to fit quadratic polynomials such as the equation show above. For this purpose a good tool for comparison is the standard error (SE) plot, such as those shown in Figure 2 for a 12-run RSM design on one factor where the points are picked I- versus D-optimally.

Figure 2: Standard error plots for I (left) versus D optimal (right) for a 12-run, one-factor RSM design

Comparing these plots side-by-side provides a picture of the relative quality of predicted response at various locations spanning the experimental region (shown here in terms of coded units from the center). The greater replication of points by the I-criterion is desirable for RSM because it lowers the standard error of prediction at the center—the point of greatest interest—and provides a fairly flat profile for a broader (relative to the D-criterion) range in the middle of the experimental region.

Another way to compare designs is via the fraction of design space (FDS) plot (Anderson-Cook, Borror and Montgomery, 2009), which consists of a single line for a given design, thus allowing display of the prediction variance for several designs at once. Figure 3 lays out the FDS curves for the aforementioned 12-run RSM design done by the competing criterion—I versus D.

Figure 3: FDS plots for I versus D optimal for a 12-run, one-factor RSM design

The legend in Figure 3 provides benchmarks on the standard error minimums, averages and maximums. For RSM purposes, the I-optimal provides a desirable tradeoff of being higher at its maximum (not good) but lower on average (good) than the D-optimal design.

At this stage we put I-optimal at the forefront. However, neither of the two optimal designs under consideration provide for a lack-of-fit (LOF) test. Therefore an experimenter cannot assess whether the model they chose provides an adequate approximation of the true response. It’s vital to keep in mind that “no postulated model can ever be assumed to be fully correct [therefore] the basic assumptions underlying the alphabetic-optimality approach are often unrealistic from the practical viewpoint of actually designing real experiments” (Draper & Guttman, 1988).

Modifying Optimal Designs to Make Them More Robust to Model Misspecification
We will now do another comparison of point selection using I- vs D-optimal on a one-factor design for a quadratic model, but this time the designs will be modified with LOF points to check for model misspecification. We recommend 4 runs for these ‘check-points’—chosen to maximize the minimum distance from existing design points; thus filling ‘holes’ in the experimental space. This is known as the “distance” criterion. Figure 4 provides a side-by-side view, by way of standard error plots, of how this modification affects the spread of points compared to the purely optimal selection shown in Figure 2.

Figure 4: SE plots for LOF-modified I-optimal (left) versus LOF-modified D-optimal (right) for a 12-run, one-factor RSM design

Observe how gaps in the experimental range have been plugged.

Next let’s recalculate the FDS for the two alternative optimal criteria.

Figure 5: FDS plots for LOF-modified I versus D optimal for a 12-run, one-factor RSM design

Compare and contrast the curves and data on Figures 3 and 5—the first being purely optimal and the second modified with lack-of-fit points. Notice that by all statistical measures and the curves themselves that not much differs. In this case the advantage of having a check for model misspecification outweighs the minor loss in optimality and slight degradation in FDS quality.

Extending These Findings to Two Factors
A similar case can be made for two factors and, by extension, beyond—once the minimum points needed to fit the chosen polynomial model are selected via an optimal criterion, adding LOF points causes little harm and they create a lot of good. Figure 6 lays out the points on SE plot for a 14-run, I-optimal, RSM-quadratic design with zero versus four LOF points.

Figure 6: SE plots for purely I-optimal (left) versus LOF-modified (right) for a 14-run, two-factor RSM design

The space-filling effect of shifting some points from optimal to distance-based criterion (for LOF) is very apparent when comparing these plots from left to right. This comes at little cost to the I-optimal design quality as evidenced by the FDS curve and properties laid out in Figure 7.

Figure 7: FDS plots for purely I-optimal versus LOF-modified I-optimal for a 14-run, two-factor RSM design

However, when we make the same comparison for D-optimal designs, the trade-off of 4 purely optimal points for ones chosen by distance (layouts shown in Figure 8) to provide LOF does not go quite as well—see how the FDS curve shifts in Figure 9 and note the degradation in D-optimality as evidenced by the determinant results being increased.

Figure 8: SE plots for purely D-optimal (left) versus LOF-modified D-optimal (right) for a 14-run, two-factor RSM design

Figure 9: FDS plots for purely D-optimal versus LOF-modified D-optimal for a 14-run, two-factor RSM design

This lends further support for the use of I over D-optimal designs for response surface method optimization.

Bolstering the Design with Replicates
Replicates are needed to provide the pure error needed for a LOF test. In the cases presented above they came with the optimal portion of the designs. A better way to build a practical experiment is to detail the number of replicates and choose them by the optimal criterion—we recommend a minimum of 4 points being selected in this manner. This provides enough degrees of freedom for a reasonably-powerful LOF test.

In consideration of the practical aspects of algorithmic design for RSM we recommend adding by distance-base at least 4 points for testing lack of fit, even though this makes the experiment less alphabetically optimal. This a good trade off.  Furthermore, in physical experiments it is desirable to build in an estimate of experimental error based on at least 4 degrees of freedom via replicated design point(s). For choice of optimal criterion we advise using I- for empirical modeling (RSM)—reserving D-optimal (because of their more-precise estimation of model coefficients) for factorial screening.

However, even with these practical aspects incorporated, optimal design cannot make up for:

  • Choosing the factors that are not among the vital few.
  • Designing for an inadequate empirical model.
  • Going out of the region of operability.
  • Measuring the wrong responses.

“The exact functional relationship is usually unknown and possibly unknowable. We have only to think of the flight of a bird, the fall of a leaf, or the flow of water through a valve to realize that we are likely to be able to approximate only the main features of such a relationship.”
– Box & Draper


Anderson, M. J. and Whitcomb, P.J. (2007) RSM Simplified, Optimizing Processes Using Response Surface Methods for Designs of Experiments, Computer-Generated Optimal Designs,” 7: 149-150, New York: Productivity Press.

Anderson-Cook, C.M., Borror, C. M. and Montgomery, D. C. (2009) “Response surface design evaluation and comparison,” Journal of Statistical Planning and Inference, 139: 629–641.

Box, G. E. P., and Wilson, K B. (1951) “On the experimental attainment of optimum conditions,” Journal of the Royal Statistical Society, Series B 13:1-45.

Box, G. E. P. (1982) “Choice of Response Surface Design and Alphabetic Optimality,” Utilitas Mathematica, 21B: 11-55.

Box, G. E. P., and Draper, N. R. (2007) Response Surfaces, Mixtures, and Ridge Analyses, “The Alphabetic Optimality Approach,” 14.4: 471-481, New York: Wiley.

Draper, N. R. and Guttman, I. (1988) “Rationalization of the “Alphabetic-Optimal” and “Variance Plus Bias” Approaches to Experimental Design,” Technical Report 841, Department of Statistics, University of Wisconsin.

Kiefer, J. (1959). “Optimum Experimental Designs,” Journal of the Royal Statistical Society, Series B 21: 272–319.

Myers, R. H., Montgomery, D. C. and Anderson-Cook, C.M. (2009) Response Surface Methodology, 3rd Edition, New York: Wiley.

—Mark Anderson,

Go back to the top