Two-Level Factorial

Choosing Effects to Model

Under the Effects tab, the program displays the absolute value of all effects (plotted as squares) on a half-normal probability plot. On the right is the Pareto plot (more on that later). Color-coding provides details whether the effects are positive or negative.

Note the message on your screen: “Select significant terms – see Tips.” You must choose which effects to include in the model. If you proceed without doing so at this point, you will get a warning message stating “You have not selected any factors for the model.” The program will allow you to proceed, but with only the mean as the model (no effects), or you can opt to be sent back to the Effects view (a much better choice!).

You can select effects by simply clicking on the square points. Start with the largest effect at the right side of the plot, as shown below. Notice that it also becomes selected on the Pareto plot.

By default the red “error line” will be placed such that it represents the smallest 50% of the effects. It is intended to be a visual guide to assist with selecting effects. The unimportant effects should line up on a line near zero. If the line doesn’t match the small effects, you can click and drag it to line up with group of smallest (on the left) effects.

Keep selecting individual effects from right to left until you’ve selected all the effects that are off the line. In this case, the last effect to select is factor C. There is a big gap between C and the smaller effects that line up near zero. This gap is often a good indication to stop selecting effects.

You can also select multiple effects at once by dragging a box around them.

To really see the magnitude of the chosen effects, the program displays them on an ordered bar chart called the Pareto plot. Notice the vertical axis shows the t-value of the absolute effects. This dimensionless statistic scales the effects in terms of standard deviations. In this case, it makes no difference to the appearance of the chart, but when you encounter botched factor levels, missing data, and the like, the t-value scale provides a more accurate measure of relative effects. Click the next biggest bar and notice it is identified as ABD as shown below (lower right).

Notice that the ABD bar falls below the bottom limit, so click the bar again to deselect it.

Note

To see quantitative detail on the chosen model effects and those remaining for estimation of error, click Numeric. Click the full window icon at the top for a better view. This screen enables another method of model selection via the “Autoselect…” button. For now, we’ll stick with the model chosen on the half-normal plot. Autoselect is more useful for response surface designs and will be detailed in those tutorials.

ANOVA and Statistical Analysis

It is now time to look at the statistics in detail with the analysis of variance (ANOVA) table. Click the ANOVA tab to see the selected effects and their coefficients. By default, the program provides annotations in blue text.

Check the probability (“p-value”) for the Model. By default, the program considers values of ≤ 0.05 to be significant. This can be changed via Edit > Preferences > Math Preferences > Math Analysis.

Inspect the p-values for the model terms A, C, D, AC, and AD: All pass the 0.05 test with room to spare.

Note

Context-sensitive Help: Definitions for numbers on the ANOVA table can be obtained by right clicking and choosing Help. Try this for the Mean Square Residual statistic, as shown below.

Observe the other panes of the ANOVA output for further statistics such as R-squared and the like. Refer to the annotations and also access Help for details. Take a look at the Coefficients tab to see estimates for the model coefficients and their associated statistics. Lastly, click the Coded Equation and Actual Equation tabs for the predictive equations both in coded and uncoded form.

Rather than belabor the numbers, let’s move on and ultimately let the effect graphs tell the story. However, first we must do some diagnostics to validate the model.

Validate the Model

Click the Diagnostics tab to generate a normal probability plot of the residuals.

By default, residuals are studentized – essentially a conversion to standard deviation scale. Also, they are done externally, that is, with each result taken out before calculating its residual. Statisticians refer to this approach as a “case deletion diagnostic.” If something goes wrong in your experiment or measurement and it generates a true outlier for a given run, the discrepant value will be removed before assessing it for influencing the model fit. This improves the detection of any abnormalities.

Note

Raw residuals: For standard two-level factorial designs like this, plotting the raw residuals (in original units of measure) will be just as effective. On the Diagnostics Toolbar, pull down the options as shown and choose Residuals (raw) to satisfy yourself that this is true (the pattern will not change appreciably).

We advise that you return to externally studentized scale in the end, because as a general rule this is the most robust approach for diagnosing residuals.

Ideally the normal plot of residuals is a straight line, indicating no abnormalities. The data doesn’t have to match up perfectly with the line. A good rule of thumb is called the “fat pencil” test. If you can put a fat pencil over the line and cover up all the data points, the data is sufficiently normal. In this case the plot looks OK, so move on.

Select the Resid. vs. Pred. (residuals versus predicted) tab, shown below.

The size of the residual should be independent of its predicted value. In other words, the vertical spread of the studentized residuals should be approximately the same across all levels of the predicted values. In this case, the plot looks OK.

Next select and maximize Resid. vs. Run to see a very useful plot that’s often referred to as “Outlier t” because it shows how many standard deviations (t-values) a given run falls off relative to what one would expect from all the others.

The program provides upper and lower red lines that are similar to 95% confidence control limits on a run chart. In this case none of the points stands out. Because this graph is plotted in randomized run order, the ordering of the points on your screen will be different than shown here. Normally, though, you should look for patterns, not just outliers. An obvious example would be a steady decrease in residuals from start to finish, in other words, a downward trend. That would be cause for concern about the stability of your system and merit investigation. However, by running the experiment in random order, you build in protection against trends in response biasing the results.

Note

What to do if a point falls out of the limits: If there were an outlier, you could click on it to get the coordinates displayed to the left of the graph. The program remembers the point. It will remain highlighted on other plots. This is especially helpful in the residual analysis because you can track any suspect point. This feature also works in the interpretation graphs. Give it a try! Click anywhere else on the graph to turn the point off.

Even more helpful may be the option to highlight a point as shown below.

This flags the run back in your design layout and elsewhere within the program so you can keep track of it. Check it out!

Skip ahead to the Box Cox plot. This was developed to calculate the best power law transformation. (Refer to Montgomery’s Design and Analysis of Experiments textbook for details.) The text on the left side of the screen gives the recommended transformation: in this case, “None.” That’s all you really need to know!

Note

For those of you who want to delve into the details, note that the Box-Cox screen is color coded to help with interpretation. The blue line shows the current transformation. In this case it points to a value of 1 for “Lambda,” which symbolizes the power applied to your response values. A lambda of 1 indicates no transformation. The green line indicates the best lambda value, while the red lines indicate the 95% confidence interval surrounding it. If this 95% confidence interval includes 1, then no transformation is recommended. It boils down to this: If the blue line falls within the red lines, you are in the optimal zone, so no change is necessary in your response transformation.

P.S. The Box Cox plot will not help if the appropriate transformation is either the logit or the arcsine square root transformation. See the program’s Help system write-up on “Response Transformations” for further details.

Go back and select the Report tab. Here you see the numerical values for diagnostic statistics reported case-by-case. Discrepant values will be flagged. In this case nothing is detected as being abnormal.

Examine Main Effects and Any Interactions

Assuming that the residual analyses do not reveal any problems (no problems are evident in our example), it’s now time to look at the significant factor effects.

On the analytical tool bar at the top of the screen, choose the Model Graphs tab. The AC interaction plot comes up by default. (If your graph displays the x-axis in coded units, return to actual units by choosing Display Options, Process Factors - Actual.)

The “I-Beam” symbols on this plot (and other effect plots) depict the 95% least significant difference (LSD) interval for the plotted points.

Those points that have non-overlapping intervals (i.e. the LSD bars don’t intersect or overlap from left to right through an imaginary horizontal line) are significantly different.

Note

How to do pairwise comparisons: An easy way to verify separation is to do a pairwise comparison. Click on any model prediction (for example the triangle in the middle of the red LSD bars on the left), and you will be shown the pairwise comparisons.

A horizontal line is drawn through the predicted mean of the highlighted point. Any vertical bars that overlap with this horizontal line indicate predicted means that are not significantly different from the selected point (for example, the red triangle at the right of the graph indicating the prediction for A+, C+). The legend will also tabulate which means are significantly different. Note that even though the displayed pairwise tests are two-sided, only half of the interval is displayed for easier interpretation.

Note also that the spread of the points on the right side of the graph (where Temperature is high) is smaller than the spread between the points at the left side of the graph (where Temperature is low.) In other words, the effect of formaldehyde concentration (C) is less significant at the high level of temperature (A). Therefore, the experimenters can go to high temperature and reduce the concentration of harmful formaldehyde, while maintaining or even increasing filtration rate. This combination is represented by the black square symbol at the upper right of the interaction plot.

The Factors Tool opens along with the default plot, pinned to the right side of the screen. Move the tool as needed by clicking the top blue border and dragging it. Click and drag the tool back to the right side of the screen until you see a blue shadow, then release to lock it back in place. This tool controls which factor(s) are plotted on the graph. At the bottom of the Factors Tool is a pull-down list from which you can also select the factors to plot. Only the terms that are in the model are included in this list.

Note

What happens if you pick a main effect that’s involved in interaction: Click the Term list down-arrow and select A. Notice that the Graphs Tool shifts from Interaction to One Factor.

More importantly, pay heed to the warning at the top of the plot of A (Temperature). It states “Warning! Factor involved in multiple interactions.” You should never try to interpret main effects plots of factors involved in interactions because they provide misleading information.

Let’s do something more productive at this stage: Go back to the Factors Tool and select from the Term list the other significant interaction, AD.

Notice on the Factors Tool that factors not already assigned to an axis (X1 or X2) display a vertical slider bar. This allows you to choose specific settings. The bars default to the midpoint levels of these non-axis factors.

Note

Using the factor sliders: You can change levels of fixed factors by dragging the bars, or by typing the desired level in the numeric space near the bottom of the Factors Tool. Check this out by grabbing the slide bar for factor C: Concentration and moving it to the left. Notice how the interaction graph changes.

It now becomes clear that a very high filtration rate can be achieved by going to the high stir rate (the red line for factor D). Click this high point to get all the details on its response and factor values. This is the optimum outcome.

To reset the graph to its default concentration, you could type “3” in the box at the bottom of the Factors Tool (below Stir rate). Factor C must be clicked and highlighted for this to take effect. You can also get the original settings back by pressing the Default button. Give it a try, but remember that going to low concentration of formaldehyde was a primary objective for this process-troubleshooting experiment, so be sure to slide C back to the left again.

You can view the AD interaction with the axes reversed by right-clicking the box next to D: Stir Rate and changing it to X1 axis.

It makes no difference statistically, but it may make more sense this way.

Note

One last thing: You can edit at least some text on many of your graphs by right-clicking your mouse. For example, on the interaction graph you can right-click the X1-axis label.

Then choose Edit Text. The program then provides an entry field. Try it!

Cube Plot

Now from the Graphs Toolbar select Cube to see the predicted response as a function of the three factors that created significant effects: A, C, and D.

This plot shows how three factors combine to affect the response. All values shown are predicted values, thus allowing plots to be made even with missing actual data. Because the factors of interest here are A, C, and D, the program picked them by default. (You can change axes by right-clicking any factor on the Factors Tool.)

Filtration rate is maximum at settings A+, D+, C- (upper front right corner with predicted response over 100), which also corresponds to the reduced formaldehyde concentration. Fantastic!!

Contour and 3D Plots of the Interaction

An interaction represents a non-linear response of the second order. It may be helpful to look at contour and 3D views of the interaction to get a feel for the non-linearity.

First select Contour to get a contour graph. The axes should come up as A (Temperature) and D (Stir Rate). If not, simply right click over the Factor Tool and make the appropriate changes.

You may be surprised to see the variety of colors – graduated from cool blue for lower response levels to warm yellow for higher values. To match the above graph, be sure you have set the concentration (C) to the low level.

Note

Stat-Ease contour plots are highly interactive. For example, you can click on a contour to highlight it. Then you can drag it to a new location. Furthermore, by right-clicking anywhere within the graph you can bring up options to add flags, add contours, or change graph preferences.

Now, to create an impressive-looking graph that makes it easy to see how things shape up from this experiment, select 3D Surface.

Move your mouse cursor over the graph. When it turns into a hand left-click and rotate the view however you like.

Before moving on to the last stage, take a look at the first interaction by going to the Factors Toolbar and from the Term list selecting AC. Move the slide bar on D: Stir Rate to the right (maximum level) to increase the response.

We have nearly exhausted all the useful tools offered by Stat-Ease for such a basic design of experiments, but there’s one more tool to try.

Confirmation

The last feature that we will explore appears at the lower left of the screen under the main branch of Post Analysis: Click the Confirmation node to make response predictions for any set of conditions for the process factors. Then click the Factors tab. When you first enter this screen, the Factors Toolbar palette defaults to the center points of each factor. Levels are easily adjusted with the slider bars or more precisely by using the Sheet view. In this case, the analysis suggests that you should slide the factors as follows:

• A (Temperature) right to its high level (+)

• B (Pressure) leave at default level of center point

• C (Concentration) left to its low level (-)

• D (Stir Rate) right to its high level (+)

The program uses the model derived from experimental results to predict the mean and interval (PI) for the number (n) of confirmation runs at the conditions you dialed up, which provide the highest filtration rate with the least amount of formaldehyde. This is mission accomplished!

Note

To provide greater assurance of confirmation it makes sense to increase the sample size (n). The returns in terms of improving the prediction interval (PI) diminish as n increases. You can see this for yourself by trying different values of n in the Confirmation tab. Watch what happens to the PI—it approaches a limiting value. * Running six or so confirmation runs may be a reasonable compromise.

As shown below, you can enter your confirmation runs. The program then calculates the mean for the actual results. If this falls within the PI, you can press ahead to the next phase of your experimental program. Otherwise you must be wary.

*

Statistical detail: For an infinite number of confirmation runs the PI converges to the confidence interval (CI) which is a function of the number (N) of runs in the original experiment. See the CI in the Point Prediction node. There you also find the tolerance interval (TI) calculated to contain a proportion (P) of the sampled population with a confidence (1-α). By default, the interval contains 99% of the sampled population with 95% confidence. Note that the 95% confidence interval on a mean will have the narrowest spread, with the 95% prediction interval for a single observation (given in the Confirmation node) being wider and the 95% TI which contains 99% of the population being the widest. The most rigorous interval, TI, is often required for setting manufacturing specifications, but most experimenters will settle for the PI as a way to ‘manage expectations.’ Thus the Confirmation tool will be your best friend for moving forward.

You’ve now viewed all the important outputs for analysis of factorials. We suggest you do a File, Save at this point to preserve your work. Remember that if you want to write some comments on the file for future identification, you can click the Notes folder node at the top left of the tree structure at the left of your screen, then type in the description. It will be there to see when you re-open the file in the future.