Design-Expert v23.1

Design-Expert » Advanced Topics » Mean Bias Correction

Mean Bias Correction

../../../_images/mean-bias-correction.PNG

Mean and median may differ on original scale

When a predicted mean response on a transformed scale is converted back to its original scale, the predicted mean becomes the predicted median. A correction is applied to the back-transformed prediction so it correctly predicts the mean, rather than the median [Mil84]. This correction may be toggled on/off by clicking Edit > Preferences > Math Analysis > Correct biased predictions of mean.

Use the predicted mean for making inferences and prediction about your process.

Mathematical Detail

Transformed Scale

Consider the case where we transform \(\small{n}\) response values, \(\small{y_i,~i=1,...,n}\), using transformation \(\small{f}\) to ensure normality of the residuals,

\[\small{ z_i = f(y_i) }\]

We then fit the resulting transformed values, \(\small{z_i}\), to a model, resulting in transformed scale, least squares estimates for the constant variance, \(\small{\sigma_z^2}\), response mean function, \(\small{z(A,B,...)}\), and response median function, \(\small{m_z(A,B,...)}\), as a function of the factors, \(\small{A, B, C,...}\) , in the model. These estimates are the regression mean square error (MSE), \(\small{\hat{\sigma}_z^2}\), and the mean/median function, \(\small{\hat{z} = \hat{m}_z}\),

\[\begin{split}\small{ \begin{eqnarray} \hat{\sigma}_{z}^2 &=& MSE = \text{constant},\\ \hat{z} &=& \hat{m}_z = \hat{z}(A,B,...) = \hat{\beta}_0+\hat{\beta}_A A + \hat{\beta}_B B + \hat{\beta}_{A B} A B+...\\ \end{eqnarray} }\end{split}\]

Note

This information can be found on the ANOVA tab of a response analysis, especially the Fit Summary table for the standard deviation, \(\small{\hat{\sigma}_z = \sqrt{MSE}}\), and the Actual Equation table for the estimated transformed scale mean/median equation, \(\small{\hat{z}(A,B,...)}\). (The Coded Equation could be used instead, provided the inputs \(\small{A,B,...}\) ,are in coded units.)

The normal distribution is a symmetric distribution, so the response mean function, \(\small{\hat{z}}\), is also the median function, \(\small{\hat{m}_z}\) (see the left side of the figure at the top of the page).

Predictions for the mean and median are the same on the transformed scale.

Original Scale

Because the transformed response values, \(\small{z_i}\), are used for the fit, the predictions from the estimated mean function, \(\small{\hat{z}}\), are in different units and of different size than the original data, \(\small{y_i}\). This is inconvenient and not very easy to understand, so we would like estimates on the original scale of the response data.

To obtain the median on the original scale, apply the inverse transformation \(\small{f^{-1}}\) to the mean function on the transformed scale,

\[\small{ \hat{m}_y(A,B,...) =~\text{original scale median}~= f^{-1}(\hat{z}) }\]

To obtain an estimate of the mean on the original scale, we may use an approximation involving the second derivative (curvature) of the inverse transformation and the mean squared error from the regression. To first order in the mean square error, \(\small{\hat{\sigma}_{z}^2}\) [Oeh92],

\[\small{ \hat{y}(A,B,...) =~\text{original scale mean}~\approx f^{-1}(\hat{z}) + \frac{1}{2} \left(\left. \frac{d^2 f^{-1}}{d z^2}\right\rvert_{\hat{z}}\right) \hat{\sigma}^2_z }\]

The original scale distribution is not a symmetric distribution, so the response mean function is different from the median function (see the right side of the figure at the top of the page).

On the original scale, predictions for the mean and median are not the same.

Note

It is important to verify that the model assumptions – normality of residuals, constant variance, and independence – have been achieved after transformation, and perhaps even more so when relying on original scale mean bias correction.

Example #1

Consider the inverse transformation with constant shift parameter \(k\) (used to make the response data positive, typically),

\[\small{ z = f(y) = \frac{1}{y+k},~y + k > 0 }\]

In that case, the inverse transformation and its derivatives are,

\[\begin{split}\small{ \begin{eqnarray} f^{-1}(z) &=& +\frac{1}{z}-k,~z > 0, \\ \frac{d f^{\small{-1}}}{d z} &=& -\frac{1}{z^2},\\ \frac{d^2f^{\small{-1}}}{d z^2} &=& +\frac{2}{z^3} \end{eqnarray} }\end{split}\]

Therefore, on the original scale,

\[\begin{split}\small{ \begin{eqnarray} \hat{m}_y &=& f^{-1}(z) = \frac{1}{\hat{z}}-k,\\ \hat{y} &\approx& \frac{1}{\hat{z}}-k+ \frac{1}{2} \left(\frac{2}{\hat{z}^3}\right)\hat{\sigma}^2_z = \frac{1}{\hat{z}}\left(1+ \frac{1}{\hat{z}^2} \hat{\sigma}^2_z\right)-k \end{eqnarray} }\end{split}\]

A similar example is given in [Mil84].

Note

Remember that \(\small{\hat{z} = \hat{z}(A,B,...)}\), \(\small{\hat{m}_y =\hat{m}_y(A,B,...)}\), and \(\small{\hat{y} = \hat{y}(A,B,...)}\) are functions of the factor settings, \(\small{A, B, C...}\), and change when the factor settings change.

Special Case: Log Transforms for Log Normal Data

Logarithmic transformations for log normal data are a special case where the mean function on the original scale can be computed exactly [Fin41]. For these transformations,

\[\begin{split}\small{ \begin{eqnarray} \hat{m}_y &=& \exp{\left(a\hat{z}\right)}-k,\\ \hat{y} &=& \exp{\left[a \left(\hat{z}+ a \tfrac{1}{2} \hat{\sigma}^2_z \right)\right]} - k\\ \end{eqnarray} }\end{split}\]

Here, \(a\) controls the base of the logarithm and \(\small{k}\) is a constant parameter added to all the response values. For the natural log transformation, \(\small{a = \log_e(e) = \ln{(e)} = 1}\). For the log base-10 transformation, \(\small{a = \ln{(10)}}\). This gives two sets of equations, depending on the type of logarithm transformation used,

For natural logarithm transformations the original scale median and mean are,

\[\begin{split}\small{ \begin{eqnarray} \hat{m}_y &=& \exp{\left(\hat{z}\right)} - k = {e}^{{\hat{z}}} - k,\\ \hat{y} &=& \exp{\left(\hat{z}+{\tfrac{1}{2} \hat{\sigma}^2_z}\right)} - k = {e}^{{\hat{z}+\tfrac{1}{2} \hat{\sigma}^2_z}}-k\\ \end{eqnarray} }\end{split}\]

For base-10 logarithm transformations the original scale mean and median are,

\[\begin{split}\small{ \begin{eqnarray} \hat{m}_y &=& \exp{\left[\ln{(10)}\hat{z}\right]} - k = {{10}}^{{\hat{z}}}-k,\\ \hat{y} &=& \exp{\left[\ln{(10)} \left(\hat{z}+ \ln{(10)} \tfrac{1}{2} \hat{\sigma}^2_z \right)\right]} = {{10}}^{{\hat{z}+\tfrac{1}{2} \ln{(10)} \hat{\sigma}^2_z}}-k\\ \end{eqnarray} }\end{split}\]

Example #2

Suppose that the transformation is the base-10 logarithm, \(\small{z = f(y) = log_{10}(y)}\), and the transform shift parameter is zero (\(\small{k=0}\)), as in the image displayed at the top of this help entry. We wish to calculate the mean on the original scale at the point \(\small{(B{=}1,C{=}1,D{=}1)}\) in the design (the factor \(\small{A}\) was not in the model for this example). The least square estimates for the mean square error and mean function on the transformed scale are,

\[\begin{split}\small{ \begin{eqnarray} \hat{\sigma}_z^2 &=& MSE =(0.45)^2 = 0.002025,\\ \hat{z}(B,C,D) &=& 0.683549 + 0.126689 B+0.239201 C+ 0.072841 D \\ \end{eqnarray} }\end{split}\]

The estimated mean (and median) on the transformed scale at the desired point are,

\[\begin{split}\small{ \begin{align} \hat{z}(B{=}1,C{=}1,D{=}1)~=~& 0.683549~+\\ & 0.126689(1)~+\\ & 0.239201(1)~+\\ & 0.072841(1) = 1.122279 \end{align} }\end{split}\]

The original transformation is the base-10 logarithm so we use,

\[\begin{split}\small{ \begin{eqnarray} \hat{m}_y(B{=}1,C{=}1,D{=}1) &=& {10}^{\hat{z}} = {10}^{1.122279} \approx 13.25,\\ \hat{y}(B{=}1,C{=}1,D{=}1) &=& {10}^{\hat{z} + \tfrac{1}{2} \ln{(10)}\hat{\sigma}^2_z} = {10}^{1.122279 + 0.5 (2.302585) (0.002025)} \approx 13.32 \end{eqnarray} }\end{split}\]

The estimated mean bias correction to the median is approximately \(\small{13.32-13.25 \approx 0.07}\) at the given factor settings.

References

Fin41

D. J. Finney. On the distribution of a variate whose logarithm is normally distributed. Journal of the Royal Statistical Society Supplement, 7:155–161, 1941.

Mil84(1,2)

D. Miller. Reducing transformation bias in curve fitting. The American Statistician, 38(2):124–126, 1984.

Oeh92

G.W. Oehlert. A note on the delta method. American Statistician, 46:27–29, 1992.

Rot88

P. Rothery. A cautionary note on data transformation: bias in back-transformed means. Bird Study, 35(3):219–222, 1988.

Previous Next