Stat-Ease Blog

Next in our 40th anniversary “Ask an Expert” blog series is Leonard “Len” Rubinstein, a Distinguished Scientist at Merck. He has over 3 decades of experience in the pharmaceutical industry, with a background in immunology. Len has spent the last couple of decades working on bioanalytical development, supporting bioprocess and clinical assay endpoints. He’s also a decades-long proponent of design of experiments (DOE), so we reached out to learn what he has to say!

When did you first learn about DOE? What convinced you to try it?

I first learned about DOE in 1996. I enrolled in a six-day training course to better understand the benefits of this approach in my assay development.

What convinced you to stick with DOE, rather than going back to one-factor-at-a-time (OFAT) designs?

Once I started using the DOE approach, I was able to shorten development time but, more importantly, gained insights into understanding interactions and modeling the results to predict optimal parameters that provided the most robust and least variable bioanalytical methods. Afterward, I could never go back to OFAT!

How do you currently use & promote DOE at your company?

DOE has been used in many areas across the company for years, but it has not been explicitly used for the analytical methods supporting clinical studies. I raised awareness through presentations and some brief training sessions. Afterward, after my management adopted it, I started sponsoring the training. Since 2018, I have sponsored four in-person training sessions, each with 20 participants.

Some examples of where we used DOE can be found at the end of this interview.

What’s been your approach for spreading the word about how beneficial DOE is?

Convincing others to use DOE is about allowing them to experience the benefits and see how it’s more productive than using an OFAT approach. They get a better understanding of the boundaries of the levels of their factors to have little effect on the result and, more importantly, sometimes discard what they thought was an important factor(s) in favor of those that truly influenced their desired outcome.

Is there anything else you’d like to share to further the cause of DOE?

It would be beneficial if our scientists were exposed to DOE approaches in secondary education, be it a BA/BS, MA/MS, or PhD program. Having an introduction better prepares those who go on to develop the foundation and a desire to continue using the DOE approach and honing their skills with this type of statistical design in their method development.

And there you have it! We appreciate Len’s perspective and hope you’re able to follow in his footsteps for experimental success. If you’re a secondary education teacher and want to take Len’s advice about introducing DOE to your students, send us a note: we have “course-in-a-box” options for qualified instructors, and we offer discounts to all academics who want to use Stat-Ease software or learn DOE from us.

Len’s published research:

Whiteman, M.C., Bogardus, L., Giacone, D.G., Rubinstein, L.J., Antonello, J.M., Sun, D., Daijogo, S. and K.B. Gurney. 2018. Virus reduction neutralization test: A single-cell imaging high-throughput virus neutralization assay for Dengue. American Journal of Tropical Medicine and Hygiene. 99(6):1430-1439.

Sun, D., Hsu, A., Bogardus, L., Rubinstein, L.J., Antonello, J.M., Gurney, K.B., Whiteman, M.C. and S. Dellatore. 2021. Development and qualification of a fast, high-throughput and robust imaging-based neutralization assay for respiratory syncytial virus. Journal of Immunological Methods. 494:113054

Marchese, R.D., Puchalski, D., Miller, P., Antonello, J., Hammond, O., Green, T., Rubinstein, L.J., Caulfield, M.J. and D. Sikkema. 2009. Optimization and validation of a multiplex, electrochemiluminescence-based detection assay for the quantitation of immunoglobulin G serotype-specific anti-pneumococcal antibodies in human serum. Clinical and Vaccine Immunology. 16(3):387-396.

Design of experiments (DOE), being such an effective combination of multifactor testing with statistical tools, hits the spot for engineers and scientists doing industrial R&D. However, as documented in my white paper on Achieving Breakthroughs in Non-Manufacturing Processes via Design of Experiments (DOE), this statistical methodology works equally well for business processes. Yet, non-manufacturing experimenters rarely make it beyond simple one-factor-at-a-time (OFAT) comparisons known as A/B splits—most recently embraced, to my great disappointment, by Harvard Business Review*. But to give HBR some credit, this 2017 feature on experimentation at least mentions “multivariate” (I prefer “multifactor”) testing as a better alternative.

To see an illuminating example of multifactor testing applied to marketing, see my April 21 StatsMadeEasy blog: Business community discovers that “Experimentation Works”.

Another great case for applying multifactor DOE came from Kontsevaia and Berger in a study published by the International Journal of Business, Economics and Managemental**. To maximize impressions per social-media posts, they applied a fractional two-level design on 6 factors in 16 runs varying:

A. Type of Day/Day of the week: Weekend (Sat, Sun) vs Workday (Thu, Fri)

B. Social Media Channel: LinkedIn vs Twitter

C. Image present: No vs Yes

D. Time of Day: Afternoon (3-6pm) vs Morning (7-10am)

E. Length of Message: Long (at least 70 characters) vs Short (under 70 characters)

F. Hashtag present: No vs Yes

The multifactor marketing test revealed the choice of channel for maximum impressions to be highly dependent on posts going out on weekends versus workdays. This valuable insight on a two-factor interaction (AB) would never have been revealed by a simple OFAT split.

Design-Expert® software makes multifactor business experiments like this very easy for non-statisticians to design, analyze and optimize for greatly increased returns. Aided by Stat-Ease you can put DOE to work for your enterprise and make a big hit career-wise.

*“Building a Culture of Experimentation”, Stefan Thomke, March-April, 2020.

**“Analyzing Factors Affecting the Success of Social Media Posts for B2B Networks: A Fractional-Factorial Design Approach”, August, 2020.