Virtual-DOE, Data Mining and Artificial Neural Networks

By

Thomas Pyzdek

March 1999

As beneficial and productive as design of experiments can be, the process of conducting them has its drawbacks. The workplace, be it a factory, a retail establishment or an office, is designed around a routine. The routine is the "real work" that must be done to generate the sales which, in turn, produce the revenues that keep the enterprise in existence. Experimenting, by its very nature, means disrupting the routine. Important things are changed to determine what effect they have on various important metrics. Often, these effects are unpleasant; perhaps why they were not changed in the first place! The routine was often established to steer a comfortable course that avoids the disruption and waste that results from making changes.

The problem is, unless we change things we can never improve. Six Sigma generates as much improvement by changing things as it does by reducing variability. It is part of the Six Sigma paradox mentioned in the first chapter.

In this article we present a way of conducting "virtual" experiments using existing data and artificial neural network (neural net) software. Neural nets are popular because they have a proven track record in many data mining and decision-support applications. Neural nets are a class of very powerful, general purpose tools readily applied to prediction, classification, and clustering. They have been applied across a broad range of industries from predicting financial series to diagnosing medical conditions, from identifying clusters of valuable customers to identifying fraudulent credit card transactions, from recognizing numbers written on checks to predicting failure rates of engines. In this section we explore only the application of neural nets to design of experiments for Six Sigma, but this merely scratches the surface of the potential applications of neural nets for quality and performance improvement.

Neural networks use a digital computer to model the neural connections in human brains. When used in well-defined domains, their ability to generalize and learn from data mimics our ability to learn from experience. However, there is a drawback. Unlike a well-planned and executed DOE, a neural network does not provide an explicit mathematical model of the process. It is possible, however, to include various transformed variables to "help" the neural net if one has a model in mind. For example, in addition to feeding the neural net X1 and X2 raw, one could include transformations, higher-order polynomial and interaction terms, etc. as inputs to the neural network. Still, for the most part, neural networks must be approached as black boxes with mysterious internal workings, much like the mystery of the human mind it is designed to imitate.

BUILDING A NEURAL NET FOR VIRTUAL-DOE

All companies record important data, some in well-designed data warehouses, some in file drawers. This data represents potential value to the Six Sigma team. It contains information that can be used to evaluate process performance. If the data include information on process settings, for example, they may be matched up to identify possible cause-and-effect relationships and point the direction for improvement. The activity of sifting through a database for useful information is known as data mining. The process works as follows:

1.   Create a detailed inventory of data available throughout the organization. The IS department may already have compiled this information.

2.   Determine the variables which apply to the process being improved and for which historical data exist.

3.   Using a subset of the data which include the most extreme values, train the neural net to recognize relationships between patterns in the independent variables and patterns in the dependent variables.

4.   Validate and test the predictive capacity of the neural net with the remaining data.

5.   Perform experimental designs as you would on a real process. However, instead of making changes to the actual process, make changes to the "virtual process" as modeled by the neural net.

6.   Once the sequential application of designed experiments has been completed using the neural net model, use the settings from the neural net as a starting point for conducting experiments on the actual process. If the results of experimenting confirm the results from the neural net you can reduce sample sizes and move quickly along the path discovered by the virtual-DOE process.

It can be seen that the entire virtual experimentation process helps answer the question "Where are we?" It is important to recognize that neural net experiments are not the same as live experiments, they are only simulations. However, the cost of doing them is minimal compared with live experiments and the process of identifying input and output variables, deciding at which levels to test these variable, etc. will bear fruit when the team moves on to the real thing. Also, virtual experiments allow a great deal more "what if?" analysis, which may stimulate creative thinking from team members.

Example of Building a Neural Net Model

The data in Table 1 are from the solder process described above. Data were not gathered for a designed experiment, but were merely collected during the operation of the process. Variable B represents the circuit board preheat time in seconds. Variable D represents the distance from the preheat element in centimeters. The data were used to train and validate a neural net.

Table 1: Raw Data Used to Build a Neural Net Model

A simple neural net was built to analyze the above data, as shown in Figure 1.

Figure 1: A Neural Net Model

The neural net was trained using the above data producing the model shown in Figure 2.

Figure 2: Neural Net Model for Solder Process Data

It is interesting to compare the neural net model with the response surface model (RSM) produced by classical DOE methods. The RSM is shown in Figure 3.

Figure 3: Response Surface from Designed Experiment

In the RSM the zero point for factor B was 45 seconds, zero for factor D was 22.5 cm. You can see that the surface described by the neural net is somewhat different than the one modeled earlier using DOE [using response surface methods]. However, both models direct the B and D settings to similar levels and both make similar predictions for the defect rate.

Neural net software allows "what if" analysis, as shown in Figure 4. By using What If? you can conduct virtual DOE by designing proper experiments just as you would design real world experiments. The inputs are fed into the neural net, and the outputs from the neural net are analyzed just as you would analyze the results of real experiments.

Figure 4: What If? Analysis Using the Neural Net Model

This article is based on an excerpt from The Complete Guide to Six Sigma, QA Publishing, LLC, LLC, Scheduled for publication in the Summer, 1999.

Notes:

Berry, Michael J.A. and Linoff, Gordon, Data Mining Techniques: for marketing, sales, and customer support, New York: John Wiley & Sons, 1997.

Learn more about the DOE tools for designed experiments in Six Sigma Demystified (2011, McGraw-Hill) by Paul Keller, in his online Intro. to DOE short course (only $99) or online Advanced Topics in DOE short course (only $139), or his online Black Belt certification training course ($875).