Westgard QC, A Word from JOW: Six Sigma Staffing Page 1 of 8 Six Sigma Staffing Strategies l l l l l Calculation of Process Sigma Metrics Process Sigma vs Quality Control QC "Rules of Thumb" Automation and Expected Process Sigmas Staffing Strategies A message from Dr. Westgard "Doing more with less" has become a way of life in healthcare laboratories. For the last decade, laboratories have dealt with increased demands and reduced resources by acquiring more advanced technology. Increased automation and computerization has allowed more and more testing with fewer and fewer analysts. Those analysts may also have less training and experience since staffing strategies have targeted lower cost personnel. The danger is that the advanced technology and automation also makes it easier to produce more bad results faster than ever before! That's why QC is still so important! The economic realities are further complicated by the growing shortage of well-trained clinical laboratory scientists. Laboratories today are faced with a shortage of skilled analysts! While in-service training has always been important for developing the capabilities of the laboratory staff, laboratories today often need to provide on-the-job training to assure personnel have even the most basic analytical skills. This staffing dilemma is expected to get worse before it gets better. One interesting application of Six Sigma Quality Management is to use the Sigma metrics calculated for testing processes to guide the deployment of laboratory personnel. l l High sigma values identify testing processes that can be monitored with simple QC procedures to detect problems, therefore the skills and judgments of analysts are not as critical for achieving high quality test results. Low sigma values identify processes whose performance cannot be adequately monitored by statistical QC, therefore highly skilled and experienced analysts are needed to assure the desired quality is achieved. Calculation of Process Sigma Metrics To determine the Sigma metrics for processes in your laboratory, the following information is needed: l l l The quality required for the test, e.g., the CLIA allowable total error criterion for acceptability in proficiency testing. The imprecision of the method, e.g., the standard deviation or coefficient of variation calculated from a replication experiment or from routine QC data. The inaccuracy of the method, e.g., the bias determined from a comparison of methods validation experiment or the average difference of proficiency testing specimens from a comparison group. Westgard QC, A Word from JOW: Six Sigma Staffing Page 2 of 8 For a cholesterol test, the CLIA allowable total error is 10%. l l l l A method having a CV of 2.0% and a bias of 2.0% would give a process Sigmametric of 4.0 [(10-2)/2]. A method with a 3% CV and a 1.0% bias would be a 3.0 sigma process [(10-1)/3]. A method with a 2% CV and no bias would be a 5.0 sigma process [10/2]. It would take a CV of 1.7% and no bias to achieve 6.0 sigma performance [10/6]. Note that there are calculators for sigma metrics available on this website. You can use them to confirm the values calculated above. Process Sigma Performance vs Quality Control Many of the materials on this website are concerned with a quality-planning process and related quality-planning tools. Statistical QC procedures can be selected objectively on the basis of the quality required for a test and the performance observed for the method. One of the quality-planning tools is the "power function graph" shown in the accompanying figure. This power function graph displays the rejection characteristics for different control rules and different numbers of control measurements (N). The X-axis shows the probability for rejection and the Y-axis shows the size of analytical error, in this case, systematic error. Note that all the power curves are "s-shaped" and provide only low detection of small errors. As the size of the error increases, the probability of detecting that error also increases. Each "power curve" on the graph corresponds to a specific set of rules and N, which are identified in the key on the right side, e.g., the lowest curve is identified by the bottom line in the key as a single rule having 3.5s control limits and 2 control measurements per run, whereas the highest curve is identified by the top line in the key as a multirule procedure with 6 control measurements per run. The sensitivity of a QC procedure is seen to depend on the specific control rules and the number of control measurements. The highest Westgard QC, A Word from JOW: Six Sigma Staffing Page 3 of 8 sensitivity is achieved with use of multiple control rules, or "multirule QC", and the highest number of control measurements. The vertical lines identify methods having different performance on the sigma scale, as shown at the top of the graph. These lines correspond (from left to right) to 3-sigma, 4sigma, 5-sigma, and 6-sigma processes. Notice that the intersections of these lines on the bottom scale identifies the sizes of the analytical error that must be detected to assure the desired quality is achieved, e.g., the red line (furthest left solid line) corresponds to a situation where the medically important systematic error is approximately 1.35 times the method standard deviation, whereas the orange line represents an error that is 2.35 times the method standard deviation, the yellow line represents 3.35 times the method standard deviation, and the green line (actually off-scale) represents 4.45 times the method standard deviation For a method having 6-sigma performance, as shown by the green dotted line furthest to the right, all the power curves are approaching their maximums, or a probability of 1.00. Any or all of these QC procedures would provide nearly 100% detection of medically important errors. For practical guidance in selecting QC procedure, we commonly set an objective of 90% error detection and 5% or less false rejection. The false rejections expected are given by the y-intercepts of the power curves. This type of graphic and related calculations of the process sigma and the critical sytematic error are available from the EZ Rules computer program, as shown in the accompanying screen display. This example corresponds to the earlier cholesterol application that gave a calculated value of 4-sigma for the process and a critical systematic error of 2.35s. Westgard QC, A Word from JOW: Six Sigma Staffing Page 4 of 8 QC "Rules of Thumb" Some general guidelines can illustrate the relationship between the sigma metrics for process performance and the control rules and numbers of control measurements needed. This relationship is inverse, i.e., as the reliability of statistical QC increases, the need for analytical skills and experience decrease, as illustrated in the figure below. As the observed process sigma gets larger, the reliability of statistical QC increases. As the observed process sigma gets smaller, the need for analytical skills and experience increases. QC becomes more difficult, requiring multirules instead of single rules and requiring more Westgard QC, A Word from JOW: Six Sigma Staffing Page 5 of 8 and more control measurements per run. Some more specific recommendations or "rules of thumb" follow: Performance 6-sigma 5-sigma 4-sigma 3-sigma QC Recommendation can be monitored with any of these QC procedures, but a 13.5s control rule and 2 or 3 control measurements per run will be the least costly. can be monitored with a 13s control rule or multirule procedures having 2 or 3 control measurements per run. should be monitored with a multirule procedure having at least 4 control measurement per run. should be monitored with a multirule procedures having at least 6 control measurements per run. For a 3-sigma process, even a multirule procedure with an N of 6 provides only borderline performance. The sensitivity of the QC procedure is very limited and can not by itself guarantee the detection of errors that would compromise the quality of the test results. Automation and Expected Process Sigmas A quick and approximate assessment of the expected performance of a process can be made on the basis of its level of automation and computerization. As shown in the figure, manual and 1st generation automation will tend to perform at the 3-sigma level or lower, 2nd and 3rd generation systems will be about 4-sigma, and 4th to 6th generation systems will show 5-sigma performance or greater. For example, the current high volume chemistry and hematology analyzers should perform at 5-sigma or better, which means they can be monitored very effectively by simple statistical QC procedures. Immunoassays systems tend to be in the middle generations of automation and require more QC measurements and more complicated multi-rule Westgard QC, A Word from JOW: Six Sigma Staffing Page 6 of 8 procedures. Highly specialized assays that are performed manually or are in the early stages of automation generally require more QC than anyone does, or can afford to do. Of course, there are exceptions to these "rules of thumb." New instrument systems can leapfrog earlier generations of measurement systems by taking advantage of advanced technology, automation, and computerization. One example is a new Point-of-Care instrument for cardiac markers that demonstrates 6 sigma performance for troponin, myoglobin, and CK-MB (see discussion on this website). This system is highly automated and computerized, including on-board controls and QC data analysis. While it is a new system being marketed in 2001, it performs like a later generation system because of the complete automation and computerization of all functions. Staffing Strategies The implications for staffing are obvious and are summarized in the accompanying figure. You can most readily rotate staff among processes that have sigma metrics of 5 and greater. You will be able to efficiently monitor quality by the use of statistical QC. You can design the QC procedure to detect medically important problems and alert operators to those problems. These operators require some basic QC training, but do not need a lot expertise or experience in analytical quality management. This strategy should be effective in highly automated core laboratories. Processes with 4 sigma performance require analysts with more skill and experience. They need to be well trained in statistical QC and should be comfortable in dealing with method performance evaluation and data analysis. Laboratories that perform specialized testing in chemistry, hematology, etc.,fall into this category. Processes having 3 sigma performance or less require highly skilled and experienced analysts. These analysts should be selected on the basis of an extensive experience with the Westgard QC, A Word from JOW: Six Sigma Staffing Page 7 of 8 analytical methods and instrument systems that are being utilized. They should also have an in-depth background in analytical quality management, including statistical QC, method validation, and quality-planning. Laboratories dealing with emerging technology, such as molecular diagnostics, fall into this category. James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management. Past Essays l l l l l l l l l l l l l l l l l Myths of Quality Putting Quality into Quality Control Assuring Quality through Total Quality Management Trends in quality management: Utilization and Outcomes Quality Goals, Requirements, & Specifications Future Directions in Quality Control The Myth of Medical Decision Limits Quality by Design Tools and Technology for QC Training Education and Training for Analytical Quality Management, Part I Mapping the Road to Analytical Quality with OPSpecs Charts Quality and Commerce QC - Back to Basics Education and Training for Analytical Quality Management, Part II: Developing Web-courses Method Validation - The Inner, Hidden, Deeper, Secret Meaning Education and Training in Analytical Quality Management, Part III: Basic QC Training Electronic QC and the Total Testing Process l l l l l l l l l l l l l l l l l l l l l From Rules and Tools to Technology and Training (Beijing) Quality Requirements: the debate heats up! Z-Stats: A treat and a treatment The Need for a System of Quality Standards What's wrong with traditional QC? To be Uncertain or In Error? That is the Question QC 2000 Education and Training for Analytical Quality Management, Part IV: Interactive Training Tools Do's and Dont's of QC The Abbott Consent Decree: A Wake-Up Call WQC Y2K Sage Advice about new approaches to Quality Control EZ Rules for Assuring Quality Who will care to quality tomorrow? Quality is Job 1 when the rubber meets the road Errors in reasoning about Laboratory errors Six Sigma Quality Management & Lab Precision Six Sigma Quality Managment & Requisite Lab QC 2001: Year of the Odyssey essays CLIA Postponed again and again and again Repeated, Repeated, Got Lucky Copyright © 2001. All rights reserved. Westgard QC, 7614 Gray Fox Trail, Madison WI 53717 Call 608-833-47183 or e-mail us at [email protected] A Message from JOW QC Lessons | QC Applications | Questions | Multirule CLIA Requirements | What's New?| Catalog | Demo Download Westgard QC, A Word from JOW: Six Sigma Staffing Home | Glossary | ARCHIVES | Links | Feedback Page 8 of 8
