Manufacturing Operations Management, Quality and Aerospace Talk

The Manufacturing Continuous Improvement Lifecycle

 

 

 

 

 

 

 

 

 

 

 

 

 

 

This one is common to Six Sigma, Lean Manufacturing, and any process improvement philosophy. These days... if you are not continuously improving, you are probably moving backwards and it might be time for a change.

: )

 

 

The Problem Isn't Motivation or ROI

It is a new year again and we have some resolutions to put in place. For me, one resolution is to make healthier choices—eat better, exercise more, get leaner and lose a few extra pounds. But I have trouble getting “motivated” and starting the new routine of exercising in the mornings. I keep telling myself “I’ll start next week.” What can I do to improve my motivation?

Perhaps I am not asking the right question. Is “motivation” really holding me back? I am motivated enough to put these things on my resolution list. I want to be healthier and feel better. Why would I not be motivated to do that? The Return on Investment (ROI) is clearly there. This will add more healthy years to my life. Perhaps motivation is not my problem. What if I rephrase my problem as lack of follow-through?

Perhaps the solution is not putting up motivational posters on my walls trying to visualize a healthier and more competitive me. I just get more frustrated as I see some of my friends getting better and faster at the basketball court every day. (I am the one getting slower.)

The solution to a motivational problem is the exact opposite of the solution to a follow-through problem. The mind is essential to motivation. But for follow-through, the mind can get in the way overly thinking things and sabotaging our aspirations. For example, we decide to go to the gym after work but then, when it comes time to go, we think, “It's late, I'm tired, I could get hurt, maybe I'll skip it today.”

If I want to follow through on something, I need to stop over-analyzing. Shut down the conversation that wants to start again in my head. Stop worrying and finding reasons to put it off. Just put in place a plan and follow it through. Like the ad says: “Just Do It!”

This year I am going to make a very specific decision to implement some changes in my routine and put a specific timeline on it. Something like: “Starting Monday Jan 9th, I will work out each Monday, Wednesday and Friday at 6 AM for 30 minutes”.

I am also going to put in place a reward/penalty system to counteract the natural resistance to change. If I complete my scheduled workout each week, I can eat whatever I want on Saturday; otherwise I eat only salad on Saturday.

Perhaps you can apply these same ideas to some of your personal or business improvement goals for 2012:
• Stop putting off what you know you need to do—projects with clear and known ROI; projects needed for your personal or professional competitive edge.
• Create an environment that supports change and your new goals
• Commit to a concrete plan and timeline
• Maintain a sense of urgency in this new commitment
• Put in place reward and/or penalty measures to counteract the natural resistance to change.

When your mind (or your team) starts to argue with you —remember that is the natural resistance-to-change getting in the way. Just follow the plan. This year, we are going to stop putting off the projects we objectively know will improve our personal and professional success.

WIsh me luck and good luck with your initiatives. Happy New Year!

Fishbone Diagram Templates for Root Cause Analysis with Common Manufacturing Problems

Why start a root-cause analysis with a blank canvas? Are we better served starting with a blank canvas and an open mind, or do we put some common manufacturing problem areas in our template? Will the defaults in our template bias the discussion or make sure we always touch base on certain topics?

I was creating a template to facilitate meetings and ensure certain topics were always covered during a root-cause analysis. However, I found myself quickly filling up the paper.  I ended up switching to a tabloid page size, but I thought I should capture several iterations of the template to compare and decide which one would be more conducive to getting a good discussion started. 

Which one gets your brainstorming juices flowing? Which one will be more effective?

Do we start with a blank slate? 

Staring at a blank template or whiteboard can be intimidating at first. It is now up to the participants to come up with some ideas. However, strong personalities can lead the discussion in a single direction and we risk skipping some potential areas for causes. 

I personally like a little more guidance in a template. At least have some common headings for areas of discussion. This way we ensure that certain topics are discussed in every root-cause analysis. 

On the other hand, starting with a full fishbone diagram does not seem good either. Perhaps we start with the last one we used like the one above. The risk here is that we might get a feeling that we are done already. Looks finished to me. So perhaps we need a partial template like the one below. 

Maybe even this one is too full. But we might want to make sure certain common problems are always included in the discussion. 

Consider adding the following common sources of problems in manufacturing to your templates. 

1. Materials – Wrong, Deficient, Out of Spec – Is the material correct? Does it meet quality standards and specifications?

2. People – Skills and Certifications for Technicians and Inspectors - Are the employees performing the job qualified to do so? Is the training received sufficient?

3. Methods/Instructions – Incomplete, inaccurate or ambiguous – Are standardized work instructions provided? Are instructions illustrated? Are the work instructions missing a critical sentence or warning? 

4. Environment – Controlled, temperature, humidity, vibration, etc. – Is the assembly environment adequate to perform the work? Do the environment conditions need to be monitored for the particular process?

5. Machines/Equipment/Gages – Set up, maintenance, cycles – How do you know that a machine was properly configured to perform the work? When was the last machine maintenance?

6. Configuration – Wrong, obsolete, or changed configuration –  How do you know the correct configuration was selected when the assembly was worked?

7. Measuring/Test Equipment – Insufficient, inaccurate or incorrect measurements – How would you determine that the measuring/test equipment produced the desired result? When was the tool last calibration? 

8. Design – Rework, work-arounds, ECO/ECN – Was the problem or failure due to design? Due to building the wrong configuration?  Different revision levels? 

Which template do you prefer as a starting point? Empty or half full?

Can an Aerospace Guy Help an Automotive Company?

 Indeed Alan Mulally has done it at Ford. I had the pleasure of hearing him speak at a conference this week and I am impressed with his leadership.  Under Mulally, Ford has not only turned profitable this year and survived the recession without government bailout or bankruptcy, but more importantly the company is resurrecting the Ford brand and image.

There were many skeptical in the industry when Bill Ford in 2006 recruited Alan Mulally, a top executive at Boeing, to lead Ford. But it sure looks like a good decision now. 

Ford had actually stopped making cars in the U.S. and was only making trucks here. “We went to work reducing overhead including a new agreement with the Union.” “We are now converting some truck plants to build car components again”, said Mulally.  "We are fighting for the soul of manufacturing in the US."

For example, the new Ford Fiesta shares about 65% of its parts with Fiesta models sold in Europe and China. The upcoming new-generation Focus that goes on sale here next year will share as much as 85% of its content with similar versions of the compact car sold in other regions of the world. Such a strategy enables Ford to spread design and development expenses across a greater number of vehicles. As a stand-alone American-designed and -built vehicle, the Focus would have sold about 150,000 units annually. But as a global vehicle, its development expenses are spread across sales expected to approach 2 million. 

Mulally has also been focused on quality. In the last few years, Ford vehicles have been getting better quality ratings. In a recent auto quality study, Ford moved from 14th place in 2006 to sixth place in 2010 with a quality rating above Toyota. 

Coming from the Aerospace industry has made Alan very quality oriented. As he pointed out, safety and continual improvement are very important in the Aerospace industry where products are complex systems with lots of subassemblies and parts. Alan has also been a student and fan of Toyota’s TPS, Juran, and Deming.  Ford was Lean before Lean and Alan wants to honor that tradition. 

Cars are getting more complex everyday with a lot of electronic controls, and perhaps there is a lot more to learn from the Aerospace industry. 

I really liked Alan’s thoughts on teamwork, visibility and management meetings. "Definitely... this is a team sport! In our weekly meetings, every functional area is represented including Quality. Communication is important. We need to get the whole team on-board. We need to have a plan and everyone must know and understand the plan.”

Alan celebrates finding improvement opportunities. It is an important part of the culture of process improvement. “How can all the lights in the dashboards be all green when we just announced big losses?” He applauded the manager that brought that first issue up in a meeting. "Thanks for that visibility. How can we help you solve that problem?" Next week, the charts were a rainbow of colors,” he smiled.

The pressure is immense in these meetings because there is so much visibility. It's all out there and everyone knows who is doing what. But the pressure is positive... a pressure to correct problems and improve. You have to follow up and take action. 

One-Ford Pocket Card with Expected Behaviors

 “We have to have good communication in meetings and mutual respect. We don't allow side conversations in meetings. If someone is talking, we stop the meeting until they stop or ask them to leave. We don't allow humor at anyone's expense either. It is not really funny. Our leaders are not only responsible for process improvement in their areas; they are also responsible for skills development.”

Alan is definitely an inspiring leader and I am not surprised this “aerospace guy” was named 2010 Man of the Year by Automobile Magazine. We can follow Ford’s example and keep our eyes open for talent and good ideas from other industries. 

References:

“Ford's student driver takes the wheel”,  Alex Taylor III, Fortune Magazine, 11 2 2006

http://money.cnn.com/magazines/fortune/fortune_archive/2006/11/13/8393169/index.htm?postversion=2006110209

“Fixing up Ford”,  Alex Taylor III, Fortune Magazine, 5-12-2009

http://money.cnn.com/2009/05/11/news/companies/mulally_ford.fortune/

Comparing Lean Manufacturing and Six-Sigma Continuous Improvement Efforts

Lean Manufacturing and Six Sigma practitioners use slightly different terminology and methodologies for process improvement, but they are really more similar than different. Let’s compare the different terminology and methods.

The term kaizen is used in two ways: (1) kaizen is used to represent the general continuous improvement process which would be similar to the general DMAIC process in Six Sigma practices, and (2) in relation to kaizen workshops which are intense rapid problem solving activities to tackle problems identified as high priority.

Six Sigma practitioners use the DMAIC process to systematically reduce variability in a manufacturing process. Lean practitioners refer to the Toyota problem solving methodology and to kaizen events. For diving down into specific urgent issues once they are uncovered, the Toyota Problem Solving process and the 8-Discipline (8D) process are very similar. The 8D process seems more appropriate to quality issues because it directly adds a step to contain the problem. Both are explained below.

DMAIC

Six Sigma practitioners use the DMAIC process to systematically reduce variability in a manufacturing process.

Define – This phase focuses on defining the project goalsand activities and identifying the issues that need to be addressed.

Measure – In this phase information about the target process is gathered. Metrics are used to determine current performance and to identify problems and issues.

Analyze – This phase involves identifying the root causes of the defects using appropriate statistical tools.

Improve – In this phase solutions to address the problems identified are applied. Lean methods such as 5S, Standardized work content, cellular manufacturing layouts, mistake-proofing, Total Productive Maintenance are applied.

Control – This phase involves institutionalizing the improvement methods. Policies, procedures and other management systems are modified to achieve the goals.

Performance results are periodically monitored to ensure that the improvements are sustained.

However, TPS practitioners have adopted an extended problem-solving methodology that puts more emphasis on identifying the root cause.


 

Toyota Problem-Solving Process

Initial Problem Perception – The initial problem reported by employees is documented. Problem may seem vague and complicated at the beginning.

Clarify Problem – Getting a better grasp of the situation requires an observation of the problem at the site with an open mind. Pareto diagrams use bar graphs to sort problems according to their severity, frequency, nature and source and point out the most important problems in the area.

Initial Point of Cause (POC) – Identify targets for improvements and a likely cause to use as a starting point for further cause exploration.

Investigate Root cause – Answer the “Why?” question five times to narrow down to the real root cause.

Countermeasure – Define a countermeasure to eliminate the root cause identified. 

Evaluate – It is critical to allow time or a few units to pass through the new process before we evaluate the effectiveness of the corrective action.

Standardize – Only after verification of the effectiveness of the corrective action, should we proceed to standardize the solution by modifying work instructions and standard manufacturing procedures.


Kaizen Workshops

When analyzing the macro value stream of a company, continuous improvement efforts can seem too big to tackle. It is often easier to divide the macro value stream into several smaller subprocesses. If critical problems are identified in smaller subprocesses they can be tackled with kaizen workshops.

A kaizen workshop is an intensive rapid improvement model that emphasizes creativity and radical improvement. The methodologies of value stream mapping and the Toyota problem-solving process are implemented in a compressed time frame. The workshops are typically full time for 3 to 5 days and participants analyze the current process, develop a new lean process and perhaps even begin implementation.

The team size should be limited to no more than 15. The kaizen workshop is headed by the manager responsible for the process (“process owner”) and should include people form the work area, and customers and suppliers to the process.

Preparing for a kaizen workshop involves the following:
• Identify process scope, customer, suppliers, and symptoms to tackle
• Create value stream map for current process
• Collect relevant metrics, facts and documents describing the current process
• Post the current state map in the workshop room

The kaizen workshop includes the following steps:
• Analyze and assess non-value added activities in current practices
• Brainstorm different improvements
• Develop recommendations for new process
• Create value stream map for new process
• Use a “parking-lot” list for potential process improvements with lower priorities
• Obtain management approval on recommendations
• Plan and initiate process improvements

After the kaizen workshop it is important to:
• Follow up to make sure the new process is implemented
• Measure the effect of the new process on process metrics
• Develop visual controls and standardize the new process
• Prepare presentation to management on process improvements and recognize the team achievements
• Review items left in the “parking-lot” for future improvements
• Select the next area to tackle for process improvement


8D Problem Solving Process

When responding to quality issues in the plant or supply chain, the 8-Disciplines (8D) process is often used because it adds steps to immediately contain the problem. The Toyota Problem-Solving process and the 8D process have a lot of similarities but in many ways do not exactly align. The 8D process has an emphasis on corrective and preventive actions which are more appropriate to obvious quality issues, and the Toyota Problem-Solving process has more emphasis on defining the problem which might not be obvious on productivity issues.


 
Form the Team – The 8D process recognizes the importance of assembling the right team and places it as the first step. A small team is needed with the right mix of skills, experience and authority to resolve the problem and implement solutions. Ensure that everyone on the team has the time and inclination to work on the problem together.

Formal meetings should initiate the effort and monitor progress. The company’s Corrective Action Database should be used to document findings, decisions and progress.

Describe the Problem – It is critical to define a clear problem. The problem statement provides the starting point for investigation. Terms used in the description must be understood by the entire team. Problem description should include the following:
(a) Who – Who is affected by the problem? Who first observed the problem? To whom was the problem reported to?
(b) What – What type of problem? What has the problem?  What is or isn’t happening? What is the physical evidence?
(c) When – When did it first happen? When was it first observed? Is it a recurring problem? How often? What is the trend? (random, cyclical, continuous)  Has it been observed before?
(d) How Much – What is the magnitude of the problem? How many units or components are affected? How much cost (labor and material) is associated to this problem?
(e) Why – Why is it a problem? What is the impact? What processes are affected? Is production stopped completely or partially? Is there a work-around? Are customer delivery dates impacted?

Contain the Problem – Identify interim containment actions to prevent further cost impact from the problem. Is the problem tied to a machine? Is it tied to a supplier component? Are more lots affected by this problem? Suppliers must be informed as soon as possible about issues to prevent further shipments with problems. Inventory parts which might be affected must be identified and inspected.  If problem
has a life-threatening consequence, products in the field have to be identified for potential recall.

Identify the Root cause – An investigation and failure analysis is performed to determine the root cause of the problem. It is not enough to identify the first-level or intermediate-level causes of the problem. The true root cause has to be uncovered and corrected or a similar problem will eventually manifest itself.

Root cause analysis may start with brainstorming sessions among team members and move on to data analysis that further explores multiple potential causes. Different kinds of charts and diagrams are used in data analysis including histograms, Pareto charts, scatter charts, and concentration and affinity diagrams.

It is important to document the analysis discussion, findings and conclusions using cause-and-effect charts and fault tree diagrams. A five-whys type of analysis is also useful to understand all the intermediate-level causes between the root cause and the problem manifestation.

Corrective Actions – This discipline entails identifying all possible corrective actions to address the root cause of the problem. The owners of the corrective actions, the target dates for completion and the rationale behind each should be documented. It is sometimes a good idea to have a preliminary evaluation of the corrective action to test its effectiveness before wasting time and money on fully
implementing an incorrect solution.

Corrective action includes further containment of potentially affected similar production or inventory based on the root cause identified. If problem has a life-threatening consequence, products in the field have to be identified and notifications issued for recall or repair.

Permanent Correction – Once the corrective actions have been defined and tested, the next step it to implement them and verify their effectiveness. If the corrective actions are found to have deficient effectiveness it might be necessary to return to the prior step and create new corrective actions.

Preventive Actions – Lessons learned from the above investigation and corrective actions might be able to benefit other processes or areas with similar vulnerabilities even if not affected under the current
situation. Preventive measures should be outlined if its possible to prevent similar problems in the future. Preventive actions, like corrective actions, require owners and target dates.

Acknowledge Success – The last step of the 8D process is management’s formal recognition of the accomplishments of the team. This demonstrates the commitment of the management team to the continuous improvement process and is good internal publicity for the effectiveness of these initiatives.


The Continuous Improvement Tracking System

Common to any methodology is the need to document findings, actions and results. A key component of the enterprise manufacturing and quality system is the database that will document and track on-going process improvement efforts includng corrective actions ensuring a consistent review process and verification of effectiveness of any corrective action. The same database log can be used to track and resolve productivity or quality problems. The effectiveness of process improvements should be reflected and documented in relation to improvements in key operational metrics. Consider tracking all your process improvement efforts in one place.

References

“The Toyota Way”, Jeffrey K. Liker, McGraw Hill, 2004

“Lean Six Sigma”, Michael L. George, McGraw Hill, 2002

 “Root Cause Analysis”, Bjorn Andersen and Tom Fagerhaug, ASQ Quality Press, 2006

The Role of Quality Management within the Lean Manufacturing Philosophy

The Toyota Production System recognizes that managing quality is as important as managing just-in-time strategies in a Lean Manufacturing philosophy.

Quality management topics like Six Sigma, DMAIC, and Jidoka are important in the context of Lean Manufacturing because the ultimate goal is to eliminate waste in the value stream and one of the most common types of waste is the correction waste. Correction waste happens when it is necessary to work around poor quality in components or material from suppliers or it is necessary to repair, rework or scrap defective product units.

Six Sigma techniques and strategies are widely used in conjunction with Lean Manufacturing initiatives to help eliminate waste attributed to poor quality by providing the tools to focus on the right problems, diagnose the right root-cause of issues, and apply corrective action as quickly as possible. Six Sigma methodology uses statistical tools to systematically analyze processes and reduce process variations leading to better quality and performance.

Quality management requirements can be viewed sometimes as creating a burden of cost and time on the manufacturing process. The potential burden on the overall value stream should be a considered, but more importantly we should continually stress the benefits of predictable consistent quality to the entire team and identify the cost of poor quality to the overall performance of the organization.

Quality metrics are as important as throughput, cycle-time and schedule adherence metrics in our overall assessment of a Lean organization.  Metrics are needed to narrow areas that require attention for continuous improvement (Kaizen) initiatives, and to quantify the organization’s ongoing improvement achievements. We cannot improve what we cannot measure.

Quality management starts in product and process design. Concepts like Poka-yoke and Jidoka stress that the design of a product and process should eliminate the probability of error whenever possible, making the correct process visually obvious, and making any errors immediately visually obvious. Standardization, visual work instructions, and automated inspection are some of the tools used to minimize the potential for errors.

Automation in inspection, record keeping, and verification steps can relieve some of the burden of quality processes. Automation is also a double bonus because every time we can take some manual clerical steps out of the process, we are not only cutting down cycle time and labor, but we are also eliminating potential points of human error in the equation.

Statistical sampling and auditing methodologies allow us to further reduce the cost of inspection procedures by providing the appropriate quality confidence level with less than 100% inspection requirements. Automated data collection and statistical software tools are able to continuously monitor multiple processes and increase oversight as needed on suppliers and internal processes that are falling out of normal parameters. These tools greatly decrease overhead when compared to other traditional manual oversight methodologies.

Technology solutions can automate analysis in the background and alert assigned personnel automatically based on defined rules that are consistently implemented by the system. Technology can be viewed as a sidekick that is performing analysis on the side and bringing to the attention of the operator any out of control condition as soon as it is detected.

Another aspect of Toyota’s Lean philosophy is a culture of stopping to fix problems right away--the earlier the better. In order to do this we need (1) tools to help us achieve quality the first time, (2) tools to monitor processes that are susceptible to variables like environmental conditions, material variances, equipment wear and tear, or operator training, and (3) an efficient corrective action process.

Six Sigma practitioners use the DMAIC process to systematically reduce variability in a manufacturing process. Lean practitioners also refer to the Toyota problem solving methodology and kaizen workshops. All of these different methodologies can come together in a Corrective Action system to track all continuous improvement efforts. A  Quality Management System and Corrective Action System are essential to documenting and tracking the entire problem-solving process ensuring a consistent review process and a closed-loop methodology that verifies effectiveness of the corrective action or reopens the problem for further analysis. The corrective action system can be used to also track and resolve productivity problems that are not related to quality issues.

The inspection and genealogy data recorded during manufacturing is not only used for performance metrics and root-cause analysis. In regulated industries, historical data is required to allow process certification and audits. Regulatory and audit considerations are becoming more common in many industries. Regulatory guidelines from agencies like ISO, FDA, FAA, DoD, or ASME are all aimed at encouraging process control and quality assurance throughout the entire manufacturing process in order to achieve consistent quality results at the end. This general goal of the guidelines should be embraced as a goal to eliminate waste of poor quality due to deficient quality management processes.

In today’s highly outsourced manufacturing practices, it is critical that the requirements for quality management reach out to partners and suppliers.

Manufacturers have tackled regulatory requirements in the past with tons of paperwork, labor intensive validation processes, and a variety of disconnected systems and spreadsheets. Many of these old processes require double validation because manual procedures are prone to error. However, modern manufacturing and quality management systems are able to streamline and automate processes making it possible to improve productivity while enforcing compliance.

Root-Cause Analysis – Learning From Our Mistakes – Part 2

Can we spend too much time dwelling on the past? Is it worth it? If we can learn from it and avoid repeating the same mistakes, it is definitely worth the effort.

A few months ago, I listened to a great presentation by Brian Hughitt from NASA’s Office of Safety and Mission Assurance. His presentation left on impression on me and I would like to share some highlights with you. The story is another testimonial to the importance of finding the true root-cause of a problem. However, this story is not about NASA, its about Boston’s Tunnel Project.

Boston’s Central Artery/Tunnel Project is the most expensive highway project in U.S. history. The 7.5 mile underwater tunnel to Logan International Airport used 3.8 million cubic yards of concrete (enough for a sidewalk from Boston to San Francisco and back 3 times) and 541,000 truckloads of dirt (about 15 stadiums filled to the rim with dirt.) The tunnel has been plagued by leaks, falling debris, delays and other problems linked to faulty construction. The initial price tag was $2.6 billion and it was supposed to be completed in seven years. Instead, it took nearly 15 years and repeated cost overruns drove the price tag up to $14.6 billion.

At 11:00pm on July 10, 2006 a 1991 Buick passenger car occupied by a 46 year-old male driver and his 38 year-old wife was traveling eastbound in the I-90 connector tunnel, en route to Logan International Airport. As the car approached the end of the connector tunnel, a section of the tunnel’s suspended concrete ceiling (26 tons) detached from the tunnel roof and fell onto the vehicle, crushing its right side. The driver’s wife, occupying the right-front seat, was fatally injured.  The driver escaped with minor injuries.

The investigation of the accident revealed a history of findings and reports that should have led to earlier action that might have prevented this fatal incident.

The proximate cause of the structural failure has been attributed to the use of an epoxy anchor adhesive with poor creep resistance. The anchor attaches the hanger for the ceiling panels to the concrete roof of the tunnel. Post accident testing revealed that fast-set epoxy had been used and that, while both fast-set and standard-set epoxy performed similarly in short term tests, they differed dramatically under long term load.

                             

                            ***

Creep - Epoxy is a polymer and its stiffness is time and temperature dependent.  If a load is applied suddenly, the epoxy responds like a hard solid.  But if that load is then held constant, the molecules within the polymer may begin to rearrange and slide past one another, causing the epoxy to gradually deform.  As the deformation increases, it becomes irreversible

                            ***

Fast-set epoxy was used because it was more convenient and perceived to have very similar performance qualities. Why were the engineers working on this project unaware about the creep characteristics of the fast-set? Was it negligence by designers, suppliers, procedures, or all of the above? There is much to learn from the story leading up to this event.

The following fishbone diagram gives an idea of the wide variety of factors requiring investigation on an incident like this. A thorough root-cause investigation is a big investment, and the reality is that we cannot perform analysis to this detail on every incident. The size of the potential impact of the issue at hand must be considered and balanced with how much effort goes into an investigation. However, some companies like Toyota strongly believe that it is good to contain a problem right away and investigate the root-cause to prevent further impact and variability on the planned production process.

 

The root-cause analysis on this incident yielded some interesting findings. Let’s look at the main contributing causal factors identified by the Safety Board’s report.

The Safety Board analysis starts with the tunnel design. The tunnel design didn’t have redundancy built into the ceiling suspension. The majority of U.S. tunnels have continuous ceiling panels that extend into the concrete wall. If the hangers fail, the ceiling is self-supported. Design specifications for this tunnel did not incorporate a provision for attaching a suspended ceiling, even though it was known that one would be needed.  Consequently, the tunnel had no embedded ceiling supports. The design consultant repeatedly recommended undercut anchors versus the adhesive anchors ultimately chosen. In addition, in order to save cost and time, a change was made to deviate from the original design and use heavier pre-cast concrete panels in lieu of custom-engineered laminated lightweight concrete panels.

Failure Modes and Effects Analysis (FMEA) was incomplete. Creep was not identified as a potential failure mode. Consequently, risk mitigation measures were not implemented.

The Safety Board identified a high concern with the ICBO industrial standards. The ICBO AC58 calls for either a design safety factor of 5.33 or a 120-day creep test for fast-set epoxy. The creep test is optional even though the ability to sustain a load over a period of time is a typical requirement for almost any type of fastener.

A design engineer should be provided with all of the relevant information about a product before it is used in a safety critical application. However, no documentation was provided by the supplier specifying which epoxy formulation was being supplied, and neither the contractor nor the design consultant questioned which epoxy was used. Both assumed that the epoxy provided by the supplier was suitable. 

To support product qualification, the supplier provided an Evaluation Report (ER) which included bond strength tables specifying a safety factor of 5.33 for fast-set epoxy. Nothing in the ER tables or footnotes indicated that the fast-set epoxy should be limited to use with short-term loads regardless of the safety factor employed. A restriction for use in short-term load applications was contained elsewhere in the ER, but could have easily been overlooked.

The Safety Board learned during the investigation that fast-set epoxy had been tested for creep performance in 1995 and 1996 and had failed to meet the standard, but still qualified under the current AC58 standard.

The supplier should have made a clear distinction in all its literature between the relative capabilities of its standard-set and fast-set formulations.  It did not do so, even though before the epoxy was provided, the company had conclusive evidence that its fast-set epoxy was susceptible to creep. The Safety Board concluded that the information provided by the supplier was inadequate and misleading.

The Safety Board also found fault with the construction contractor and the design consultant for not adequately reviewing the product qualification documentation.

But the story gets even better… I mean worse…

On September 9, 1999, (seven years before the accident) a construction contractor employee installing ventilation ductwork over the tunnel ceiling noticed that several of the anchors had begun to pull out.

The finding was documented and on November 12, 1999, a proof load test was performed on one of the anchors that had shown significant displacement (9/16”).  The engineer noted that “the bolt held for a few seconds, then began to pull out with almost no resistance.

When the supplier was called to examine the anchor displacements, they seemed surprised that the anchors that had been successfully proof tested only a few months before could be failing. The fact that the fast-set epoxy was subject to creep, was apparently not considered or was not known by the representatives who evaluated the failed anchors.  Even if the information about poor creep resistance was not common knowledge, some research into historical performance testing would have likely revealed it. 

Installation problems (e.g., excessive preload) were postulated by the supplier as the cause. There was no evidence of further testing or research by the supplier. Shouldn’t a supplier be more proactive about one of their safety critical components failing?

The response from the builder, design agent, and project manager was to perform more proof load testing. The original design service load was calculated at 2,600 lb-force. After original bolt installation, a proof test was conducted at 25% higher than design service load (3,250 lb-force) and the bolts passed. After the slippage was found, bolts were proof tested at the maximum load of 6,350 lb-force and the bolts passed again. Analysis determined that the actual load would be well under that, at a maximum of 2,823 lb-force.

At this point the builder and project managers decided to proceed with repairs and move on confident that the bolts would hold based on the testing performed. They do so even though a couple of employees had raised some concerns on emails.

"You’ve noted the key piece of information that is missing.  That is the cause of the anchor failure and how the repair procedure will overcome that… We are not trying to hold up construction, we are trying to make a determination that the installation is safe…”
      Design Manager e-mail concerning response to Deficiency Report

“Glaringly absent from the Deficiency Report is any explanation why the anchors failed and what steps are proposed to ensure that this problem does not reoccur.”
      Structural Engineer e-mail reply               

The root cause for the anchor displacement was never clearly identified and surveillance monitoring inspections were never implemented.

But wait there is more…

On December 17, 2001, a quality control inspector submitted a Noncompliance Report which stated: “Several anchors appear to be pulling away from the concrete.  The subject anchors were previously tested to the revised value of 6350 lbs, all of which passed…. Reason for failure is unknown.”

At this point, it should have been obvious that the prior remedies in response to the anchor displacement in 1999 had not been effective.  This was another opportunity to inspect all the installed anchors and, more importantly, to determine the cause of the anchor displacement.  Instead, the oversight team apparently considered the continuing failures as isolated instances and took no further action to address the problem in a systemic way. Even after being presented with evidence of anchor creep, project managers and overseers failed to recognize the inherent weakness in the epoxy adhesive – a weakness that could not be overcome even with the best installation practices or the most rigorous short-term proof testing.

This type of denial and cognitive dissonance is something to keep in mind when performing root-cause analysis. Our minds can filter information that conflicts with what we might already believe, making it hard to see other alternatives.

In November, 2003, the Design Agent published Inspection Manual for Tunnels and Boat Structures.  The manual required each ceiling hanger component to be inspected visually or by NDT. From the time the tunnel was opened to traffic until the day of the fatal accident, no tunnel inspections were performed.

Post accident inspection of the suspended ceiling displayed large numbers of anchors that had become displaced (~25%), and that the displacement was so obvious that even a cursory examination would have revealed that structural integrity was threatened.
 
Investigators asked MTA officials why the inspection manual was not used. The officials stated that the inspection manual was not used because (1) a tunnel inspection database needed to be developed, (2) the inspection manual was being reviewed by the FHWA and the MTA, and
(3) MTA personnel needed time to be trained on the manual.  

It is also amazing that federal regulations were not in place for tunnels. The FHWA National Bridge Inspection Program (NBIP) mandates bridge inspections at least once every two years, but there are no similar mandates for tunnel inspections. The FHWA’s Road Tunnel Design Guidelines does not address the design, construction, and inspection of tunnels.

The result of the investigation was that the panel support system had to be completely replaced with Hilti mechanical fasteners which had to be individually proof loaded.

In summary,

The investigation of this incident yielded incredible findings. Contributing causal factors included:
• Lack of Redundancy in Design
• Inadequate Regulations
• Lack of Awareness and Documentation on Product Performance
• Lack of Adherence to Quality Assurance Procedures

What sticks in my mind the most is that even though some emails expressed a concern, the root-cause was never identified and both builder and supplier went on with their business when  the hanger anchors, a critical safety component, appeared to be failing.

When it comes to safety, we need to perform adequate FMEA, we cannot afford to focus on project dates, and we definitely cannot afford to move on without identifying the root-cause of a failure. At a minimum, surveillance inspections should have been implemented to monitor the situation.

Paraphrasing Brian Hughitt, "The study of incidents like this one, incidents of “un-quality”, helps us become better students of quality."

Juran - The Father of Quality, Pareto, and perhaps Six Sigma

The field of quality management lost one of its founding fathers last week. Dr Joseph M Juran was a charismatic figure whose contributions helped shape the Quality function as we know it today. Juran’s contribution to the Quality field goes beyond statistics and equations. He helped define the terminology, methodologies and processes for the Quality field.

Juran's Quality Handbook is the Quality Bible and his “trilogy” books (Juran on Leadership for Quality, Juran on Planning for Quality and Juran on Quality by Design) should be required reading for the Quality professional.

As an Industrial Engineer, I had to work with the very thick Juran’s Quality Handbook in college. I had worked through many of the statistics and exercises, but I never fully appreciated Juran’s contributions until recently.

Juran wanted us to see Quality as more than just inspecting and counting defects on the factory floor. We can see this in his broad definition of quality terms below like Fitness For User and Quality Function.

• Fitness for Use – the extent to which the product successfully serves the purposes of the user during usage. Parameters for Fitness for Use include: quality of design, quality of conformance, abilities, and field service. 

• Quality Function - the entire collection of activities through which we achieve fitness for use, no matter where these activities are performed.

In the 1940’s, Juran defined the Pareto principle―the principle of the vital few and the trivial many. In general, the principle states that by tackling the top 20% of the issues we can address 80% of the cost of poor quality. This concept is critical to focusing improvement projects on the areas that will have the bigger impact, but the Pareto principle can be applied to many things.

The function of the Inspector might be as old as the pyramids of Egypt, but the Quality Control function as we know it today was defined during the World War II years.

Juran was a pioneer in Customer Satisfaction and Total Quality Management. He understood that the Customer and the Manufacturer have very different views of product quality and cost. In more recent years, our views of Total Cost of Ownership are starting to align with the Customer’s view.

Fool-proof the process if possible, train and empower the operators as much as possible, and put standards in place. Humans are fallible, biased and at times inattentive; therefore we must have procedures in place to compensate for human error.

Juran also understood that we need to motivate people to take action and reduce errors. Motivation is driven by upper management communications and campaigns for quality. Even though, he did not believe in the general premises of the Zero Defect philosophies of the early 1960’s, he did acknowledge that they had some positive results. The personal pledge was a good motivator. He also studied other incentive and motivational methods including the Japanese QC Circle, the USSR Saratov System and the Polish DO RO (Dobra Robota) System.

Juran spoke frankly about current quality practices. “Numerous specific quality crises and problems have been traced to the way in which quality was planned in the first place. In a sense, we planned it that way.”
• Quality professionals not involved early enough in the design process and we do not give Quality the importance it needs during product and process design.
• In many factories today, product designers are still developing new products, and then delivering the product specifications to the Manufacturing Department.
• With the advent of electronic systems, many companies proceeded to convert their manual systems directly into electronic systems without first getting rid of the deficiencies in the manual systems. As a result, their manual mess became an automated mess.
• Quality Planning has been done by amateurs. Process planners need to be trained to become professionals at quality planning.

Juran also stressed that upper management had to drive a Quality philosophy in the company.
• Quality must be a part of every agenda in the company.
• The business plan should contain quality goals.
• Each level must establish goals.
• Everyone must be trained.
• To establish and meet a goal, it must be measured...measure everything possible.
• Review progress.
• Give recognition for excellent achievements.

The legacy of this hard working immigrant will be with us for a long time. The Juran Institute will continue to promote the Quality field, and the next generation of Quality thought leaders are sure to be inspired by Juran’s life and work.

Other References on Juran:

New York Times: Joseph Juran, 103, Pioneer in Quality Control, Dies
http://www.nytimes.com/2008/03/03/business/03juran.html?ex=1362286800&en=ac0c70aadae138d9&ei=5088&partner=rssnyt&

JMJuran.com An Immigrant’s Gift
http://www.jmjuran.com/

Juran Institute
http://www.juran.com/

Quality Lab – 100 Years of Juran
http://qlab.ieem.ust.hk/qlab/download/Juran%20100%20Years.pdf