September 16 to 19, 2013
Mark Kasunic and James McHale
Software Engineering Institute
Competition and the desire for increased quality, profit, and customer satisfaction are what motivate companies to continually search for improvement. Data compiled by Capers Jones and others demonstrate that the TSP is one of the most effective approaches to improving cost, schedule, and quality in small, medium, and large software projects. This one-day tutorial will provide an overview of key TSP concepts, principles, methods, and benefits.
Key topics include
Gene Miluk and Timothy Chick
Software Engineering Institute
This tutorial will describe how to integrate agile goals, objectives, and techniques into the TSP methodology. TSP is a very robust methodology that provides flexibility in choosing system development lifecycles and development techniques. This tutorial will describe how to use this flexibility to incorporate the best of agile to achieve the performance results of TSP.
The tutorial is divided into three sections:
Understanding and tailoring the TSP cycle script
Alex Obradovic and Anthony Camisa
Software development in IT environments is a balancing act that requires negotiating release schedules with timely incident resolutions, business user assistance, software maintenance, technology upgrades, and many other activities. Since unpredictable events and changing business priorities can tie up IT software teams for hours or even days, schedule predictability and controlled defect density can seem like unattainable goals. In 2011, the Beckman Coulter IT department ran a pilot TSP program for a non-medical-device software project to determine if TSP could be used to improve quality and schedule predictability for IT teams.
The team was successful in reducing code and configuration defect density per thousand lines of code by 6.5 times from Cycle 1 to Cycle 3. The number of production incidents in the data center was reduced from 47 in 2011 to one in 2012. The cost of producing a defect-free, single line of new code was reduced from $5.95 to $1.53, and schedule accuracy was improved from 50% schedule overage during pre-TSP development to less than 5% schedule deviation at the end of Cycle 3.
This presentation will explore Beckman Coulter's IT TSP experience and pilot results and address TSP challenges that remain unresolved as the IT organization continues to focus on budget discipline while improving the speed, agility, and predictability of IT services and operations.
Have you ever asked yourself, "Now that we have all this historical data, what do we DO with it?" To maximize the value of TSP, it is essential to use your time and size data to improve team performance. In Launch Meetings 3 and 4, teams new to the launch process often have trouble with the accuracy of relative size estimates for work products and the effort required to create them. These doubts could increase the amount of time spent in Launch Meeting 7 if the team struggles to properly mitigate the potential uncertainty in the estimates.
A common challenge faced by such teams is the inability to use historical team performance data to create accurate relative size tables and productivity rates, assess the confidence and accuracy of the tables, and finally, calculate the appropriate risk reserves required when using this information—all in a timely manner. This presentation demonstrates a set of Excel-based statistical analysis tools developed by Urban Science based on the SEI's PSP training practices. These tools can dramatically improve the ability of teams to move with speed and confidence through Launch Meetings 3, 4, and 7. This will lead to a more successful Launch Meeting 9 since leadership will be impressed by the use of historical data to create a plan based on rational fact rather than best guesses and hope.
Oracle Canada ULC
Here's the first secret: TSP checkpoints are not really about data collection, estimation, quality, and schedule. They're about team dynamics, efficiency, motivation, passion, and empowerment.
This presentation will sum up the experience gained through hundreds of hours of interviews with developers and from conducting checkpoints for very diverse teams, including distributed multi-teams and cross-product teams. It will describe how broadening the scope of the checkpoint and refining the approach can help you uncover deep-rooted problems and dysfunctions that may cripple the teams' operations, and leadership misalignment that can contribute to low morale. The checkpoint can also show how teams work in reality instead of on paper and what developers value most.
You will gain valuable insight into the qualities needed to conduct a checkpoint that makes a difference, hear about techniques for uncovering major issues that can derail a team, learn how to create a report that is actionable, and discover how to lead or influence the needed improvements. The secret is using questions about data, quality, and schedule to transform the team into one that owns its risks, manages its project, and accelerates to success.
For planning and tracking the work to be performed, engineering teams have team processes, and individuals have personal processes. While these work well, the rest of the story—the steps needed to define and deliver products—may not be included. These additional steps are needed to help people understand what they are going to accomplish and how to track the accomplishment.
While Meeting 3 of the team launch does include a step for defining a process for the products, it is often inadequate due to time available during the launch and the technique used to document the information. Inserting an event ahead of the launch called a Process Modeling Workshop is effective in helping teams capture product processes in a manner comparable to TSP processes. With a well-defined set of product processes, teams perform effectively as they generate top-down and bottom-up plans during launch. In addition, plans based on accurate processes also help with the effective identification of measurements to be captured during operations.
This presentation will demonstrate NAVAIR's top-level approach to the relationship between product processes, the data collected and used during development, and how the data is used to analyze the quality of the product. It will also include a discussion of the process for future improvement.
Software Engineering Institute
When quality is a critical product attribute, software developers and teams are expected and motivated to perform at high levels. Paradoxically, numerous studies have shown that in high-stakes situations, many highly-skilled and otherwise successful individuals frequently experience a decline in performance, making more mistakes than novice practitioners in the same field. Other individuals actually seem to thrive in high-stakes situations and achieve their best results under extreme stress or in suboptimal conditions. Research on human learning and performance has identified specific neurological factors that cause some individuals to "choke" in high-stakes situations while enabling others to achieve "habitual excellence," the ability to perform at consistently high levels under a variety of circumstances.
By understanding the biochemical underpinnings of human performance and the external environmental factors that affect performance, coaches can be more effective in helping teams and individuals to overcome or prevent responses that result in suboptimal or faulty outcomes. This presentation will provide an overview of the neurological processes and biochemical changes associated with learning and performing various types of skills. It will also include guidelines that coaches can use to help their clients to know how and when to mindfully focus on their performance, when to "go with the flow," and how to use guided practice and other techniques that lead to automatic high-level performance. Individuals and teams who are able to achieve habitual excellence will be better able to perform well even under stressful conditions and achieve quality outcomes in those situations where quality really counts.
Most of the change agents I work with, including executives, leaders, and TSP coaches, feel overwhelmed and behind. They have many ideas that they feel they should be implementing, but they don't have the time. They also feel they should have more influence over a larger part of the organization, but they don't have the position or the time to make it happen.
This presentation will focus on developing the mindset of an exceptional change agent. I will present eight ways in which change agents traditionally think about and approach specific change challenges. In contrast to each of these, I will supply a different mindset that will enable change agents to do things in new ways to improve their value to the organization, decrease their labor, and ultimately change the world they care about.
Diego Vallespir, Fernanda Grazioli, Leticia Pérez, and Silvana Moreno
Universidad de la República, Uruguay
Data collected in the PSP courses indicate that the PSP improves the quality of the products developed and reduces the development effort. One way this has been determined is through statistical analysis of the evolution of the results (for example, defect density in unit test) obtained by the students in each program of the PSP training course. However, since the programs are in the same application domain, the improvement could be due to programming repetition (i.e., the learning effect). To explore the reasons for the improvements, we asked the following research question: Are the improvements observed in the PSP courses due to the introduction of the phases and techniques of the PSP or to program repetition?
To investigate this we designed and performed a controlled experiment with twelve software engineering undergraduate students at the Universidad de la República. The students performed the exercises from the PSP for Engineers I/II course without applying the PSP techniques.
The overall results, which will be discussed in detail in this presentation, indicate that the practices introduced by the PSP and not program repetition contributed to the performance improvements.
Cadence Design Systems, Inc.
Together, the PSP and TSP provide a revolutionary framework for individuals and teams to plan, track, and manage their work. The power of this framework is supported by an unprecedented level of integration between data at the team and personal levels.
But TSP teams do not work in a vacuum; they operate as part of a larger organization that has its own set of business processes and tools for managing projects. Thus, as TSP teams create plans and collect metrics, they almost universally face the need to integrate their planning and tracking data with external systems, including:
This presentation will discuss the challenges and pitfalls TSP teams often face as they attempt to bridge the divide between TSP tools and external systems. It will also highlight some of the unique and valuable opportunities TSP teams have for making use of external data and supplying rich data to external systems and stakeholders. Through case studies of successful integrations, we will share underlying tips, tricks, and best practices.
Noopur Davis and Darryl Davis
Retrospectives (known as postmortems in the TSP) are an essential part of the plan-do-check-act cycle, targeted to the "check" and "act" items of the cycle. They are an essential tool for harnessing the knowledge of the team and using it in a collaborative fashion to improve team processes. Yet retrospectives can quickly become routine and boring. Ask yourself the following questions: Do we learn from our retrospectives? Do any changes result from the learning? Do we understand our processes better? Do our retrospectives help create a team culture that reflects our values? Are we true to the concept of "empirical process," where data is used to support improvement decisions?
If you answered "no" to any of these questions, or just want to add some tools to your toolbox, this presentation will be helpful as we share failures, successes, principles, and quantitative and qualitative techniques learned over years of coaching retrospectives. We will discuss different levels of retrospectives (component/feature, cycle, project, and personal), how to prepare for retrospectives, analysis of planning and quality empirical data, mixing-it-up with different tools for conducting the retrospective, and follow-through. Real world stories and results will be used throughout.
Alliant Techsystems, Inc.
ATK Business Systems has been working to improve project performance and reduce cycle times for application development. Although getting management buy-in for TSP/PSP was challenging, a simple statement made about time on task at a TSP conference two years ago inspired the first step of our implementation. By using time on task exercises we determined our capacity and used LEAN techniques to reduce waste. Continuous improvement cycles led to the adoption of defect tracking, code reviews, scripts, and standard work. After applying a combination of LEAN and PSP techniques, average project durations were reduced by more than 50%, issue resolution improved from two weeks to two days, and development capacity increased about 30%.
Although it might not be the preferred method, we used an opportunistic approach to achieve a grassroots adoption of PSP techniques in the organization. At this point, we have generated demand and sponsorship to train all development resources in PSP during the next year. Modified programs for process management and quality assurance are expected for management, business intelligence, and infrastructure resources.
This presentation will review each improvement made during the past two years on our path to implementation of PSP across ATK. It will cover the impact of each change, including the changes in development approaches and management responses.
The Wall Group
In many sports, the top coaches are becoming as well-known as their players. While there is not much difference in the coaches' technical knowledge of the game, in the information they have access to, or in the rules they must follow, what often differentiates them is their ability to be effective teachers of what it takes to "move the ball" in the process of creating a winning organization.
TSP coaches play a very similar role. This session will review some of the practices and principles that top sports coaches use to teach and motivate their athletes to perform at their highest levels and explain how you can use the same techniques for your teams. TSP coaches and team leaders who adapt these ideas can improve the performance of team members and increase the team's ability to succeed.
Masanobu Umeda, Keiichi Katamine, and Masaaki Hashimoto
Kyushu Institute of Technology
Next Process Institute Ltd.
In cooperation with the SEI, the Kyushu Institute of Technology introduced software process education for graduate students based on the PSP and TSP. Students in PSP courses have generally achieved overall performance results similar to those using PSP in industry. In contrast, students in TSP courses, which are based on TSPi, a simplified introductory version of the TSP for educational purposes, were not as successful and could not complete more than two cycles due to repeated schedule delays (although quality was remarkable). We wanted to determine if course management problems were causing these delays, resolve any related issues, and allow TSPi teams to finish projects successfully.
We studied one team in 2010 and two teams in 2011 and determined that the main reasons for the schedule delays were insufficient engineering knowledge to produce deliverables, improper performance of unfamiliar team processes, and time wasted on understanding the out-of-date TSPi tool. In 2012 we implemented several improvement actions to help with these issues.
Our teams in 2012 demonstrated that TSPi works for creating high performance teams if several improvements are incorporated for educational purposes. In the future, we also plan to examine related lectures to ensure they are effectively presenting the engineering knowledge required for TSPi. This presentation will describe our efforts in evaluating and improving the use of TSPi in our classes.
Launching a project can be a difficult and stressful situation, and it can become even more complicated when you have different functional silos such as Business Analysts, Developers, and Quality Assurance. Each team is focused on what may be viewed as separate and distinct areas of the project.
If this is familiar to your company, your coach's time could be occupied for three times longer than it should be – a clear sign that all of your teams need to develop a better approach now! When inspections require all functional areas to be present, how do you ensure that you have allocated enough time in each schedule? How do you identify dependencies among the functional silos that can cause gaps in your schedule because deliverables don't line up correctly?
What if Quality Assurance has finished their test cases but Development has not yet completed the coding? Do you combine them together in a single team launch? How can you keep them all engaged in the launch process when discussion is focused on a single group? Should you use a multi-team launch to coordinate the activities of these different functional teams?
Once you have answered these questions and launched the project, more challenges still exist, and leadership views these post-launch actions as a single project. How do you keep management informed of the status of the project when there are many different groups involved? How do you perform defect analysis when the Business Analysts are working on requirements for Cycle 3 and the Developers are working on coding for Cycle 2?
This presentation will address these issues that arise with functional silos and share Urban Science's successful approach of using the Process Dashboard to manage projects – one that can be applied to any organization or tool.finally went live. Just a few weeks later, the largest initial public offering in Mexican market history went off without a problem. The system was nearly defect free during its first seven months of operation, and a sister system running another version of the same software went live on April 15, 2013 to replace several other systems trading options, futures, and fixed income instruments.
The original software development was done between October 2009 and July 2011 using the TSP and Architecture Centric Engineering (ACE). This presentation will tell the story of what the team did after development and describe operational experiences with the new systems.
University of the Witwatersrand, Johannesburg
Even after a successful TSP pilot program, it has been remarkably difficult to achieve widespread adoption of the TSP in South Africa, and impossible to ensure that TSP principles are applied consistently and correctly. This presentation describes an innovative program launched in January 2013 by the Joburg Centre for Software Engineering that aims to significantly increase the adoption of the TSP. The program aims to set up a number of High-Maturity Software Development (Hi-Mat) Units. Drawing on the concept of a franchise, a standard "operations manual" has been defined for the Hi-Mat Units that draws heavily on the TSP and PSP.
Each unit is recruited from scratch and trained to follow the standard operations manual. Just as franchisees are not permitted to deviate from the processes laid out in the operations manual, Hi-Mat Units will be obliged to consistently follow the defined standard operations manual.
Between January and May 2013, four Hi-Mat Units were set up in three South African cities and began work on development projects. A research project involving researchers from the University of the Witwatersrand in Johannesburg and the University of Cape Town measured the performance of these units. This presentation will include initial results of the project.
Tom Beckley, Al Schmidt, and Elias Fallon
Cadence Design Systems, Inc.
In May 2012 Cadence began implementing a company-wide quality initiative. While the electronic design automation market Cadence participates in has historically had a high tolerance for software defects due to the constant need for rapid innovation, Cadence determined that it needed to provide significantly higher quality products and solutions to enable scalability, productivity, and continued high investment in innovation—and address advanced nanometer requirements at the same time.
In this presentation we will outline customer challenges in the global electronics market and describe the paramount requirement for higher quality products and solutions. We will then focus on the software development aspects of the Cadence quality initiative and describe how we are driving improvements throughout our organization, which includes ~2,500 software developers, dozens of distinct product lines, and disparate global functional teams. We know this will be a long journey requiring a fundamental culture change based on a sustained, top-down leadership commitment coupled with a bottom-up commitment by development teams to own their quality using agile techniques ("building it right the first time").
The Cadence quality initiative will deliver improvements in product and solution quality in the short, medium, and long term. A key cultural change aspect of the long-term effort includes implementing TSP across all software development teams. In the near to medium-term, team and organizational best practices are being adopted in addition to TSP, which are built into annual Quality Action Plans (QAPs) provided by approximately two dozen technology organizational units across Cadence. Moreover, all QAPs must assess and address the longstanding legacy code base ("software debt") to enable quality and long-term scalability. Now about one year into the formal initiative, we will share lessons learned to date and observed results.
Tecnológico de Monterrey
Since August 2006, we have been teaching PSP training to undergraduate and graduate students at Tecnológico de Monterrey. At the TSP Symposium 2008 I presented a preliminary report with results to that date and the future work to be done. Since then I have tried some different approaches to overcome the challenges reported. In this presentation an analysis of seven years of student PSP data, results achieved so far, the different approaches tried and which one is now working, and the tools I have developed to help in reviewing the PSP assignments will be discussed. Also, the challenges we still have not overcome, and some improvements to teaching PSP to university students will be proposed.
Session Speakers and Paper Authors: If you have been approved to submit a presentation, a paper, or both for inclusion in the Symposium proceedings, you can now sign and submit copyright permission online.
Please note that each author must sign off on the copyright permission, not just the primary speaker or author.
July 22 –Final Papers Due to SEI
August 30 – Final Presentations Due to SEI
August 15 - Early-Bird Registration Deadline
August 31 - Hotel Room Block Closes
Sign up to receive email updates about the TSP Symposium.
For questions regarding the conference or more information about using the TSP, contact
SEI Customer Relations
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA 15213-3890
Please tell us what you
think with this short
(< 5 minute) survey.