SATURN 2009 Presentations

This file contains presentations from the SEI Software Architecture Technology User Network Workshop (SATURN 2009) event (May 4-7, 2009 in Pittsburgh, Pennsylvania).

Sessions and presentations included:

*A Framework for Making Architectural Decisions in a Business Context, Jeromy Carriere, Rick Kazman, and Ipek Ozkaya
*Architectural Measurements & Metrics at All Scales, Joseph Starwood
*Architecture Governance and Rules Enforcement Using AOP and SonarJ—A Case Study, Srini Penchikala
*Architecting for Highly Available, Scalable, and Reliable Mission-Critical Applications, Diego Dagum
*Architecting the Organization, Kenneth Kunkel
*A Simple and Flexible Specification for an Enterprise Architecture Practice, David Cuyler 
*Bottom-Up Software Product Line Design A Case Study Emphasizing the Need for Stakeholder Commitment, Jens Doppelhamer, Heiko Koziolek, and Roland Weiss
*Bridging Systems and Software Architecture, Anne-Marie Buibish, Caleb Conley, Amy Lange, James Lewis, and Elizabeth Penisten
*Career Track for Architects in IT Organizations, Shankar Kambhampaty 
*Dealing with Quality Attribute Requirements as the Hero, Not the Witch, Andre Gous
*Experience with the Architecture Improvement Workshop, Lawrence G. Jones and Rick Kazman
*How Acquisition Practice Can Impede SOA Governance, Lloyd Brodsky
*How to Conduct a Contextually-Driven Systems Architecture Review, Michael F. Dedolph
*Improving the Definition of Quality Attribute Scenarios by Using Requirements Patterns, Aldo Dagnino
*Integrating Usability Supporting Architectural Patterns in a Product Line System's Architecture, Len Bass, Elspeth Golden, Bonnie E. John, and Pia Stoll 
*Leveraging ADD for Objective Technology Evaluation and Decision Analysis, Elizabeth Kielczewski
*Limits to the Use of the Zachman Framework in Developing and Evolving Architectures for Complex Systems of Systems, Philip J. Boxer and Suzanne Garcia-Miller
*Software Architecture Beyond Blueprints, Issac Eldo
*The Impact of Conducting ATAM Evaluations on Army Programs, John K. Bergey, Stephen Blanchette, Jr., Mark H. Klein, and Robert Nord
*The Role and Development of an Enterprise Architect: A Devil's Advocate’s Perspective, Robert Ellinger 







A Framework for Making Architectural Decisions in a Business Context, Jeromy Carriere, Rick Kazman, and Ipek Ozkaya

In any organization, it is useful to have a model to associate a value with architectural decisions. More generally, any effort needs to have an associated value to offset its labor and capital costs. For projects that deliver direct functionality that increases an organization's revenue, value is easily determined, and thus the return on investment argument is clear. Nowhere is this more true than in a dynamic, young, agile, organization with a large and growing market footprint; such organizations are commonly driven unswervingly by analytical assessment of the efficacy of the business (sales, marketing, and manufacturing) and technology delivery functions. Unfortunately, "architecture projects"—those that aim to improve one or more quality attributes of a system via a structural transformation system without (generally) changing its behavior—are extremely difficult to evaluate in the same fashion. The general case is to resort to anecdotal and informal arguments that speak in general terms of risk reduction (i.e., improved reliability) or increased developer productivity (i.e., improved maintainability). These arguments are typically unsatisfying to organizations accustomed to decision making based on concrete metrics.

This presentation discusses research done to address this dilemma, in a collaboration by VistaPrint, the Software Engineering Institute, and the National Research Council of Canada. Specifically, we present a model derived from analyzing and modeling actual projects undertaken at VistaPrint, a rapidly growing marketing services company targeting small businesses globally. The presentation shares the details of the approach and the specific results derived in the context of VistaPrint's portfolio of architectural transformations.




Architectural Measurements & Metrics at All Scales, Joseph Starwood

The value of IT architecture is difficult to demonstrate. The right set of measurements and metrics is essential. Such measurements and metrics must be relevant and traceable to the business, and understandable to its stakeholders. They must also support decision making, and be meaningful to executives, managers, and practitioners. Establishing measurements and metrics is not easy or reliable. Many IT organizations fail on their first attempts. This presentation describes the use of definitions, models, and selection criteria for establishing a rational and manageable set of measurements and constituent metrics to demonstrate the value of IT architecture and support operational decision making as well as continuous improvement. Upon overcoming several challenges, dashboards and scorecards report the measurements and metrics for architecture maturity, governance vitality, and solution conformity. These communicate the value of IT at all levels.




Architecture Governance and Rules Enforcement Using AOP and SonarJ—A Case Study, Srini Penchikala

Although many companies have some kind of application architecture standards, they don’t usually have a mechanism to enforce those standards. As a result of this lack of architecture governance, the implementation (code) often doesn't match the requirements (reference architecture). Enforcing reference architecture guidelines promotes consistency and modularity in the system. In this presentation, I will talk about the significance of enforcing architecture rules and how to implement policy enforcement in software development projects. I will also discuss an architecture enforcement framework created to “inject” architecture rules and design policies into the continuous integration (CI) process using aspects to enforce quality of the code. The framework also uses tools like Eclipse, AJDT and Maven to integrate policy enforcement into the agile development process to detect architecture deviations early and often and validates that the design and code are in compliance with the reference architecture (RA).




Architecting for Highly Available, Scalable, and Reliable Mission-Critical Applications, Diego Dagum

Mission-critical applications are vital for the normal flow of operations in any organization. This range of applications is characterized by a series of attributes, such as scalability, fault-tolerance, resiliency, and performance. While it's pretty easy to get any of these attributes in isolation, meeting all them together is not a straightforward process. It typically happens that, when applying techniques intended to maximize—let's say—scalability, performance might be eroded; performance enhancement may cut graceful degradation capabilities, and so forth. This session explores all those tradeoffs, offering effective guidance to deal with mission-critical attributes altogether. After attending this session, the attendee will be able to identify those friction points, applying integral techniques in order to maximize the fulfillment of those attributes at once. This session also offers health checking techniques for diagnostics, to guarantee that quality-of-service agreements are being fulfilled.




Architecting the Organization, Kenneth Kunkel

There is a ton of technologies out there but the real challenge is absorbing those technologies into your organization. This presentation will offer you different approaches that organizations have taken. The discussion arises from the experiences of Ken Kunkel at IBM and Deloitte Consulting and from the experiences of your fellow attendees.





A Simple and Flexible Specification for an Enterprise Architecture Practice, David Cuyler 

This presentation describes a simple and flexible enterprise architecture practice employed by Sandia National Laboratories that is designed to help mitigate the tendency toward "too much information." The practice provides enterprise architects with a framework and principles that guide their work and defines an overall process for enterprise architecture, the outputs of which are the following: as-is (baseline) and to-be (target) descriptions of business, information systems, and technology architectures, the interactions and alignments within and among these architectures, and a road map for transforming the enterprise from the baseline to the target state. The architecture descriptions identify the scope and bounds of the enterprise, characterize enterprise stakeholders and the strategic intent of the enterprise toward them, describe the current state of the enterprise, and recommend a future state that facilitates the realization of the enterprise strategic intent. The roadmap, in turn, defines a transition plan to guide progress toward the target architecture and provides methods to measure progress during the transition. The work products of enterprise architecture are intended to provide concise, relevant, timely, and accurate information to decision makers and program managers regarding the realization of the desired future state of the enterprise.





Bottom-Up Software Product Line Design A Case Study Emphasizing the Need for Stakeholder Commitment, Jens Doppelhamer, Heiko Koziolek, Roland Weiss 

In 2007 an internal survey of more than 50 of ABB's robotics software applications revealed a substantial functional overlap in many of these applications. Similar functionality had been developed more than seven times for different applications by independent teams. The lack of systematic reuse causes high development and maintenance costs and delays time to market for new products. On this behalf, we analyzed three different robotics software applications developed by locally distributed development teams. We proposed the design for a software product line architecture (PLA) based on the identified core assets and variation points. The presentation describes our key design principles and describes the lessons learned from introducing a PLA based on existing products. The goal of our project was to increase reuse among the software applications, lower maintenance costs, and ensure faster time to market for new applications within the robotics domain. Therefore, we first separated concerns in the analyzed robotics applications, as they partially bundled distinct features. The PLA design was constrained by the expectation of retaining existing functionality and providing a common look and feel among the applications. Furthermore, multiple extension mechanisms within the PLA were desired to support user customizations.

During the project, we interacted with many stakeholders of the architecture (including product managers, development teams, and customers) for requirements gathering and iterative confirmation of the product line architecture.

As a result, we have designed a PLA bottom-up from existing ABB robotics software. Stakeholder commitment is essential for a successful product line adoption and must be ensured proactively.





Bridging Systems and Software Architecture, Anne-Marie Buibish, Caleb Conley, Amy Lange, James Lewis, and Elizabeth Penisten 

The increasing complexity of large-scale software-intensive systems, along with well- documented software project failure rates, has brought attention to ensuring the architecting process provides value to the system development effort. As architecting processes become integrated into traditional engineering processes, it is important to seamlessly transition among enterprise, systems, and software architecture methods to realize the benefits of architecting to the overall system development. Ineffective transition has introduced the likelihood of increased cost and risk due to "disconnects" as architecture transitions through the various levels of decomposition.

Raytheon has explored the relationship between systems architecture and software architecture in an effort to improve the capability to transition between these architecting activities and improve internal engineering processes. This paper will present Raytheon's findings, including identification of key cost and risk drivers, recommendations for improving the capability to transition, and a discussion on the use of architecture methodologies to analyze the problem.




Career Track for Architects in IT Organizations, Shankar Kambhampaty 

IT service provider organizations are being increasingly challenged to develop larger and more complex solutions to address the rapidly changing business needs of their customers. Developing such solutions requires mature architecture processes and practices within an organization, supported by highly experienced and knowledgeable pool of architects. Several challenges exist in not only creating and nurturing such teams but also in ensuring that over a period of time talented and experienced architects aspire to be part of such teams, creating substantial value for the organization and its customers.

One of the critical success factors in developing high-performance architecture teams in IT service organizations is a well-formulated career track that provides a structured mechanism for growth and maturity of architects and architecture practices within the organization.

This paper describes the needs of IT service organizations and the required roles of architects in these organizations for providing large and complex solutions to their customers. It also discusses the different architecture team compositions possible in an enterprise along with their pros and cons. Based on experience in constituting and managing such architecture teams, this paper presents the significance of defining a career track for architects in IT service organizations. It also provides insights into the results seen in institutionalizing such a career track for architects.




Dealing with Quality Attribute Requirements as the Hero, Not the Witch, Andre Gous

Placing software into production will uncover quality-related problems at a time when it's expensive to correct them. If the neglected quality attribute requirements have highly visible results, they empower political opportunists with 20/20 hindsight to suddenly become interested in the project like never before, and the neglected quality attribute requirements become the subject of a witch hunt.

Whoever is funding a project owns the responsibility for conveying the requirements, including quality attribute requirements. And, in many cases, the project sponsor neglects conveying requirements in general and quality attribute requirements in particular. In essence, the project sponsor refuses to assume the responsibility, thus in effect conveying "Build me something. But I don't want to tell you what I need."




Experience with the Architecture Improvement Workshop, Lawrence G. Jones and Rick Kazman

Many organizations have successfully applied the Software Engineering Institute's Architecture Tradeoff Analysis Method® (ATAM) to evaluate software architectures relative to the achievement of system quality attributes. While the benefits of the method are widely recognized, there has been less guidance about how to systematically use the evaluation results to improve the architecture. In this presentation, we describe the Architecture Improvement Workshop (AIW), which answers the "Now what do I do?" question that naturally arises after the evaluation. We describe the steps of the method, outline the information collection techniques used, and report on some of the benefits achieved during pilot use. We will describe progress made in an engagement with the U.S. Air Force, using the AIW to support the evolution of the Space Surveillance Network Analysis Model (SSNAM) system. We will also describe an extension to the method: the Extended Evolutionary Architecture Improvement Workshop (E2AIW).




How Acquisition Practice Can Impede SOA Governance, Lloyd Brodsky

Much of the work in implementing and operating service-oriented architecture is outsourced. All of this work is governed by the contract signed with the company or government entity purchasing the work. It's generally acknowledged that governance is crucial to SOA success, but governance is rarely integrated into the acquisition process or the resulting contract. This paper identifies what should be included in a model contract and the potential consequences if it isn't. Particular attention is paid to establishing a market-feasible dispute resolution process to deal with service-level agreement breaches and establishing incentives to align contractor interests with those of the purchaser.




How to Conduct a Contextually-Driven Systems Architecture Review, Michael F. Dedolph

When the World Trade Center collapsed, switching systems in the basement correctly diagnosed which lines were still working, and continued to connect calls using backup power for several days. One factor contributing to this remarkable product reliability was the Bell Labs practice of early architecture reviews.

The Software Engineering Institute Architecture Tradeoff Analysis Method (ATAM) provides a standardized technique for evaluating software architecture. The review method presented here provides an alternative (and complementary) approach to architecture reviews that can be flexibly tailored based on the context for many kinds of systems, and used in any problem domain.

In this session, the presenter will

provide a simple model for defining and categorizing systems architecture that is representationally independent
describe how to conduct an architecture review, using methods based on Bell Labs Systems Architecture Review Board (SARB) process
discuss how the SARB methods can complement and supplement ATAM reviews





Improving the Definition of Quality Attribute Scenarios by Using Requirements Patterns, Aldo Dagnino

This presentation will provide requirements patterns that can be used to define non-functional requirements, with a good level of detail, precision, and clarity. This presentation assumes that the non-functional requirements are the basis for defining the software quality attributes, which in turn are fundamental for the development of the architecture of the software product. This presentation is motivated by the fact that in the software industry, software engineers seem to have a great deal of difficulty defining clear and precise non-functional requirements.



Integrating Usability Supporting Architectural Patterns in a Product Line System's Architecture, Len Bass, Elspeth Golden, Bonnie E. John, and Pia Stoll

This presentation will describe the application of Usability Supporting Architectural Patterns (USAPs) to the architecture design of a product line of systems. The patterns were delivered by a web-enabled prototype tool. The tool prompted the designers with architecture responsibilities specific to their chosen usability scenarios, then required the designers to respond to each responsibility and additionally provided the designer with implementation instructions in a textual, rather than a component, form. The designers used the USAP delivery tool in the early design phase and were extremely positive. Two designers using the USAP delivery tool for six hours discovered 14 major issues related to usability support in the core architecture that were not previously incorporated in the architecture design. Two potential new subsystems were identified during this exercise.




Leveraging ADD for Objective Technology Evaluation and Decision Analysis, Elizabeth Kielczewski

Technology evaluations can present significant challenges in software engineering practice. In this presentation, we describe how we addressed them in an industrial project. We successfully leveraged and extended the Attribute-Driven Design (ADD) method. To obtain a greater objectivity and finer granularity in prioritization and evaluation of criteria, and to generate the overall evaluation results in sensitivity analyses, we used the Analytic Hierarchy Process (AHP). In this presentation, we discuss our experiences, in particular

defining the project scope of the technology evaluation initiative
designing comparable prototype applications in multiple technologies with a common set of measurements    
balance between efficiency of nominal scales and accuracy of AHP weighting technique    
benefits of AHP for increasing the accuracy and objectivity of technical judgments, delivery of quantitative results, and ease of performing sensitivity analyses

Organizations dealing with the challenges of making technical decisions regarding software technologies may find this approach beneficial.





Limits to the Use of the Zachman Framework in Developing and Evolving Architectures for Complex Systems of Systems, Philip J. Boxer and Suzanne Garcia-Miller

Software architects are increasingly being asked to address how their architectural representations relate not only to those of systems (of systems) engineers, but also to the views commonly found in DODAF (Department of Defense Architecture Framework) or other enterprise architecture frameworks. In many cases, these requests made to software architects are part of trying to understand how one software system is likely to interoperate with others that are either inside or outside of the enterprise. Understanding some of the limitations of the Zachman framework and DODAF 2.0 in understanding both software architectures and interoperability in complex systems of systems should make it easier for software architects to place their architectures in relation to these other common frameworks.

This presentation describes proposed modifications to the Zachman framework that are required to account for the needs for cross-enterprise collaboration and for accommodating new user needs at a rapid pace. The presentation also highlights a set of modeling elements that are commonly found in multi-enterprise situations. These modeling elements are illustrated in reference to DODAF 2.0 entities to emphasize what is currently missing. The presentation concludes with an example from a modeling approach that addresses these gaps, which is used at the SEI to describe not only the social and technical aspects of systems (including software systems), but also their relationship to the changing demands (especially user needs) placed upon them.

This presentation describes proposed modifications to the Zachman framework that are required to account for the needs for cross-enterprise collaboration and for accommodating new user needs at a rapid pace. The presentation also highlights a set of modeling elements that are commonly found in multi-enterprise situations. These modeling elements are illustrated in reference to DODAF 2.0 entities to emphasize what is currently missing. The presentation concludes with an example from a modeling approach that addresses these gaps, which is used at the SEI to describe not only the social and technical aspects of systems (including software systems), but also their relationship to the changing demands (especially user needs) placed upon them.





Software Architecture Beyond Blueprints, Issac Eldo

Every software has an architecture; hence every organization dealing with the software life cycle is dealing with architecture in some form or other. In the case of software, a fitting architecture would definitely increase the potential for its success; likewise, a well- aligned Software Architecture Organizational Entity (SAOE, consisting of architecture, architect(s), and architecture and design process) will have a great impact on the organizational success. To be effective SAOE must support and be supported by the ecosystem around it, which consists of four facets of software development—business, engineering, management and organization, each with different expectations. A lack of this alignment will 1) undermine the value and ROI of architecture, and 2) reduce the quality and effectiveness of the architecture itself. A good architecture starts with a well- positioned SAOE, which must be seeded by the realization of the fact that “architecture is not just about the blueprints.”




The Impact of Conducting ATAM Evaluations on Army Programs, John K. Bergey, Stephen Blanchette, Jr., Mark H. Klein, and Robert Nord

The Army Strategic Software Improvement Program (ASSIP) is a multiyear effort targeted at dramatically improving the way in which the Army acquires software-intensive systems.  It is predicated on the idea that better acquisition practices will lead to better systems and overall results. During the past five years, the Army has funded a number of programs, in conjunction with the Software Engineering Institute, to conduct software architecture evaluations using the Architecture Tradeoff Analysis Method® (ATAM®).  During the same period, several other programs funded their own ATAM evaluations.  With the Army taking a leadership role among the U.S. armed services in applying architecture practices to its programs, it seemed natural to inquire about the value received.

This presentation describes the results of a study of the impact of software architecture practices conducted with Army programs.  All twelve programs that employed the ATAM responded to a questionnaire that addressed the impact of conducting the ATAM, follow-on ATAM activities, adoption of the ATAM as part of program practices, and the overall value of the engagement.

The majority of respondents to the survey indicated they received some measure of value from the ATAM evaluation conducted for their programs. Exact quantification of that value in terms of dollars is difficult because the focus of the ATAM is risk reduction, which is a cost avoidance. However, by gauging respondents' views and actions throughout the ATAM engagement, a picture of overall estimated value begins to emerge. The presenters will show how the findings benefit the broader software
architecture community and will provide their recommendations for collecting data and assessing impact for future ATAMs conducted in any context.





The Role and Development of an Enterprise Architect: A Devil's Advocate’s Perspective, Robert Ellinger 

System engineering, system architecture, and enterprise architecture skills are necessary in one half of the processes of any successful development or implementation program. This half identifies the customer's requirements, creates the tests that verify the components and assemblies perform the functions they are supposed to perform, and validates that the system or product meets the customer's requirements. The problem is that the talents required for system engineering and system architecture are little understood.

This presentation will posit that systems engineers must serve apprenticeships in two or more IT skills (such as database design, network implementation, or software development) and serve as a requirements manager before he or she can become an independent systems engineer. The systems engineer must serve two or more years in that position before serving an apprenticeship as a system architect. Finally, for a systems architect to become an enterprise architect, he or she must be mentored by a successful enterprise architect.

The paper will identify the purpose and skills required by each position, the reasons for the described growth path, and the reasons why the coaching and mentoring is needed rather than simply training programs.
