CERT-SEI
SEI Podcast Series
September 11, 2014

Four Principles for Engineering Scalable, Big Data Systems

 Jeffrey Smith (Microsoft)

Ian Gorton

"The more nodes, the more hardware you have, the more software you have, the law of averages is going to dictate that things will fail. You have to handle this. The bigger your system, the more things will fail. So, failures become common. "

Categories: Software Architecture

August 28, 2014

An Appraisal of Systems Engineering: Defense v. Non-Defense

 Jeffrey Smith (Microsoft)

Joseph P. Elm

"In today's systems, it is very difficult to draw a line between where the system ends and the software begins, because the software becomes an integrating part of the system.  "

August 14, 2014

HTML5 for Mobile Apps at the Edge

 Jeffrey Smith (Microsoft)

Grace Lewis

"HTML5 is receiving a lot of attention from developers these days because one of the promises of HTML5 is portability: the fact that you write the application once in HTML5, and all you really need to run it is a browser, right? You can run it on an Android device. You can run it on an iPhone device. "

July 24, 2014

Applying Agile in the DoD: Seventh Principle

 Jeffrey Smith (Microsoft)

Suzanne Miller

 Jeffrey Smith (Microsoft)

Mary Ann Lapham

"This is a case where the cultural aspects of Agile are sometimes ahead of where some of the stakeholders are in the acquisition process. This working software principle really brings that to the forefront. "

Categories: Acquisition Support

July 10, 2014

AADL and Edgewater

 Jeffrey Smith (Microsoft)

Serban Gheorghe

 Jeffrey Smith (Microsoft)

Peter H. Feiler

"You can now create AADL components and fully characterize them in what you expect to get from them in terms of assumptions and guarantees. "

Categories: Software Architecture

June 26, 2014

Security and Wireless Emergency Alerts

 Jeffrey Smith (Microsoft)

Christopher Alberts

 Jeffrey Smith (Microsoft)

Carol Woody

"Someone can create a fake alert but make it look legitimate. There are real challenges with that in terms of the mechanisms that are controlling authenticity."

June 12, 2014

Safety and Behavior Specification Using the Architecture Analysis and Design Language

 Jeffrey Smith (Microsoft)

Julien Delange

"Having all the different aspects of your system in a single location also brings you the ability to check inconsistencies between different aspects of your system. For example, if you have a late value, this is an error, but this error can be triggered by a bad behavior specification."

Categories: Software Architecture

May 29, 2014

Applying Agile in the DoD: Sixth Principle

 Jeffrey Smith (Microsoft)

Mary Ann Lapham

 Jeffrey Smith (Microsoft)

Suzanne Miller

"The one thing I will say is that teams that don't pay attention to this do have trouble. We have seen that more than once. "

Categories: Acquisition Support

May 15, 2014

Using Quality Attributes to Improve Acquisition

 Jeffrey Smith (Microsoft)

Patrick Place

"We have these meaningless words like staffability. Just as they are meaningless in the software world, we are trying to give them meaning using scenarios. So, we have an exact analog of the architectural side of the house, the software-architecture side, to try and structure the acquisition strategy."

Categories: Acquisition Support

April 29, 2014

Best Practices for Trust in the Wireless Emergency Alerts Service

 Jeffrey Smith (Microsoft)

Robert Ellison

 Jeffrey Smith (Microsoft)

Carol Woody

"The capability itself just went live last April, and all of these alert originators are now adding this to their systems to try and understand what capability they need, and how they can integrate it with what they're already using. "

April 10, 2014

Three Variations on the V Model for System and Software Testing

 Jeffrey Smith (Microsoft)

Don Firesmith

"Historically this has been something that the safety community has known for a long time. If you have safety-critical software or hardware and you test it, then anything you use to generate and test that safety-critical software or hardware is considered safety-critical itself."

Categories: Acquisition Support

March 27, 2014

Adapting the PSP to Incorporate Verified Design by Contract

 Jeffrey Smith (Microsoft)

William Nichols

"One of the powers of design is that you don't just do a specific instance. You aren't just thinking about a specific test case. You can start to expand the test case into ranges of behavior."

Categories: TSP

March 13, 2014

AADL and Aerospace

 Jeffrey Smith (Microsoft)

Myron Hecht

 Jeffrey Smith (Microsoft)

Peter Feiler

"That constant feedback between design and analysis, which now becomes a very tightly coupled loop in a very, very rapid process, is one of the key enablers to enable us to build complex safety-critical, life-critical, and mission-critical systems."

Categories: Software Architecture

February 27, 2014

Assuring Open Source Software

 Jeffrey Smith (Microsoft)

Kathryn Ambrose-Sereno

 Jeffrey Smith (Microsoft)

Naomi Anderson

"Everyone seem to be playing in this space right now...The government certainly is looking at this for cost efficiencies. We're seeing an emergence of social platforms. The software repositories are serving as an opportunity for developers who have an interest in similar products to work on each other's software"

Categories: Software Assurance

February 13, 2014

Security Pattern Assurance through Roundtrip Engineering

 Jeffrey Smith (Microsoft)

Rick Kazman

"I can check at any point whether that architectural representation matches the stuff that has been developed, the stuff that I want to develop. That gives you control over the whole round-trip, and that's what gives you predictability."

Categories: Software Architecture

January 30, 2014

Applying Agile in the DoD: Fifth Principle

 Jeffrey Smith (Microsoft)

Mary Ann Lapham

 Jeffrey Smith (Microsoft)

Suzanne Miller

"Inside their own organization they don't have this trust either because, again, they've been developed and trained in the old system. This is a different paradigm. We know that."

Categories: Acquisition Support

January 16, 2014

Software Assurance Cases

 Jeffrey Smith (Microsoft)

Charles "Chuck" Weinstock

"The point of the assurance case and the thing that the government is understanding is that it creates an artifact that allows them or their independent assessor…to evaluate that the evidence shows the claim's been satisfied."

Categories: Software Assurance

December 26, 2013

AADL and Télécom Paris Tech

 Jeffrey Smith (Microsoft)

Etienne Borde

 Jeffrey Smith (Microsoft)

Peter Feiler

"The operating systems in safety-critical, embedded systems have very different characteristics than in standard computer systems. Of course, you can't accept that your operating system fails the same way that your home operating system could fail."

Categories: Software Architecture

December 12, 2013

From Process to Performance-Based Improvement

 Jeffrey Smith (Microsoft)

Timothy A. Chick

 Jeffrey Smith (Microsoft)

Gene Miluk

"If you are at the top of your field, your competition is always nipping at your heels. If you're not going to continuously innovate, they are going to catch up."

Categories: TSP

November 27, 2013

An Approach to Managing the Software Engineering Challenges of Big Data

 Jeffrey Smith (Microsoft)

Ian Gorton

 Jeffrey Smith (Microsoft)

John Klein

"Testing these systems in all of the various ways that they may fail can be very difficult. So, it puts more importance on developing an architecture that satisfies good design principles in analyzing that architecture up front."

Categories: System of Systems

November 14, 2013

Situational Awareness Mashups

 Jeffrey Smith (Microsoft)

Soumya Simanta

"Think about a mashup as a piece of software that combines data from different sources."

October 31, 2013

Applying Agile in the DoD: Fourth Principle

 Jeffrey Smith (Microsoft)

Mary Ann Lapham

 Jeffrey Smith (Microsoft)

Suzanne Miller

"Agile is an evolving learning environment. So, you have the top-level requirements. But, then, as you evolve and learn more about it, different requirements will emerge. And, you need to verify those with the actual operational end user."

Categories: Acquisition Support

October 17, 2013

Architecting Systems of the Future

 Jeffrey Smith (Microsoft)

Eric Werner

"I think as we look at the curve of technology, graphical processing units (GPU's) are only the start. People are building systems that have multiple GPUs in the same system. If you look at recent mobile phone releases, not only do you have a CPU and a GPU, but you have extra processing units—auxiliary processing units that might do special purpose stuff, for example, understanding motion sensors inside of a phone."

September 26, 2013

Acquisition Archetypes

 Jeffrey Smith (Microsoft)

William Novak

"An acquisition archetype describes a situation where an action that's being taken may appear to be sensible and…promising. At the same time, it has unintended, counterproductive effects to what was desired by that action. It might even make things worse than they were in the first place, even though it seemed to make perfect sense."

Categories: Acquisition Support

September 12, 2013

Human-in-the-Loop Autonomy

 Jeffrey Smith (Microsoft)

James Edmonson

"We focus on partial human-in-the-loop autonomy...No one wants a system that has no ability for a human to feedback into the system. You always want to have the ability to at least look into what it's deciding."

August 29, 2013

Mobile Applications for Emergency Managers

 Jeffrey Smith (Microsoft)

Adam Miller

"The moment of opportunity exists now to prevent what's going to happen in the future. We want to move further away from the reactive side of the spectrum and closer to the proactive and preventative side of the spectrum."

August 15, 2013

Applying Agile in the DoD: Third Principle

 Jeffrey Smith (Microsoft)

Mary Ann Lapham

 Jeffrey Smith (Microsoft)

Suzanne Garcia-Miller

"It has become evident within the DoD community and the government in general that we need to do software delivery sooner. So, how do we do that?"

Categories: Acquisition Support

July 25, 2013

Application Virtualization as a Strategy for Cyber Foraging

 Jeffrey Smith (Microsoft)

Grace Lewis

"Actually, we call that 'the Starbuck's scenario,' where basically you have all the resources you want. You're relaxed. You're sitting in your office, and you're offloading whatever it is. You don't need to worry about resources. At the edge, you do need to worry about that, because battery is expensive and heavy. The network is limited. You don't know if you're going to have connectivity around the clock."

July 11, 2013

Common Testing Problems: Pitfalls to Prevent and Mitigate

 Jeffrey Smith (Microsoft)

Donald Firesmith

"Testing by itself just isn't going to get the job done. Testing typically only finds 50 percent of the problems in the code. Since a lot of the problems are introduced during requirements engineering and architecting, it really makes sense to try to both prevent those problems up front and to find the problems then instead of during the typical test cycle when they're much, much more expensive to fix."

Categories: Acquisition Support

June 27, 2013

Joint Programs and Social Dilemmas

 Jeffrey Smith (Microsoft)

Bill Novak

"Social dilemmas come in many different forms with different properties, which is partly why they can be hard to fix. That's why we keep seeing them, not just in acquisition but in public policy, economics, sociology, and many other areas."

Categories: Acquisition Support

June 13, 2013

Applying Agile in the DoD: Second Principle

 Jeffrey Smith (Microsoft)

Mary Ann Lapham

 Jeffrey Smith (Microsoft)

Suzanne Miller

"One of the key things, if you're going to use Agile methods, is have enough definition up front of what you want to do, but not so much detail that you can't learn, that it can't change, because your environment changed."

Categories: Acquisition Support

May 23, 2013

Reliability Validation and Improvement Framework

 Jeffrey Smith (Microsoft)

Peter Feiler

"When you look at the problem, what we have done is identified four areas that are contributors to making a system that is heavily reliant on software of higher quality."

Categories: Software Architecture

May 09, 2013

The Business Case for Systems Engineering

 Jeffrey Smith (Microsoft)

Joseph Elm

"The purpose of this research was to develop the quantitative evidence that would convince the skeptics of the value of systems engineering."

April 18, 2013

Applying Agile in the DoD: First Principle

 Jeffrey Smith (Microsoft)

Mary Ann Lapham

 Jeffrey Smith (Microsoft)

Suzanne Miller

"One of the things that we found with DoD and federal clients is that these principles are a little bit new. Some of them feel good—they feel like they fit within the DoD culture—and some of them don't."

Categories: Acquisition Support

April 04, 2013

The Evolution of a Science Project

 Jeffrey Smith (Microsoft)

Andrew P. Moore

 Jeffrey Smith (Microsoft)

William Novak

"When the project first starts out, initially we're ticking off progress at a pretty regular basis…but what can happen as you start nearing completion—the 70, 80, 90 percent done—is that progress as measured can begin to stall out."

Categories: Acquisition Support

March 21, 2013

What's New With Version 2 of the AADL Standard?

 Jeffrey Smith (Microsoft)

Peter Feiler

"Today almost everything is software intensive. If you have a car or an aircraft, they don't work anymore if the software doesn't work."

Categories: Software Architecture

March 07, 2013

The State of the Practice of Cyber Intelligence

 Jeffrey Smith (Microsoft)

Jay McAllister

 Jeffrey Smith (Microsoft)

Troy Townsend

"I think it's becoming really important because 'cyber' transcends every aspect of business, whether you're doing human resources or business intelligence or physical security."

February 21, 2013

Technology Readiness Assessments

 Jeffrey Smith (Microsoft)

Michael Bandor

"A TRA is not a documentation review. There's a lot of planning that goes into it, six months to a year's worth of planning out front. You actually get into design details, engineering studies, &test reports. It's really a heavy-duty engineering level review."

Categories: Acquisition Support

February 07, 2013

Standards in Cloud Computing Interoperability

 Jeffrey Smith (Microsoft)

Grace Lewis

"The biggest fear is really vendor lock-in. People want to have the freedom to move from one cloud provider to another in case the relationship between them isn't working, service-level agreements aren't being met, other providers have better prices, or even if their provider goes out of business, which is not unusual in today's world. If there aren't standards, then moving between providers could be very difficult."

January 17, 2013

The Latest Developments in AADL

 Jeffrey Smith (Microsoft)

Peter Feiler

 Jeffrey Smith (Microsoft)

Julien Delange

"When people do the system-safety analysis, they are focused on the physical parts failing, and they understand that part. But the consequence of that in software today is still not very well understood."

January 03, 2013

The Fundamentals of Agile

 Jeffrey Smith (Microsoft)

Tim Chick

"Some people I've talked to, they really love agile. They love the techniques. And it's working really well for their team, for their project, but they are really having a hard time getting other projects in the organization to be just as successful as they are. That really is the key."

Categories: TSP

December 20, 2012

Software for Soldiers who use Smartphones

 Jeffrey Smith (Microsoft)

Edwin Morris

"Now imagine you're walking into a village in Afghanistan. There may be some people that you or your colleagues have made contact with before that you know are friendly. It would be very useful to know about those people. In addition, it would be very useful to know about where there are potential threats."

December 06, 2012

Architecting Service-Oriented Systems

 Jeffrey Smith (Microsoft)

Grace Lewis

"If you make an architectural decision that promotes interoperability or modifiability, this can have a negative impact on other qualities such as availability, reliability, security, or performance. Making these trade-offs is one of the hardest parts of architecting and designing any system."

November 15, 2012

The SEI Strategic Plan

 Jeffrey Smith (Microsoft)

Bill Scherlis

"The idea is to be able to develop highly capable, rapidly evolving, innovative systems, but to do so in a way where the risk of completion of projects is within the bounds of acceptability for major systems developments."

Categories: Software Assurance

November 01, 2012

Quantifying Uncertainty in Early Lifecycle Cost Estimation

 Jeffrey Smith (Microsoft)

Jim McCurley

 Jeffrey Smith (Microsoft)

Robert Stoddard

"We needed a radically new method to address the type of information that is present that early in the lifecycle."

October 18, 2012

Architecting a Financial System with TSP

 Jeffrey Smith (Microsoft)

Felix Bachmann

 Jeffrey Smith (Microsoft)

Jim McHale

"We did not have any evidence, any clear evidence, that they actually would work together. It was just driven by the necessity to help that customer."

Categories: Software Architecture, TSP

October 04, 2012

The Importance of Data Quality

 Jeffrey Smith (Microsoft)

David Zubrow

"It's always going to cost you more to fix it after the fact, and it's very hard to go back to the point of origin and correct data once it's entered the system. Now, our specific research last year was to investigate the use of some statistical techniques, primarily associated with outlier detection."

September 20, 2012

Misaligned Incentives

 Jeffrey Smith (Microsoft)

Bill Novak

"Misaligned incentives usually occur in the absence of well-designed rules that control the rewards or penalties for participants. The underlying idea is that unless the rules incentivize them to do otherwise, people and organizations both tend to act in their own self interest, which may not always be what was wanted."

Categories: Acquisition Support

September 04, 2012

How a Disciplined Process Enhances & Enables Agility

 Jeffrey Smith (Microsoft)

Bill Nichols

"The biggest problem today is software is getting bigger and bigger. The big question naturally is how do you scale? How do you make this work for larger organizations, for larger project sizes? Things that work within a small team, with people that can talk face-to-face, don't necessarily scale when you go to bigger projects. "

Categories: TSP

September 04, 2012

Agile Acquisition

 Jeffrey Smith (Microsoft)

Mary Ann Lapham

 Jeffrey Smith (Microsoft)

Suzanne Miller

"Today the Department of Defense needs to acquire software faster than it ever has in the past."

Categories: Acquisition Support

September 04, 2012

An Architecture-Focused Measurement Framework for Managing Technical Debt

 Jeffrey Smith (Microsoft)

Ipek Ozkaya

"Of course the other big question is, do we really need to go into a quantifiable aspect of debt, or is it good enough to just state in the metaphor realm? I tend to believe that if we can't measure it, we can't control it."

Categories: Software Architecture

September 04, 2012

Cloud Computing for the Battlefield

 Jeffrey Smith (Microsoft)

Grace A. Lewis

"In essence cloudlets are localized, lightweight servers, very lightweight, that are running one or more virtual machines. The idea is that soldiers can offload expensive computations from their handheld mobile devices onto these virtual machines."