The SEI helps advance software engineering principles and practices and serves as a national resource in software engineering, computer security, and process improvement. The SEI works closely with defense and government organizations, industry, and academia to continually improve software-intensive systems. Its core purpose is to help organizations improve their software engineering capabilities and develop or acquire the right software, defect free, within budget and on time, every time.
"We talked to a number of program managers. We talked to Agile software team members. The thing that surprised me the most was that all of the feedback that we got about these interactions was about the service side of systems engineering. It wasn't really about the transformation of artifacts. It was about how the teams engage with each other, how they communicate all across the lifecycle of the program. "
"One of the challenges in product line deployment and maintenance is managing the variation and making sure you understand whether you have deployed the correct configuration to each individual instance. So, AADL can be the home of the performance parameters, the different configuration settings that are related to each of the different environmental instances. Oh, that is very cool. "
"If you have 6 people, you put them in a room, give them access to users, and you are done with the story. If you have 20, you’ve got walls between the people, so you’ve got conversation blocks. If you’ve got 50, you’ve got multiple technologies, multiple teams, and multiple floors…If you have 100 people, you’ve got consistency problems across teams."
"We have seen teams that have devolved into just racing as fast as they can because they don't have the management top cover. We have also seen teams where the management understands that keeping that sustainable pace is part of what keeps the software coming. It is part of what makes their users happy. So, they work really hard to protect the teams in maintaining that pace. "
"One of the challenges we have is to find a good way to answer the needs of the large-scale-program-management focus without violating the kind of space that is created for a self-directed team to use Agile methods and be successful. "
"The more nodes, the more hardware you have, the more software you have, the law of averages is going to dictate that things will fail. You have to handle this. The bigger your system, the more things will fail. So, failures become common. "
"HTML5 is receiving a lot of attention from developers these days because one of the promises of HTML5 is portability: the fact that you write the application once in HTML5, and all you really need to run it is a browser, right? You can run it on an Android device. You can run it on an iPhone device. "
"This is a case where the cultural aspects of Agile are sometimes ahead of where some of the stakeholders are in the acquisition process. This working software principle really brings that to the forefront. "
"Having all the different aspects of your system in a single location also brings you the ability to check inconsistencies between different aspects of your system. For example, if you have a late value, this is an error, but this error can be triggered by a bad behavior specification."
"We have these meaningless words like staffability. Just as they are meaningless in the software world, we are trying to give them meaning using scenarios. So, we have an exact analog of the architectural side of the house, the software-architecture side, to try and structure the acquisition strategy."
"The capability itself just went live last April, and all of these alert originators are now adding this to their systems to try and understand what capability they need, and how they can integrate it with what they're already using. "
"Historically this has been something that the safety community has known for a long time. If you have safety-critical software or hardware and you test it, then anything you use to generate and test that safety-critical software or hardware is considered safety-critical itself."
"That constant feedback between design and analysis, which now becomes a very tightly coupled loop in a very, very rapid process, is one of the key enablers to enable us to build complex safety-critical, life-critical, and mission-critical systems."
"Everyone seem to be playing in this space right now...The government certainly is looking at this for cost efficiencies. We're seeing an emergence of social platforms. The software repositories are serving as an opportunity for developers who have an interest in similar products to work on each other's software"
"I can check at any point whether that architectural representation matches the stuff that has been developed, the stuff that I want to develop. That gives you control over the whole round-trip, and that's what gives you predictability."
"The point of the assurance case and the thing that the government is understanding is that it creates an artifact that allows them or their independent assessor…to evaluate that the evidence shows the claim's been satisfied."
"The operating systems in safety-critical, embedded systems have very different characteristics than in standard computer systems. Of course, you can't accept that your operating system fails the same way that your home operating system could fail."
"Testing these systems in all of the various ways that they may fail can be very difficult. So, it puts more importance on developing an architecture that satisfies good design principles in analyzing that architecture up front."
"Agile is an evolving learning environment. So, you have the top-level requirements. But, then, as you evolve and learn more about it, different requirements will emerge. And, you need to verify those with the actual operational end user."
"I think as we look at the curve of technology, graphical processing units (GPU's) are only the start. People are building systems that have multiple GPUs in the same system. If you look at recent mobile phone releases, not only do you have a CPU and a GPU, but you have extra processing units—auxiliary processing units that might do special purpose stuff, for example, understanding motion sensors inside of a phone."
"An acquisition archetype describes a situation where an action that's being taken may appear to be sensible and…promising. At the same time, it has unintended, counterproductive effects to what was desired by that action. It might even make things worse than they were in the first place, even though it seemed to make perfect sense."
"We focus on partial human-in-the-loop autonomy...No one wants a system that has no ability for a human to feedback into the system. You always want to have the ability to at least look into what it's deciding."
"The moment of opportunity exists now to prevent what's going to happen in the future. We want to move further away from the reactive side of the spectrum and closer to the proactive and preventative side of the spectrum."
"Actually, we call that 'the Starbuck's scenario,' where basically you have all the resources you want. You're relaxed. You're sitting in your office, and you're offloading whatever it is. You don't need to worry about resources. At the edge, you do need to worry about that, because battery is expensive and heavy. The network is limited. You don't know if you're going to have connectivity around the clock."
"Testing by itself just isn't going to get the job done. Testing typically only finds 50 percent of the problems in the code. Since a lot of the problems are introduced during requirements engineering and architecting, it really makes sense to try to both prevent those problems up front and to find the problems then instead of during the typical test cycle when they're much, much more expensive to fix."
"Social dilemmas come in many different forms with different properties, which is partly why they can be hard to fix. That's why we keep seeing them, not just in acquisition but in public policy, economics, sociology, and many other areas."
"One of the key things, if you're going to use Agile methods, is have enough definition up front of what you want to do, but not so much detail that you can't learn, that it can't change, because your environment changed."
"One of the things that we found with DoD and federal clients is that these principles are a little bit new. Some of them feel good—they feel like they fit within the DoD culture—and some of them don't."
"When the project first starts out, initially we're ticking off progress at a pretty regular basis…but what can happen as you start nearing completion—the 70, 80, 90 percent done—is that progress as measured can begin to stall out."
"A TRA is not a documentation review. There's a lot of planning that goes into it, six months to a year's worth of planning out front. You actually get into design details, engineering studies, &test reports. It's really a heavy-duty engineering level review."
"The biggest fear is really vendor lock-in. People want to have the freedom to move from one cloud provider to another in case the relationship between them isn't working, service-level agreements aren't being met, other providers have better prices, or even if their provider goes out of business, which is not unusual in today's world. If there aren't standards, then moving between providers could be very difficult."
"When people do the system-safety analysis, they are focused on the physical parts failing, and they understand that part. But the consequence of that in software today is still not very well understood."
"Some people I've talked to, they really love agile. They love the techniques. And it's working really well for their team, for their project, but they are really having a hard time getting other projects in the organization to be just as successful as they are. That really is the key."
"Now imagine you're walking into a village in Afghanistan. There may be some people that you or your colleagues have made contact with before that you know are friendly. It would be very useful to know about those people. In addition, it would be very useful to know about where there are potential threats."
"If you make an architectural decision that promotes interoperability or modifiability, this can have a negative impact on other qualities such as availability, reliability, security, or performance. Making these trade-offs is one of the hardest parts of architecting and designing any system."
"The idea is to be able to develop highly capable, rapidly evolving, innovative systems, but to do so in a way where the risk of completion of projects is within the bounds of acceptability for major systems developments."
"It's always going to cost you more to fix it after the fact, and it's very hard to go back to the point of origin and correct data once it's entered the system. Now, our specific research last year was to investigate the use of some statistical techniques, primarily associated with outlier detection."
"Misaligned incentives usually occur in the absence of well-designed rules that control the rewards or penalties for participants. The underlying idea is that unless the rules incentivize them to do otherwise, people and organizations both tend to act in their own self interest, which may not always be what was wanted."
"The biggest problem today is software is getting bigger and bigger. The big question naturally is how do you scale? How do you make this work for larger organizations, for larger project sizes? Things that work within a small team, with people that can talk face-to-face, don't necessarily scale when you go to bigger projects. "
"Of course the other big question is, do we really need to go into a quantifiable aspect of debt, or is it good enough to just state in the metaphor realm? I tend to believe that if we can't measure it, we can't control it."
"In essence cloudlets are localized, lightweight servers, very lightweight, that are running one or more virtual machines. The idea is that soldiers can offload expensive computations from their handheld mobile devices onto these virtual machines."