In my experience, there are two reactions when I introduce someone to systems engineering (SE) : either they have the flash of revelation, wondering how they ever worked without this way of thinking, or they see nothing but a useless pile of bureaucratic processes, only valuable for looking busy for management. I am obviously in the former category, and I think most people who have seen a project go sideways are also in this group.
The INCOSE Systems Engineering Handbook has a helpful chart which explains this phenomenon:
In this graph, we see historical project performance using two datasets. Equivalent SE effort is the fractional cost of SE processes, normalized for the quality of those processes (how quality is defined is outside the scope of this discussion). Actual/planned schedule is the ratio of the actual project duration to the originally planned project schedule.
Evidently, without systems engineering, complicated technical projects can overshoot their budget targets by up to 3x. This is inline with the rule of thumb that the project cost will vary by a factor of 3 of the initial estimate, usually chalked up to “unkown unkowns”. On the other hand, too much SE overhead slows down a project unnecessarily, which is reflected in the aversion of many engineers to byzantine bureaucratic processes which have little obvious value. What this graph shows is that not only are these positions both true, but we can know, in general, what the optimal amount of SE effort should be (and the answer is 10%-14% of total program cost, when relying on the SEROI data in the chart above).
To some, this would be the end of the discussion. The empirical conclusion is obvious. Personally, however, I have always been unsatisfied by purely empirical models, and have sought mechanistic explanations for the deeper insights they can provide. These insights, in turn, can sometimes lead to significant process improvements. Further, as an SE practitioner, it is important for me to understand the specific boundaries between “too much SE” and “not enough SE”.
What is Systems Engineering
SE is a holistic approach to the design, production, and operation (and disposal) of technical systems. It shares many responsibilities with project management, and can be roughly described as the technical aspects of project management. SE includes a number of processes to record, manage, and coordinate important information about the system in order to avoid system-level problems such as integration errors, unmet requirements, “System Accidents”, or other issues stemming from the scope or complexity of a project.
As a result, SE has a broad focus. The INCOSE SE handbook lists 30 “life cycle processes” that are considered important to system design and operation, and eight different analytical tools to manage them. The practitioner of SE is responsible for ensuring that those processes are covered (or explicitly excluded within the scope of the project). A thorough discussion of these tools and processes is outside the scope of this piece.
Not enough Systems Engineering
At one end of the spectrum, projects are realized by technical experts each solving their part of a larger problem. Each of these subsystem solutions may be well crafted, but are often unable to work harmoniously together. Good project management without structured SE is difficult and requires a project manager to intimately understand how each element works together and to guide the team towards a functional system. Unfortunately, this is not very common in practice, and is usually limited to small teams of 5-10 engineers, where informal communication can overcome many of the frictions of engineering many parts of a whole. Documentation tends to be limited to the amount required to reproduce the system (manufacturing drawings), or to some functional or performance analyses of specific elements.
When these elements are integrated together, interface errors are common, forcing re-work of elements to resolve issues. If good change management practices are not enforced, these re-works can in turn cause configuration errors (where element revisions become out of sync, eg. Rev 2 of part A was made to interface with Rev 1 of part B, which itself moved to Rev 3 in the interim).
All of this duplicative effort drives up project costs and is often unforeseeable in the absence of SE, as the conflicts are a result of the specific implementations of each of the system elements. Beyond very limited projects, the complexity of the system as a whole makes these issues inevitable.
Too much Systems Engineering
At the other end, too many processes and checks can be a brake on design work and motivation. Documentation isn’t a creative activity, and few engineers enjoy it. It takes time, and therefore has a direct cost. When the need for a process is not obvious, it also fosters resentment. In general, a good engineer is there for the challenge and the paycheck. If they feel their time is being wasted, all they’re getting is the paycheck and the will leave for a team that offers both.
There is a second, more insidious cost to overprocessing: if every decision is made by the process, there is a diffusion of responsibility. Not only will teams be less invested in their work, but they will shift more responsibility onto the process over time precisely because they are uninvested. Processes will multiply, until no one in the entire organization ever has to make a real decision as the process machine executes its inscrutable mandate.
Proper SE practice involves a process of “tailoring”, to adjust the level of SE effort to the specific needs of the organization, or even the specific project. In effect, proper SE is itself a product of SE, where the “System of Interest” (SoI in SE parlance) is the engineering organization itself.
Just Right
At the right balance, SE processes properly define and plan work packages, catch and mediate conflicts, and generally assure coherent design throughout the system lifecycle. This is done without adding undue busywork to engineering teams or causing friction. Modern Digital Engineering (DE) methods allow for robust traceability and modeling with virtually no added work, once the systems are in place to do so.
SE in practice is principally a matter of modeling the SoI sufficiently to provide traceability, configuration control, and change management to the degree necessary for the project. This can be accomplished in a traditional V structure, in an iterative design framework, or an agile development framework. Traceability makes sure that features are linked to requirements, while change management allows for design evolution as more is learned about both the problem and the implementation of the solution. Configuration control (or Versioning) maintains increments of project state so that working states can be restored and bad updates can be rolled back.
Finally, the only truly difficult part of SE is interface management. It is this practice whereby an SE practitioner segregates system elements, defines interfaces, and foresees conflicts. At Mumbri-systems, we utilize Model Based Systems Engineering (MBSE) to accomplish this task. This involves building a low to medium fidelity “functional” model to explicitly define dependencies and interactions.
The Value of Systems Engineering
SE is at its core a set of information management methods. The goals are to gather as much relevant information as early in a project as possible, to communicate that information to the right stakeholders, and to ensure that information generated throughout the project is not lost.
The value of systems engineering is in mitigating risk related to complexity of engineering projects. Complex systems do not suffer integration errors due to bad engineering at earlier project stages; rather, they are a result of two things: decisions that were made with insufficient information, and teams that should have been communicating and did not (siloing)
The former issue is an inevitable consequence of engineering solutions to unsolved problems. By definition, the details of the solution are unknown at the beginning of the project. Furthermore, later stage design decisions are for the most part either implementation details or direct consequences of decisions made much earlier in the project. Much of SE is dedicated to solving this problem through traceability, modeling, and change management. Traceability and change management has already been discussed. Through modeling, we are able to explore the consequences of decisions before they are “locked in”, allowing for detailed trade studies at the early stages of a project and eliminating as much as possible design choices which conflict when implemented in detail.
Inter-team communication is a complicated topic, but the easiest issue to address is which teams should be communicating with each other in the first place. Without modeling, teams may communicate only when it is obvious that they have shared interfaces – or when integrating their implementations reveals problems, by which point it is too late. With modeling, shared interfaces are defined explicitly, and in principle completely. For example, should two unrelated subsystems require a shared resource (eg. electrical power), it may not be obvious to each that their resource use should be communicated not only to the resource provider, but also among each other to deconflict demand. Of course, the opposite may be true: the resource provider should dictate allocations according to some rule. In either case, however, modeling these relationships allows the project manager to make proactive decisions on how these issues are resolved.
SE fundamentally provides transparency to otherwise opaque systems, at all levels and at all stages of their development. With it, leaders are empowered to make informed strategic decisions.
Leave a Reply