Modeling and simulation (M&S) plays a pivotal role in the development of modern systems. Ranging in application from Formula One race cars to advanced weapons systems, models across ranges of domains, fidelity, time, and spatial scales are leveraged to make decisions on designs, material specifications, and requirements, which ultimately impact system performance. Model-Based Systems Engineering (MBSE) is experiencing a resurgence as system developers leverage the Systems Modeling Language (SysML). This article highlights the development of an MBSE-based architecture—the Joint Capability as a Service (JCaaS)—and a test-driven and -designed feature set—Full-Spectrum Survivability Tools (FSST)—that can enable system stakeholders to bring domain-specific models, test data, and system requirements into a federated model that can inform future test events and design criteria.
The Challenge
When designing or refining a complex system, stakeholders are often interested in how one change can have cascading effects on system performance under both ideal and worst-case conditions. With respect to full-spectrum survivability, interoperability between engineering models and tools is a primary challenge, in particular the chaining together of models from various domains (e.g., kinetic, cyber, and the electromagnetic spectrum) in sequence or synergistically to approach an initial analysis case. Different analysts, services, and systems are comfortable with their own specific sets of tools, and often models may only be available under certain classification restrictions. To solve this full-spectrum problem, domain-specific models must be able to exchange data without knowing about one another. Each model also needs the capability to collect all upstream changes from any number of previous models without knowing anything about the preceding models. Because each domain effector can potentially change the vulnerability topology of the SUT, there must be a way to track that topology as we expose it to other models.
Hypothesis
MBSE and multi-fidelity, multi-scale models that have been through a verification, validation, and accreditation (VV&A) process can inform test scenarios and refinements in the design and deployment of a system. There are two common approaches to facilitating communication and integration among models of various fidelities and time/space scales. The first involves developing all models in the same underlying technology stack, where each model must be directly integrated with the others. This approach quickly becomes problematic when models are developed by different teams and are not continuously updated to accommodate changes to other models (also violating our assumption that models should not know specifics of another model). Likewise, as the list of models grows, it becomes exponentially more difficult to maintain adapters among models.
Another approach, examined herein (and illustrated in Figure 1), is to develop an architecture that uses well-defined application programming interfaces (APIs) and a common intermediary language so models can be updated and added as long as they leverage the same underlying architecture. This common language ensures model developers are aware of ontologies and schemas used by the underlying architecture to support adapting their models for use in the overall system. To solve this challenge, our team has leveraged SysML 2.0 as the backbone of the architecture. SysML 2.0 can model many phenomena, including, but not limited to, structural components, behaviors, performance requirements, dependency structures, and data flows.
As illustrated in Figure 2, SysML 2.0 provides three notations, which is an improvement over the single graphical notation present in SysML 1.0. First, SysML 2.0 provides a textual notation with a metamodel based on the Kernel Modeling Language (KerML), which allows one to easily parameterize fields and perform computations over them. Second, SysML 2.0 has a graphical notation similar to SysML 1.0 that can be leveraged to make the models human-readable and intuitive to building relationships between system blocks. Lastly, and most importantly, SysML 2.0 includes a pilot implementation of a REST/HTTP API that stores the model in a database and can be queried for JavaScript Object Notation for Linked Data (JSON-LD) elements to access models in a standardized manner. This approach allows our tool to query for the data it needs to perform analysis from the SysML 2.0 Characterization.
SysML Characterization
We define the system we are examining as the System Under Test (SUT). Characterization starts by mapping the mission and performance requirements to the SysML v2 model. This model is called the SUT Structural Model. The characterization of the SUT Structural Model considers both quantitative and qualitative metrics, such as weight and flight parameters, survivability requirements, and mappings to other data. By using an MBSE approach, we can break the system down to the component level, while maintaining the relationships among those components. The components can then be tagged with domain specifics such that any current or future domains can be examined in a simulation.
As noted previously, SysML v2 enables the definition of behaviors and activities to which the SUT could be exposed based on system requirements. This model is defined as the SUT Analysis Model. The corresponding sequence or activity diagram in SysML v2 is defined in a standardized way that allows for machine readability and parameterization using any programming language. This approach provides the analyst with a method to change the sequence of an analysis, material property parameters, and the domain-specific models available in a simulation. Further, the SUT Analysis Model provides a standardized location and interface to define an API for new models brought into the system to allow them to be interoperable with other existing models.
Effects Estimation as a Service (E2aaS)
The SUT Analysis Model provides a method for setting up the sequence of domain-specific effects that will be used for analysis. Because the models do not know about each other and may not necessarily be developed in the same technology stack, we designed this framework to support any application technology stack by containerizing the model invokers, deploying model APIs, and using Argo Workflows inside a Kubernetes cluster to execute what is defined in the SUT Analysis Model (as illustrated in Figure 3).
Each effect model is considered a service to the overall analysis of the system under test. If a user is interested in only specific domains, he/she can restrict the SUT Analysis Model to those specific domains. Each domain could potentially have several models to choose from, but because the models are fundamentally interoperable on the SUT Structural Model, a user can swap out models as desired and can examine how Model A and Model B impact the system survivability compared with one another. For example, if a parameter study on fidelity is of interest, this framework would enable the user to perform analyses using models of varying fidelity.
Data Enrichment as a Service (DEaaS)
Once the SUT Structural Model is defined, it can be enriched by using various data sources. For example, when the system is in developmental, operational, or Live Fire Test and Evaluation phases, the raw data, test reports, and system changes can be pushed to various databases. This architecture can enable a notification system such that the SUT Structural Model is updated when new data are provided (as illustrated in Figure 4). This update can trigger the SUT Analysis Model to run pre-analyzed scenarios to examine how the new information impacts the survivability of the overall system. Future development can include other application containers that are specifically designed for data enrichment, processing, and analysis, which could also be also defined in the SUT activity diagram. For example, a user could define an analysis that collects information, uses machine learning or artificial intelligence algorithms to enhance or process the data, and prepares the overall model for analysis. The E2aaS and DEaaS combined make up the JCaaS [1].
Vulnerability Topologies
Under guidance from the Director, Operational Test and Evaluation (DOT&E), we are developing a feature set in JCaaS focused on modeling survivability of weapon systems and platforms. Using a mission-level model as a Universal Integrator, we explore how domain-specific survivability can be aggregated and examined from a full-spectrum viewpoint [1]. Our hypothesis is that the full-spectrum vulnerability topology of a system is some function of the domain-specific vulnerability topologies. We define a system’s vulnerability topology as the set of all vulnerabilities that can be exploited to achieve some adverse effect against that system’s primary and secondary missions. Note that by defining this set as a topology, we are stating that this space is malleable and can be transformed by some function into another topology. We believe this definition of vulnerability topologies is a key component to full-spectrum modeling and simulation and a fundamental reason why traditional fault tree analysis is likely not comprehensive enough for this work.
FSST
After defining and assuming the SUT Structural Model as the distributed authoritative source of truth, we perform mission engineering and determine effects and effectors that the SUT will be exposed to during an engagement. We then start our analysis using the E2aaS as defined in some order in the SUT Analysis Model. When the SUT is insulted by a domain, we reach out to a domain-specific model and obtain results to determine how the vulnerability topology has been updated by that effector. This approach provides a dynamic state change to the SUT that can be examined by downstream models. The next model is executed and the steps repeated until the overall sequence defined in the SUT Analysis Model is complete. Each step can be run using a Monte Carlo approach so probability distributions and uncertainty are captured along the way. The results from the analysis are then stored and provided to the analyst for examination.
As noted in the referenced Universal Integrator [1], we believe that using a Pareto Frontier at this point can help test designers and system owners determine which analysis cases should be further examined through actual testing of the system. A detailed example of this tool and process being used is provided in a separate article in this journal issue [2].
Conclusion
The FSST architecture will enable users to examine domain-specific vulnerabilities and models used in sequence and conjunction to inform full-spectrum test and evaluation. The system provides an architecture that will enable analysts to bring their own models of choice to assess the survivability of a system against various domain effectors. This article has highlighted design decisions and progress completed during the first full year of development. While there is much research to be done to test the hypotheses provided herein, initial examples run through the system have provided promising and interesting results that can serve as the foundation for future development.
About the Authors
Mr. Charles Fisher is currently the Director of the Cyber and Non-Kinetic Effects directorate at Applied Research Associates, Inc. (ARA). He has more than 15 years of experience developing vulnerability and lethality models for the kinetic and nonkinetic weapons effects communities. Mr. Fisher holds a B.S.in mathematics from Fitchburg State University and an M.S. in applied mathematics from Worcester Polytechinic Institute.
Ms. Ashley Henderson is currently the Program Manager for JCaaS and FSST and leads a diverse team of software developers and engineers at ARA. She has 8 years of experience developing software systems for the DoD. Ms. Henderson holds a B.S. in electrical engineering from Auburn University.
Dr. John Crews is currently the Intelligent Digital Environments Director at ARA. He has more than 15 years of experience developing M&S capabilities for the DoD. Dr. Crews has a B.S., M.S., and Ph.D. in mechanical engineering from North Carolina State University.
References
- Bryant, W., C. Fisher, D. Boseman, and J. Ivancik. “Digital Technology—A Universal Integrator—Enabling Full-Spectrum Survivability Evaluations.” Naval Engineers Journal, vol. 136, no. 1–2, pp189-198, March 2024.
- Bryant, W. “Using M&S to Determine Cyber Survivability: Score Small and Let the Machines Do the Math.” Aircraft Survivability, spring 2025.
