Verification, Validation and Accreditation (VV&A)

IMC is one of the most experienced and recognized providers of verification, validation and accreditation (VV&A) services working in the Department of Defense (DoD) today. IMC has conducted or supported the verification, validation and or accreditation effort for several programs including:

• Joint Warfare System (JWARS);
• Synthetic Theater of War Operations Research Model (STORM);
• Port Simulation System (PORTSIM);
• Air Mobility Simulation System (AMSS);
• Flexible Asymmetric Simulation Toolkit (FAST);
• Warfighting Construction Federation (WARCON);
• Joint Effects Model (JEM); and
• Joint Operational Effects Federation (JOEF).

In addition IMC has been a very active participant in the Defense Modeling and Simulation Office's (DMSO's) VV&A research and development effort providing updates to DoD VV&A policy and guidance to the simulation community since 1998.

DMSO'S VV&A TECHNICAL SUPPORT TEAM AND TECHNICAL WORKING GROUP PARTICIPATION

Working with DMSO, IMC VV&A analysts have been part of the DMSO VV&A Technical Support Team (TST) since 1998 and have been part of the primary writing team for the updated DoD VV&A Recommended Practices Guide (RPG) Millennium Edition that is available on the DMSO website at http://vva.dmso.mil . In this role IMC has been a leader in analyzing and formulating DoD VV&A policy and revamping recommended guidance. IMC analysts have also been active members of DMSO's VV&A Technical Working Group. This group meets quarterly to bring together government, academic and industry experts in VV&A to review policy, guidance and see new VV&A processes and techniques that are proposed or in use in the simulation VV&A community. IMC's participation in the DMSO VV&A TST and TWG keeps us linked to the VV&A policy and guidance efforts of the DoD simulation community. Michael Metz of IMC is currently supporting the TST in drafting of NATO standards for simulation federation VV&A and in the Simulation Operability Standards Organization's (SISO's) product development group for a VV&A overlay to the Federation Development and Execution Process (FEDEP).

DMSO'S VV&A R&D EFFORT - PRIME CONTRACTOR

IMC's Modeling and Simulation VV&A Research and Development (R&D) effort under contract to the US Navy's NAVAIR Orlando Training Systems Division supports DMSO in meeting the requirements of the Department of Defense Modeling and Simulation Master Plan. This work demonstrates IMC's VV&A R&D capabilities and ability to manage a multi contractor team made up of VV&A experts from CACI, Boeing, Aegis TG, and Zetetix. IMC leads this team of VV&A experts in accomplishing DMSO's objectives in V&V policy research, validation research, and verification technique research.

Examples of the IMC Team's research include:

Substantive and technical interoperability research for simulation federations;

Objective validation research for simulation validation;

Demonstration of low cost verification and validation techniques using the JWARS simulation;

Independent verification and validation of the FAST Operations Other Than War simulation federation;

Human behavior validation research;

Specifications for simulation reuse development;

A proposed validation process maturity model;

Setup, conduct and reporting for the Foundations '04 VV&A symposium; and

Identification of risk factors to identify the validation processes required.

JWARS V&V EFFORT - MULTI YEAR CONTRACT

In a joint venture with BMH Associates, IMC was awarded the contract to conduct JWARS IV&V in September 1997 and continued that work through March 2003. IMC provided the Program Manager, Technical Director, two Mid-Level Analysts and administrative support for the JWARS V&V joint venture. IMC's JWARS V&V process significantly reduced JWARS development risk and has provided a body of evidence for use by future accreditors who will evaluate JWARS as a potential tool to support their analysis efforts. IMC collaborated with BMH and the JWARS V&V - T&E Working Integrated Product Team (IPT) to develop the JWARS V&V process, the JWARS V&V Plan, and to execute the V&V Plan.

In 2003 IMC began a Low Cost V&V effort for JWARS under DMSO sponsorship. This low cost V&V demonstration has provided a few validation tasks and continued to build the JWARS V&V Database that will be critical to future accreditation processes.

These projects demonstrate IMC's abilities to utilize the DMSO VV&A RPG as a basis of a tailored V&V process and to use an IPT structure to maintain active user involvement in a system development oversight process. It also demonstrates IMC's ability to follow OSD and Joint Staff VV&A guidance. Following are highlights from the various efforts related to this major effort.

STORM VV&A EFFORT

IMC is the V&V Agent for the STORM analytic simulation under development for Air Force Studies and Analyses Agency (AFSAA). This effort demonstrates IMC's performance in Analysis Support, Cooperative Analytic Efforts, and Analysis Integration. IMC's STORM V&V team has been successful in: developing a tailored V&V process for STORM; writing and coordinating a STORM VV&A Plan; focusing on the requirements for entering STORM in the Air Force Standard Analysis Toolkit (AFSAT); conducting an analysis of the existing THUNDER capabilities and the STORM user space analysis (USA) to develop a set of requirements for STORM to support Air Force theater level analysis; and, planning for STORM data development.

This work includes the THUNDER to STORM transition within AFSAA and support for the VV&A of JWARS within AFSAA. IMC is conducting verification of STORM C++ code, validation of STORM requirements, verification of STORM object-oriented design, and validation testing of the STORM code. IMC has supported the data development for large scale models and has extensive experience in developing database schemas and tools to share the data with the analysis community. By guiding AFSAA through the AFSAT process for STORM IMC will help AFSAA and other organizations have the evidence required for accreditation of STORM for specific analysis purposes.

IMC also provides AFSAA, via the STORM V&V Team, with over 60 years of experience in theater level planning, analysis, and modeling to support USAF and other DoD decision makers and the decision making process. This effort also demonstrates IMC's ability to work with AFSAA, AFSPACE, RAND, ACC and other Air Force analysis agencies using Air Force VV&A policy and guidance to facilitate the cooperative analytic effort.

PORTSIM VV&A EFFORT

PORTSIM is an automated method for timely, accurately planning of military movement of forces through seaports. PORTSIM assists planners in comparison of multiple ports and port selection, determines port throughput capability and utilization of critical resources, and animates port processes to identify problems. IMC provided PORTSIM IV&V services to the Surface Deployment and Distribution Command's Transportation Engineering Activity (SDDCTEA) in 2002 and 2003. In this work, IMC developed and executed a cost effective and innovative V&V process and subsequent plan for PORTSIM Version 5.0.

PORTSIM Version 5.0 V&V included V&V of the GUI re-design and integration of two existing applications, developed by different contractors, into a single port model software system -- a $1.3M software development effort over a one-year period. It required requirements analysis, conceptual model validation, design verification, and system testing expertise. It also required close coordination with Government management, the contractor developers, and future system users.

IMC introduced MTMCTEA to the DoD VV&A RPG processes with the implementation of an Oversight Committee to manage the development of PORTSIM 5.0 by the prime contractor. IMC reversed-engineered requirements for PORTSIM Version 5.0 and documented them in a system software functional specification. To support GUI efforts, IMC actually constructed a conceptual model of the graphical user interface (GUI) in VISIO. IMC conducted the Subject Matter Expert validation and made resulting changes of the Unified Modeling Language (UML) conceptual port cargo model. The prime contractor's system software was delivered unfinished for Government acceptance testing. IMC evaluated the delivery and supported the salvage of the GUI software by functionally testing it, documenting defects and fixing the most serious ones so that the evaluation could be completed. With IMC's efforts, the Government was able to salvage the GUI, a significant portion of the $1.3 million development effort.

PORTSIM requirements were non-existent, IMC reversed-engineered the PORTSIM user, simulation and problem domains requirements from existing artifacts. When the initial developer GUIs where rejected, IMC developed a VISIO GUI model with MTMCTEA engineers for the developer. When the developer delivered the UML drawings for port operations that lacked essential process details, IMC took control of the UML drawings and added needed process details. In order to capture software problems and lessons learned from PORTSIM 5.0, IMC created and implemented a Microsoft Access tracking tool to log and prioritize discovered software defects for future PORTSIM development.

The PORTSIM VV&A process demonstrated IMC's ability to work with Army VV&A policy and guidance and to build a body of evidence from a collection of documents and products.

Other IMC VV&A Related Activities

AMSS - IMC was the prime contractor for Air Mobility Simulation System (AMSS) V&V for the Office of the Secretary of Defense, Program Analysis & Evaluation (PA&E). AMSS is an airlift logistics model developed by OSD (PA&E) and this effort involved the development of a complete set of user requirements for assessing its validity for its intended use using a similar methodology for work IMC had previously done for JWARS, STORM and PORTSIM. Like JWARS, this process was completed using OSD and Joint Staff VV&A policy and guidance.

FAST - As a participant in the Flexible Asymmetric Simulation Toolset (FAST) IV&V IPT, IMC V&V analysts created the V&V process and wrote the V&V Plan that is now being executed by DMSO with the support of Army CAA. IMC also supported the identification and approval of accreditation criteria for FAST accreditation, development of the FAST accreditation process, and development of a draft FAST Accreditation Plan following Army VV&A policy and guidance..

WARCON - IMC was a subcontractor for VV&A support (to BMH Associates) on the Warfighting Construction (WARCON) contract in 2000 and 2001. IMC was a member of the WARCON VV&A IPT and wrote both the WARCON V&V Plan and the WARCON Accreditation Plan for this analysis federation effort. This work was done using Navy VV&A policy and guidance and NAVMSO VV&A templates for the products.