Gaming the System: An Agent-Based Model of Estimation Strategies and their Effects on System Performance

Authored by John Meluso, Jesse Austin-Breneman

Date Published: 2018

DOI: 10.1115/1.4039494

Sponsors: No sponsors listed

Platforms: No platforms listed

Model Documentation: Other Narrative

Model Code URLs: Model code not found

Abstract

Parameter estimates in large-scale complex engineered systems (LaCES) affect system evolution, yet can be difficult and expensive to test. Systems engineering uses analytical methods to reduce uncertainty, but a growing body of work from other disciplines indicates that cognitive heuristics also affect decision-making. Results from interviews with expert aerospace practitioners suggest that engineers bias estimation strategies. Practitioners reaffirmed known system features and posited that engineers may bias estimation methods as a negotiation and resource conservation strategy. Specifically, participants reported that some systems engineers ``game the system{''} by biasing requirements to counteract subsystem estimation biases. An agent-based model (ABM) simulation which recreates these characteristics is presented. Model results suggest that system-level estimate accuracy and uncertainty depend on subsystem behavior and are not significantly affected by systems engineers' ``gaming{''} strategy.

Tags

Uncertainty Design Margins Quantification