System Health Assessment

System Health Assessment David H. Collins, Christine Anderson-Cook, Aparna V. Huzurbazar Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, New Mexico

ABSTRACT Complex systems are increasingly confronted by two conflicting sets of requirements: on the one hand, demands for continuous operational readiness with high reliability and availability; on the other, the need to minimize lifecycle cost, implying reduced inspections, maintenance and logistics support. An emerging paradigm to address this challenge is prognostics and health management (PHM) 1, where measures of system health are used to determine needs for preventive and corrective maintenance, to optimize maintenance scheduling and parts stocking, and to forecast when a system will reach the end of its useful life. Two key components of PHM are a definition of “system health,” and a strategy for how it is to be measured as part of “system health assessment” (SHA). In this paper we discuss system health as a general concept, illustrate its application with examples, and describe how the use of system health metrics as part of a SHA program can facilitate PHM. KEYWORDS System health, prognostics, reliability, statistical methods, cost justification

INTRODUCTION As the current (2010) healthcare debate illustrates, even defining the word “health” in its most common context is not straightforward. Should health be taken to mean a state of well-being, happiness, and fulfillment? Or is it just the absence of any identifiable disease state? Metrics abound, ranging from posing simple questions such as “how do you feel” to complex and expensive measurement techniques for such variables as blood chemistry, bone density, and brain activity. Almost everyone can agree that good health encompasses multiple aspects and should not be characterized by just a single measure. Beyond assessing the state of a person’s health at a given point in time, we are interested in prognosis and prevention: how long will this person live? In what state of health? Capable of what level of physical and mental activity? What regimen should be followed to maximize health over time? An overarching concern is cost

1

PHM is sometimes also taken as an abbreviation for “prognostic health management.”

versus benefit: given the cost of a diagnostic test or treatment, what value does it return in better health or longer life? How do we assign a monetary value to a given state of health? Analogous questions and distinctions arise in considering the “health” of systems in general; these must be answered in an operationally meaningful way based on expectations of what functions the system is expected to perform, and at what standard of performance, in order to implement system health assessment, monitoring, and PHM. We consider PHM as a broad umbrella for management tools that encompass aspects of data collection for evaluation of the system, assessment through statistical analysis to quantify its health based on observed data, and decision-making tools that help guide current and future actions to maximize the long term health of the system. Some of the challenges of SHA and more broadly, PHM, are as follows: 1. The characteristics that collectively define health are diverse and measured in different units and on different scales. 2. Health is typically a function of the interdependencies of the different measures, and hence looking at individual measures in isolation is unlikely to give complete information about general health. 3. The connection between an individual measure and health is dependent on underlying science to establish appropriate thresholds for good and bad health. In some cases, this science may not yet exist: there is a basic understanding that suggests a relationship between the measure and health, but there is insufficient knowledge to precisely quantify it. By exploiting the knowledge that there is a connection, even if imprecisely characterized, changes in the pattern of the measure can forewarn of changes in health, even if the mechanism is not formally understood. 4. Uncertainty in the assessment of system health can be driven by gaps in the science connecting measures to health, miscalibration of the thresholds which drive changes in health, or inadequate data to precisely characterize the trends. A SHA program should strive to reduce the uncertainty from each of these categories. 5. Systems frequently interact with each other. Hence it is important to specify precisely what is included and excluded from the definition of the system, and also to identify what functions are required to be successfully performed for the system to be deemed healthy. A useful distinction, also with a parallel in medicine, is that between acute conditions with sudden onset, such as traumatic bone fracture or cardiac arrest, and chronic conditions such as hypertension or atherosclerosis, which progress slowly over time and may eventually lead to acute crises. A distinction in many contexts which has no direct analogy in medicine is that between repairable and non-repairable items—e.g., a light bulb is discarded when it fails, whereas a flashlight with a burned-out bulb can be repaired by replacing the bulb. (Procedures such as hip and knee replacement offer a partial analogy in medicine.) Similarly, medical testing

is always intended to be “non-destructive,” whereas in other contexts destructive testing must be used to evaluate the performance of the system, for instance to estimate the breaking strength of a cable or the number of defective rounds in a sample of rifle cartridges. In non-destructive testing a particular aspect of the health of a unit can be assessed directly from the condition of that unit; in destructive testing, it must be inferred from test results on other units. Any health assessment procedure with claims to precision requires quantitative metrics— procedures and units of measure for specific quantities with known or hypothesized relationships to “health,” however defined. Examples range from blood pressure and pulse rate in a medical setting, to failure probability, fatigue crack length, and usage hours in a military or industrial reliability context. Measurements have unavoidable errors and associated uncertainty, and the relationships are often modeled empirically using statistical methods. Therefore, health metrics should also include quantifications of uncertainty, such as confidence or credible intervals reflecting a range of plausible values for the quantity of interest, for example a forecasted lifespan or probability of failure. Qualitative health assessments are also frequently used; however, in order to provide a meaningful basis for decision-making, such assessments must be based on more rigorous (usually quantitative) criteria. For example, if a medical doctor reports that blood chemistry is “normal,” this is based on comparing test measurements to numerically-specified standard ranges. As another example, highway bridges in the U.S. may be classified as “structurally deficient,” which is qualitative but backed up by a detailed quantitative condition description, prepared by a trained bridge inspector according to rating guidelines developed by the Federal Highway Administration (Dunker and Rabbat, 1990). These quantitative underpinnings are critical to remove subjective interpretation as the primary driver of the assessment, and also to allow for more precise monitoring of trends and patterns. As the science relating the measure and health evolves, the monitoring can be updated and made more precise. For example, as medical research has evolved, healthy ranges for blood pressure have changed. Similarly, how cholesterol assessment is done has evolved from reporting total cholesterol (the sum of bad (LDL) and good (HDL) cholesterol), to the LDL/HDL ratio, and most recently to monitoring the measures separately.

Examples As a prelude to a general definition of “system health,” we present some illustrative examples of system health and health assessment from various fields: automotive, structural engineering, aviation, and munitions. These also serve to show how varied and complex health assessment can be.

Automotive health assessment Consider a common system—the automobile. It is used for “missions” such as commuting to work, shopping, and family vacations. Given the impact of unplanned failure, most people invest time and money into assessing the health of a personal automobile, and take steps to correct problems as they arise. Automobiles are subject both to acute problems, such as a tire blowout or failure to start due to a dead battery, and to chronic conditions such as engine wear, an oil leak or a slow-leaking tire that requires frequent inflation. Addressing chronic conditions often prevents failures; e.g., finding and fixing the source of the slow loss of tire pressure may prevent a flat tire on the road. Health assessment, as typically conducted, should be specified relative to the mission. Prior to a short commute, a driver might visually inspect tires for under-inflation, check dashboard gauges and indicators, and listen for abnormal engine noise. Prior to a cross-country trip, a much more detailed assessment might be conducted—perhaps checking tires with a pressure gauge, measuring tread wear, inspecting lug nuts for tightness, etc. As automobiles become more sophisticated, more of the typical health assessment procedures are automated with sensors, e.g., for engine oil level and tire pressure; these provide immediate visual indicators, rather than relying on periodic manual inspection. A reliable automated assessment of an aspect of system health can give more timely data, be less expensive in the long term, and provide more detailed information that allows for better understanding of the pattern of change in the measure. Besides being mission-directed, health assessments and subsequent preventive or corrective maintenance may be time-based, i.e., according to a schedule, such as changing engine oil every three months or 3,000 miles, or replacing the battery when it reaches the end of its expected service life. In place of time, a surrogate measure that captures usage or exposure to harsh environments may be used, typically mileage in the case of automobiles. Frequently the time or usage schedule has been selected based on historical evidence of typical patterns of change in the measure. Time-based maintenance can be effective for a homogeneous population of systems all subject to similar usage and exposure, but may be wasteful for a diverse population. For example, automobile manufacturers commonly recommend that vehicles driven at high speeds or carrying heavy loads should have a different schedule for oil changes than vehicles used just for routine personal travel. Rule-of-thumb schedules can be expensive in one of two directions: taking maintenance action before it is needed (additional maintenance costs) or missing the opportunity for action (actual failure at an inopportune time, resulting in a more costly repair). Health assessment and maintenance may also be condition-based, e.g., based on dashboard warning indicators or gauges, engine noise, etc. In principle, assessment of the need for an oil change could be based on chemical analysis of the lubricant; though this is currently not practical for automobiles, it illustrates the possibility of reducing maintenance cost by condition

monitoring. The preferred approach, when possible, is to measure and respond to the direct measurement of the condition, since this allows for heterogeneity in the population and triggers action at an appropriate time for each system. Structural engineering Corresponding to absence of disease, one important component of general system health is the absence of defects or damage. This is particularly relevant in structural engineering, where failure is often a consequence of damage over time, damage being defined as “changes to the material and/or geometric properties of . . . systems, including changes to the boundary conditions and system connectivity, which adversely affect the system’s performance” (Farrar and Worden, 2007). Damage might be measured by distortion of structural elements, corrosion, or fatigue cracking, with metrics based on specifications of the maximum allowable change in measurable parameters. Measurement may be done periodically by inspection, or by continuous monitoring (Lynch, 2007). Direct inspection may be supplemented with nondestructive evaluation of structural material properties by such means as radiography or ultrasound (Shull, 2002). In this context, a single area of defect or damage could compromise the entire structure very directly. In addition to monitoring the condition of the structure itself, monitoring environmental factors such as temperature and loading may be valuable in predicting damage to a structure (Sohn, 2007). In the developed world, the structural engineering of buildings, bridges, etc. is a mature discipline, regulated by industry and government standards. Such structures usually have very large safety margins (ability to bear a multiple of the anticipated load) in order to allow for unforeseen loading, events such as seismic shocks, and loss of strength due to material degradation. These margins can be justified by the very large human, economic, and political costs of structural failures, such as the catastrophic collapse of the I-35W Mississippi River bridge in 2007 (Kirk and Mallett, 2007). In other areas, such as aerospace, considerations of weight and size may constrain the provision of safety margins for structural strength. This can be compensated for by more detailed standards and safety regulations, more frequent inspections or monitoring, and more expensive materials and fabrication techniques; see, e.g., (DOD, 2002), (FAA, 2009), (Mueller et al., 2009). The difference between the strategies used for buildings and aerospace serve as a useful illustration of the balance between implementation constraints, cost, health and consequence. Unanticipated failure in these situations is prohibitively expensive relative to the cost of scheduled maintenance being performed before it is strictly necessary. Aviation The U.K. Ministry of Defence defines airworthiness as “the ability of an aircraft or other airborne equipment or system to operate without significant hazard to aircrew, ground crew, passengers (where relevant) or to the general public over which such airborne systems are flown”

(MOD, 2006). This typically is taken to apply over the duration of a given flight, or the interval between inspections or planned maintenance. Additional criteria for health, particularly for commercial aircraft, would include the general condition and comfort of passenger accommodations, maintenance history and trends, and predictive information on the useful life of the aircraft. The U. S. Federal Aviation Administration (FAA) defines “airworthiness” as follows (FAA, 2009): •

The aircraft must conform to its TC [type certificate]. Conformity to type design is considered attained when the aircraft configuration and the components installed are consistent with the drawings, specifications, and other data that are part of the TC.

•

The aircraft must be in a condition for safe operation. This refers to the condition of the aircraft relative to wear and deterioration, for example, skin corrosion, window delamination/crazing, fluid leaks, and tire wear.

These definitions, of course, are far too general for any particular situation. Modern aircraft are tremendously complex (FAA, 2008), and the actual “health check” mandated for commercial carriers by the FAA may be based on thousands of pages of procedural and technical specifications, which differ based on the particular type of aircraft. The tolerance for failure in this application is extremely low, and also includes substantial margin in order to cope safely with unforeseen stresses and exposures. Munitions Small arms ammunition furnishes an example where it is not possible to fully test a unit without destroying it. A small-arms cartridge is itself a somewhat complex system, consisting of casing, primer, projectile, and propellant, the latter a chemical mixture including one or more base propellants, a deterrent to regulate burning, and a stabilizer to inhibit chemical decomposition; predicting cartridge malfunctions is complex, and becomes even more so when considered in the context of a complete weapon system (Goldstein, 1979, Hirlinger and Kukuck, 2007, U.S. Army, 2007, Vogelsanger et al., 2001). From the user’s point of view, assessment of the health of ammunition is based on acceptably low safety risk, low failure rate (jams, misfires, excessive bore pressure, etc.), and ability to function as specified (muzzle velocity, accuracy, etc.) over its expected life. Larger munitions have many of the same attributes, but with increasing complexity, more redundancy to protect the system from individual part failure, and more complex mission requirements. There are typically many mission profiles, and careful consideration should be given to how system health is calibrated to the requirements of various missions.

Aspects of health can be assessed prior to use by visual inspection for damage or corrosion, testfiring according to a sampling plan, analysis of prior field reports of failures, and by modeling: since the chemical decomposition of propellants is fairly well-understood, the age and history of storage conditions such as temperature enables prediction of the probable reliable life of the ammunition. All of these methods are used in conjunction for complete health assessment. Typically, the interdependence of the different component measurements is an important consideration, as a degraded but non-failing state in several components could lead to a system failure.

SYSTEM HEALTH AND SYSTEM RELIABILITY The standard definition of “reliability” is “the ability of an item to perform a required function, under given environmental and operational conditions and for a stated period of time” (ISO, 1986). Reliability is measured as a probability: given a mission definition, including duration and environmental conditions, reliability is the probability that a system will complete the mission without failure. Reliability can be estimated by testing the system multiple times under mission conditions and measuring the proportion of successes, or by developing a model for the system reliability function R(t), which gives the probability that a system started at time zero has not failed at time t. Other metrics for reliability also exist, such as mean time between failures (MTBF), which is more appropriate for a system that is operated continuously rather than for intermittent missions. Reliability is a necessary condition for system health, but is not a complete characterization of health. All real systems are subject to various forms of degradation over time, from such causes as corrosion and metal fatigue; in addition, the ability to procure “good as new” spare parts and trained maintenance personnel typically declines over time. Thus a single reliability estimate at a fixed point in time only tells part of the story. To be considered healthy, we propose that a system must have other characteristics as well: •

The stated reliability should be a lower bound on reliability over the period (perhaps years) when the system is expected to be operational, or should be accompanied by an estimate of how reliability will change over time. This implies that causes and preventive and corrective actions for degradation are well-understood. Among other things, the effect of environmental covariates such as temperature and humidity on reliability should be taken into account.

•

For repairable systems, the anticipated future cost and difficulty of maintenance must be understood; this includes understanding degradation issues for stocked spare parts. A healthy system should have an acceptable anticipated availability (based on time to

perform corrective and preventive maintenance) and maintenance cost for its intended lifetime. •

Every system will eventually need to be retired or replaced; a health assessment should include an estimate with appropriate uncertainty of the system’s remaining useful life.

•

Just as healthy people get periodic medical checkups, a plan should exist for monitoring or periodic testing to verify the ongoing health of a system. Lack of periodic verification leads to an ever-larger margin of uncertainty regarding the actual reliability (Vander Wiel et al., 2009).

A GENERAL MODEL FOR SYSTEM HEALTH ASSESSMENT The examples show that the exact definition of health, and the details of health assessment, are context-specific. Nevertheless, from analyzing many such examples we can extract a general structure for health assessment. It includes, for any given system, a definition of health, metrics that determine whether characteristics of the definition are satisfied, scientific or statistical models to connect the metrics to the overall assessment of health, and a procedure for using the analysis of the observed data for all metrics to provide an overall indication of health. Health is characterized by •

Design specifications providing adequate safety margins for the intended use, taking into account uncertainties in measurements of system integrity, load levels and environmental conditions.

•

Absence of observable damage or material changes beyond specified levels.

•

Operating performance measures within specified ranges.

•

Predicted reliability within a specified range.

•

No predicted damage or degradation that would compromise system integrity, reliability or safety within a specified time period.

Terms like “specified levels” and “specified time period,” of course, must be defined for each type of system, and may be based on usage criteria as well. Corresponding to characteristics such as “observable damage” and “degradation,” metrics are required which specify how they are to be measured. Paraphrasing Sharp and Wood-Schultz (2003), the metrics of interest are measurable high-level indicators of some significant aspect of a system’s operation. A complete set of metrics is one that, taken as a whole, is sensitive to all aspects of the system that are important to its functioning safely, reliably, and to specifications. While it is ideal to have a

scientific or engineering basis for connecting the metrics to system health, it is often the case that the development of this basis may be difficult for some potential or observed failure modes. In these cases, monitoring of trends and patterns in the observed data for a given metric may indicate an increased potential for deterioration of system health, even if the mechanism for that change is not fully understood. Health is assessed by •

Periodic inspection or continuous monitoring, verifying continued absence of observable damage or material changes beyond specified levels.

•

Regular monitoring of loads, usage, and environmental conditions to verify that design points are not exceeded. For systems that are not in continuous operation, this means regular assessment of anticipated operating conditions and expected performance levels.

•

Use of physical and statistical models, based on measurements of material properties and damage, to verify that the conservatively estimated useful life will not be exceeded within a specified time period. In particular, this may require models for predicting system reliability and reliability trends based on component reliabilities.

•

Verifying the performance of scheduled or condition-based preventive maintenance, and correction of observed damage beyond specified levels.

•

Monitoring of trends in the metrics relative to known or estimated tolerance limits. Typically it is helpful to project the trend with an appropriate associated uncertainty beyond the range of the observed data, in order to provide insight into the pattern that might be expected in the range of future time that is considered of interest.

The output of a health assessment procedure is some overall indication of health, which in turn aids decision-makers in determining the risk level associated with the system in its current state, and what action needs to be taken if the risk is excessive. Ideally, one might wish for a single number characterizing the overall state of health, or perhaps a binary indicator of “healthy” versus “unhealthy.” These simplistic characterizations are likely to hide key information about areas of potential concern. Rarely if ever are systems so simple that they can be characterized in this one-dimensional fashion. Even where a single indicator of current health can be given, this leaves out other important indicators such as the direction of health trends, prognosis for problems found, potential cost of maintaining or improving the state of health, and so forth. SHA should be a rich summary of overall health that groups some common characteristics together, while separating different categories of health. It is much more informative to know that a particular sub-system is unhealthy, than just to be told that there is a problem with the overall system.

Various methods exist for displaying multidimensional assessments of overall health in a form that is easy to grasp and facilitates comparison. One example is the star plot, shown for automotive health in Figure 1. Measures of health for subsystems of a motor vehicle are first scaled to a common range (e.g., if the common range is 0 to 10 and a ≤ x ≤ b, then x′ = 10(x − a)/(b − a) is the scaled measure); the measures are then shown on radial axes. Connecting the values on the axes produces an overall shape that, in this case, makes it easy to judge the overall health of a particular vehicle relative to the “perfect” vehicle, which is characterized by a star of maximum area. This plot provides a simple yet multidimensional summary of the major categories of health, with a visual display representing the overall health of the system. The summary must be tailored to a particular system based on major categories of failures, subsystems or data.

The perfect vehicle

Your vehicle's health

Engine 10

Engine 10

8

8 Maintenance trend

6 4

Transmission

6

Maintenance trend

4

2

2

0

0

Body and interior

Brakes/tires

Steering/suspension

Body and interior

Transmission

Brakes/tires

Steering/suspension

Figure 1 Star plots of overall health for a motor vehicle

Other methods exist for presenting this kind of multivariate data (Chambers et al., 1983); displays should be chosen based on the needs of a particular audience.

SYSTEM HEALTH ASSESSMENT AND PHM Along with growing awareness that any system has an associated health and that there are advantages to assessing and understanding health proactively, the nature of system tests has begun to evolve. Historically, full systems tests were the focus, with the simple goals of quantifying and tracking reliability. For example, computer software reliability has typically been assessed by full system testing; full systems tests have also been widely applied to military missiles, where the testing is destructive. A second level of evaluation, called “health monitoring,” then emerged with component, subsystem and system level tests being jointly

considered to monitor the system, with a broader definition of health. The third and most comprehensive phase, which has only recently been considered for many complex systems, is Prognostics and Health Management (PHM), which seeks to monitor and react to changes in system health. This phase seeks to exploit understanding of systems to guide decision-making and management of the population of systems, in order to maximize performance while minimizing maintenance and management costs. PHM is the process of obtaining and using measurements of system health to predict the condition of a component or system at a point in the future. The implication is that predictions are used to help manage maintenance, operations and logistics; the goal of PHM is improved reliability and reduced lifecycle costs through effective assessment and strategic management of assets. In addition to measurements, the prognostic component uses physical and statistical predictive models to extrapolate trends beyond their observed ranges with appropriate measures of uncertainty. The management component connects prognostics to processes for maintenance scheduling, logistics, etc. Clearly, key components of PHM include a definition of “system health,” definitions of metrics to specify how it is measured, and strategies to combine the metrics into a unified assessment. PHM has both associated costs and benefits. Costs may include installation of sensors, testing (with higher costs for destructive testing), other forms of surveillance, and development of predictive models. Benefits of a well-executed PHM program include reduced downtime, replacing unscheduled with scheduled maintenance, fewer mission failures, and reduced inventory of spares. Considerable research has been done on cost-benefit analysis of PHM, and methodology and case studies are available in the literature; see, e.g., (Banks et al., 2005), (Feldman et al., 2009), (Hines et al., 2009), (Novis and Powrie, 2006), (Scanff et al., 2007). The cost-benefit analysis, as well as the overall PHM process, can be synergistic with tools currently used for system-level reliability analysis, such as FMECA (Failure Mode, Effects, and Criticality Analysis).

STATISTICAL METHODS IN SYSTEM HEALTH ASSESSMENT Health assessment can be viewed as inference about the present and future states of a system based on a limited set of measurements. This requires models, representations of components or failure modes of the system defining quantitative relationships between system states and measurement data. All models exhibit uncertainties, such as measurement error, sampling variation from selecting and measuring just a subset of the population, and uncertainty regarding the connection between the chosen model and the true relationship between states and measurements. Clearly, it is important to know how much confidence can be placed in the accuracy of predictions (how close they are on average to true values) and in their precision (how

widely spread out they are). Thus quantifying the uncertainties associated with health assessment is critical to sound decision-making. Quantification of uncertainty and natural variation are main focuses of the discipline of statistics, and statistical methods systematically account for the uncertainties mentioned above. Unlike ad hoc methods of dealing with uncertainty, formal statistical methods can be subjected to rigorous analysis to determine their suitability for dealing with a given set of measurement data, and to characterize the uncertainty associated with estimates and predictions. They also provide an objective approach for making statements about the quality of measurement systems, reproducibility and uncertainty. Complex systems typically provide very heterogeneous environments for statistical data collection and analysis: •

Extensive unit test data may be available for some, but not all, components.

•

Computer simulations for some component and system failure modes may exist, based on scientific understanding of processes leading to failure.

•

In some cases, due to very high reliability requirements or constraints on testing, the best available information may be expert opinion based on the experience of scientists and engineers.

•

Tests of the full system under mission conditions are usually expensive and difficult to perform, and thus data on system-level performance may be scarce or nonexistent.

•

Data types are also heterogeneous, and include pass/fail (the unit either worked or failed), lifetime (time until failure when the unit is operated continuously), and degradation (continuous measures of condition relative to stipulated or anticipated operational limits).

•

Data will exhibit differing levels of uncertainty, depending on the source and quantity of data.

•

Costs of obtaining different types of data vary widely, leading to issues of costeffectiveness and resource allocation.

Situations with multiple sources and types of data present many challenges to statistical methodology (Anderson-Cook, 2009). Among the most important are how to aggregate heterogeneous data in order to estimate health or reliability at the system level, and how to appropriately propagate uncertainty from multiple types of data in a single analysis. Significant progress has been made toward meeting these challenges, and research is ongoing in various application areas; see (Wilson et al., 2006), (Graves et al., 2007, 2008), (Anderson-Cook et al., 2008), (Huzurbazar et al., 2009), (Lorio et al., 2009) for details and examples.

SUMMARY AND CONCLUSIONS “Health” is not a simple concept, and considerable analysis is required to define it in the context of a given system. We have provided a generic framework for health assessment metrics based on specifications for safe and reliable operation, absence of damage and degradation, and predictions of future conditions. In a specific context, selection of appropriate individual metrics to capture the important and driving mechanisms that affect performance of the system for identified tasks and missions is of central importance to a complete and representative assessment. These metrics may define measurement of environmental variables as well as component conditions. Health assessment also requires physical and statistical models for predicting system reliability and reliability trends beyond the range of observed data, based on component conditions. System health metrics include traditional metrics for reliability, but extend beyond them to include maintenance trends, future costs, remaining life, and plans for ongoing verification of reliability, safety and surety. System health assessment is a foundational component of Prognostics and Health Management (PHM), which includes data collection for evaluation of systems, using health measurements to predict the future condition of components and systems, and providing guidance about effective and efficient management of maintenance, operations and logistics. Ultimately, a strategic and comprehensive PHM program can result in improved reliability, less downtime, and reduced maintenance and logistics costs. Key components of PHM include the development of a comprehensive system view of individual failure modes, the identification of appropriate metrics to represent all aspects of system health, data collection for balanced and representative health assessment, the use of physical and statistical models, and analyses to obtain statistical quantification of uncertainties. PHM can be cost-justified based on its benefits, including reduced downtime, replacing unscheduled with scheduled maintenance, fewer mission failures, and reduced inventory of spares. In addition to having a healthier population of systems, there are also inherent advantages to the improved understanding of predicted performance at current and future times. These factors provide the return on an investment in data acquisition and development of prognostic models for health. PHM can integrate existing processes and tools for assessing system-level reliability, thus allowing a graceful transition from the more limited scope of “traditional” reliability engineering processes to the broader, more encompassing approach of PHM.

ACKNOWLEDGEMENTS The authors thank Todd Graves, Mike Hamada, and Rick Picard for helpful comments and discussion. The work of the authors was performed under the auspices of the Los Alamos

National Laboratory, an affirmative action/equal opportunity employer, operated by the Los Alamos National Security, LLC, for the National Nuclear Security Administration of the U.S. Department of Energy under contract DE-AC52-06NA25396

REFERENCES Anderson-Cook, C. M. (2009). Opportunities and issues in multiple data type meta-analysis. Quality Engineering, 21: 241-253. Anderson-Cook, C. M., Graves, T., Hengartner, N., Klamann, R., Wiedla, A., Wilson, A. G., Anderson, G., Lopez, G. (2008). Reliability modeling using both system test and quality assurance data. Military Operations Research 13: 5-18. Banks, J., Reichard, K., Crow, E., Nickell, K. (2005). How engineers can conduct cost-benefit analysis for PHM systems. Proceedings of the 2005 IEEE Aerospace Conference. Chambers, J. M., Cleveland, W. S., Kleiner, B. and Tukey, P. A. (1983). Graphical Methods for Data Analysis, Pacific Grove, CA: Wadsworth & Brooks. DOD (U. S. Department of Defense) (2002). Aircraft Structural Integrity Program, General Guidelines for, MIL-HDBK-1530B (USAF). Dunker, K. F., Rabbat, B. G. (1990). “Highway bridge type and performance patterns,” Journal of Performance of Constructed Facilities 4: 161-173. FAA (U. S. Federal Aviation Administration) (2008). Joint Aircraft System/Component Code Table And Definitions. FAA (U. S. Federal Aviation Administration) (2009). Order 8130.2F, Airworthiness Certification of Aircraft and Related Products. Farrar, C., and Worden, K. (2007 ). An introduction to structural health monitoring. Philosophical Transactions of the Royal Society A 365: 303-315. Feldman, K., Jazouli, T., Sandborn, P. (2009). A methodology for determining the return on investment associated with prognostics and health management. IEEE Transactions on Reliability 58: 305-316. Goldstein, S. (1979). Interior ballistics modeling applied to small arms systems, U.S. Army Armament Research and Development Command technical report ARSCD-TR-79001. Graves, T. L., Hamada, M. S., Klamann, R., Koehler, A., Martz, H. F. (2007). A fully Bayesian approach for combining multi-level information in multi-state fault tree quantification. Reliability Engineering and System Safety 92: 1476-1483. Graves, T. L., Hamada, M. S., Klamann, R., Koehler, A., Martz, H. F. (2008). Using simultaneous higher-level and partial lower-level data in reliability assessments. Reliability Engineering and System Safety 93: 1273-1279. Hines, J., Bennett, L., Ligetti, C., Banks, J., Nestler, S. (2009 ). Cost-benefit analysis tradespace tool as a design aid for the U.S. Army Vehicle Health Management System (VHMS) program. Proceedings of the 2009 Annual Conference of the Prognostics and Health Management Society.

Hirlinger, J., and Kukuck, S. (2007). M789 LW 30mm HEDP cartridge in-bore detonation investigation. http://www.dtic.mil/ndia/2007gun_missile/GMThurAM2/HirlingerPresentation.pdf. Huzurbazar, A., Anderson-Cook, C., Wilson, A. (2009). Reliability uncertainty aggregation case study for the B-61. Technical report LA-UR 09-00172. Los Alamos, NM: Los Alamos National Laboratory. ISO (International Standards Organization) (1986). ISO 8402, Quality – vocabulary. Kirk, R. S., Mallett, W. J. (2007). Highway Bridges: Conditions and the Federal/State Role, Congressional Research Service report RL34127. Lorio, J., Dvorack, M., Mundt, M., Diegert, K., Ringland, J., Zurn, R., Anderson-Cook, C., Huzurbazar, A., Wilson, A., Fatherley, Q., (2009). Quantifying reliability uncertainty: a proof of concept. Technical report SAND2009-2173. Albuquerque, NM: Sandia National Laboratory. Lynch, J. (2007). An overview of wireless structural health monitoring for civil structures. Philosophical Transactions of the Royal Society A 365: 345-372. MOD (U. K. Ministry of Defence) (2006). Military Airworthiness Regulations. Mueller, I., Larrosa, C. , Roy, S., Mittal, A., Lonkar, K., Chang, F-K. (2009 ). An integrated health management and prognostic technology for composite airframe structures. Proceedings of the 2009 Annual Conference of the Prognostics and Health Management Society. Novis, A., Powrie, H. (2006 ). PHM sensor implementation in the real world – a status report. Proceedings of the 2006 IEEE Aerospace Conference. Scanff, E., Feldman, K., Ghelam, S., Sandborn, P., Glade, M., Foucher, B. (2007). Life cycle cost impact of using Prognostic Health Management (PHM) for helicopter avionics. Microelectronics Reliability 47: 1857-1864. Sharp, D. H., Wood-Schultz, M. M. (2003). QMU and nuclear weapons certification: What’s under the hood? Los Alamos Science 28: 47-53. Shull, P. J., editor (2002). Nondestructive Evaluation: Theory, Techniques, and Applications, New York: Marcel Dekker. Sohn, H. (2007). Effects of environmental and operational variability on structural health monitoring. Philosophical Transactions of the Royal Society A 365: 539-560. U.S. Army (2007). Test Operations Procedure (TOP) 3-2-045 Small Arms - Hand and Shoulder Weapons and Machineguns. Vander Wiel, S., Wilson, A., Graves, T., Reese, S. (2009). A random onset model for degradation of high reliability systems. Technical report LA-UR 09-04731. Los Alamos, NM: Los Alamos National Laboratory. Vogelsanger, B., Ossola, B., Schädeli, U., Antenen, D., Ryf, K. (2001). Ballistic shelf life of propellants for medium and small calibre ammunition – influence of deterrent diffusion and nitrocellulose degradation. 19th International Symposium of Ballistics, 7–11 May 2001, Interlaken, Switzerland. Wilson, A. G., Graves, T. L., Hamada, M. S., Reese, C. S. (2006 ). Advances in data combination, analysis, and collection got system reliability assessment. Statistical Science 21: 514-531.