www.sensorsmag.com
December 2003
|
 |
|
DATA ANALYSIS
|
Test Data Anomalies
–When Tweaking’s OK
Cleaning up your data set doesn’t have to be a fraudulent maneuver. Just use methods whose validity stands up to the demands of physics and common sense.
Strether Smith, DSPCon
|
- Publish the data as they are.
- Examine the data sets and evaluate their validity. If the validity is questionable, do we annotate the anomalies or make adjustments that “improve” the data?
Most of us would reject the first option. After all, measurement errors do crop up. So we go with the second choice, which leads to more questions:
- How do we evaluate the validity of the data?
- How do we “improve” the data?
In other words, when, how, and how much can we cheat? Then we need to figure out how much we’ve improved the data, how we can determine this, and what the downsides to this exercise might be.
Errors
In most measurements we can expect to generate data that are within a few percent of the true behavior. But we don’t know what the true behavior is, which is the reason we did the test in the first place. So we need to develop criteria for deciding whether the data are correct. If they turn out to be erroneous for some reason, what can we do about the problem?
Of course, the characteristic of the data, and associated errors, is dependent on the type of test performed. There are no general rules that work with every data set, but there are some principles that can be universally applied. In this article we’ll look at these principles and describe how they apply to one type of experiment: pyrotechnic testing.
The Basic Principle
We acquire a data set, evaluate it in some way, and decide whether it is a good representation of the true behavior. How do we do that? Many of us just eyeball it to assess validity. But the underlying tool is physics. If the data show behavior that violates known physical principles, then our measurements must somehow have been corrupted. If we understand the violation in terms of its physics, we
|
Some errors can be easily characterized. In Figure 1, saturation has caused an obvious clip at about +5 units. Complete recovery is probably impossible for data this badly corrupted. But what if the saturation were not so obvious? Can we detect it, figure out its effect on our results, and make some adjustments that will improve the final outcome?
As an example, let’s examine a problem common in aerospace structural testing. Many spacecraft use pyrotechnic devices to release appendages such as solar arrays and antennas that are deployed after launch. These devices use explosives to pull a pin connection or drive some other release mechanism. While pyrotechnics are very reliable, the explosion induces high-frequency, high-level accelerations into the surrounding structure. Because of potential damage to components on the spacecraft, the shocks must be measured and characterized. A data acquisition/analysis bandwidth of 15 kHz can usually do the job, so the customary practice is to acquire the data with a bandwidth of 20 kHz (sample rate ~50 kHz). However, the actual motions may contain significant energy at higher frequencies, which leads to the problem we’ll now address.
The Physics of Pyrotechnic Measurement
Let’s take a look at what is happening in a “near-field” pyrotechnic measurement, one taken “near” the event and without dissipative (or low-pass filtering) joints between event and measurement. Figure 2 shows the time history and spectrum acquired with a broad-band (1 Msps) data acquisition system.
|
The raw data have been “adjusted” slightly to make it “ideal” and give us a baseline for comparison of error effects. When viewed as shown, the time history looks like what we would expect, but the spectral view reveals a variety of features that are of concern:
- There’s a lot of energy above 15 kHz. It is above the normal range of experimental interest (does not break things). This is called out-of-band energy.
- There’s a peak in the spectrum at ~90 kHz resonance. This matches the manufacturer’s specification for the resonance frequency of the accelerometer and is an error. What’s important is that the experiment has energy that excites the resonance and which the instrumentation system (transducer, amplifiers, filters) must “absorb,” even though it is outside our frequency range of interest.
Now, let’s look at the same data as acquired by a conventional data acquisition system. The characteristics of the system are:
- High-pass (AC coupling) filter, 2-pole Butterworth at 2 Hz
- Low-pass (anti-alias) filter, 8-pole Butterworth at 15 kHz
Figure 3 shows the effect of filtering.
|
The heavy black lines represent what the data acquisition system would see. The gray traces are the broadband motions experienced by the transducer. The effects are:
- The time history peaks have been reduced from +880/–970 g to +410/–570 g.
- The spectral plot shows that much of the high-frequency energy has been rejected. In fact, the in-band (measured) energy is only ~40% of the total.
A Data Validity Test
Shock-testing experience teaches us that it is essentially impossible to rely on visual examination of the raw time history or spectrum to detect data corruption caused by out-of-band saturation. The problem is that anti-alias filtering after the saturation rounds off the “square” corners and reduces the reported responses to below nominal saturation levels. As we will see later when we do a shock response spectrum (SRS) analysis, significant errors can be present when the raw data look fine in a straightforward examination. So more sensitive tests have been developed.
Figure 4 shows the result of integrating the measured acceleration to velocity and displacement.
|
Although there are probably some errors in the results, they are not absurd. The transducer (and the spacecraft) did not move much in the test—it is not headed out the side of the laboratory. From that standpoint, at least, basic physics is satisfied.
If there is a problem with the data, the diagnostic results may look like Figure 5: The velocity is continuously increasing, and the displacement is “headed for the moon.”
|
This is obviously not correct. Contrary to our experimental results, our spacecraft is still in the test laboratory. So what caused the problem? What can we can do to assess the effect on our results and make adjustments to correct them?
The Shock Response Spectrum
The calculation/data presentation of the SRS data set in Figure 6 is a widely accepted criterion for damage and the end result of most aerospace shock tests.
 Figure 6. The shock response spectrum (SRS) is often used as the criterion for damage resulting from shock tests. |
The analysis process calculates the peak response of a set of logarithmically spaced-in frequency, single degree-of-freedom oscillators that are driven by the input motion. The calculation is very sensitive to errors at low frequencies, where damage is likely to occur.
Corrupt Data: Is Out-of-Band Saturation the Villain?
Figure 7 shows the result of clipping of the broadband signal before filtering.
 Figure 7. Real instrumentation systems will clip the broadband data if they exceed the range of any of the individual components. |
The data emulate a measurement with the transducer and front-end electronics set for a maximum signal of 500 g, a level that is outside the range of the measured signal but less than the total broadband data range:
- The red curve is the broadband acceleration experienced by the accelerometer.
- The raw acceleration is clipped at ±500 g by the input amplifier to produce the gray data.
- The clipped data are low-pass filtered with an 8-pole Butterworth filter at 15 kHz to produce the black trace.
Note that:
- The data recorded by the data acquisition system do not come close to the ±500 g nominal limit.
- The trace “looks” fine; there is no obvious evidence that the measurement has been compromised.
- Using the tools discussed above produces the velocity, displacement, and SRS shown in Figure 8.
|
Despite the fairly radical data-clipping:
- The velocity diagnostic looks good.
- The displacement diagnostic is wandering off but is not headed for the moon.
- The SRS result (red) is not far from the correct (black) value.
So the conclusion reached here is that direct out-of-band saturation is not a significant contributor to our problem. In fact, it looks as though we can allow very significant saturation of the out-of-band signal without serious degradation of the results. We need to look further.
Slew Rate Limiting
A few years ago, investigators began to suspect that the shock data corruption was caused by the inability of the amplifiers in the signal path to track rapidly changing signals. The problem is illustrated in Figure 9, which shows the derivative of the raw (broadband) and filtered signals.
 Figure 9. If we differentiate the data, we see that the broadband slew rate (jerk, in the mechanical world) is much larger than what our low-pass filtered measurement system sees. |
The peak broadband slew rate of the acceleration signal (~280 × 106 in./s/s/s) translates to 2.8 V/ ms when a 10 mV/g accelerometer is used. If the amplifier can’t slew that fast, the signal will be compromised.
 Figure 10. The signal is clipped in slew data at levels that are well outside the range of the measurement range indication. |
For instance, if the amplifier’s slew rate capability is 1.5 V/ ms (see Figure 10), the result for acceleration, velocity, displacement, and SRS will be as shown in Figure 11.
|
This relatively small, inconspicuous error produces huge (and familiar to pyroshock testers) errors in the results. Slew rate limiting is a likely source of many of the errors experienced in shock testing.
So, Where Are We?
We have identified a physical cause that produces data diagnostic and results errors that look like those we’ve seen in practice. Although slew rate limiting is probably not the only culprit, it’s certainly a prime candidate. So, for the following discussion we’ll assume that this is the cause.
Searching For a Fix
The next step is to experiment with the data to see whether we can modify them in a way that gives us a good approximation of the truth. The strategy we will use is to:
- Use “perfect” data sets as a reference to calculate the corrected answer (SRS).
- Corrupt the perfect set by clipping the out-of-band slew rate.
- Devise “correction” methods and see if they produce reasonable agreement with the “true” SRS.
We begin by noting that the slew rate clipping produces a ramp in the velocity, caused by an offset in the acceleration data (although in this example the offset is too small to detect by eyeball).
Previous investigations [1,2] have used high-pass filtering or wavelet processing on the acceleration to remove the effects of the offset. Figure 12 shows the effect of processing the acceleration signal with an 8-pole Butterworth high-pass filter at 50 Hz.
|
The results are better, but certainly not satisfying. Nor should they be. High-pass filtering will only distort a step in the time history and, perhaps worse, it has removed the critical energy below 50 Hz.
An alternative approach is to take advantage of the fact that the saturation produces an essentially straight line in the velocity domain. View the velocity to see whether it is characterized by the straight-line characteristic shown in Figure 11. If it is:
- Perform a least-squares fit with a low-order polynomial to the deviant part of the velocity data. (A second-order polynomial is used; the second component is added to match the AC-coupling characteristics of the mesurement systm and other nonideal behaviors.)
- Differentiate the polynomial results to convert them to acceleration. (This produces a step for the linear velocity excursion.)
- Subtract the correction from the acceleration.
- Perform the SRS analysis on the result.
We are obviously trying to fool Mother Nature—the question is, how far can we push the correction? The process is not amenable to a closed-form analysis, so we’ll take an experimental approach:
- Use a “good” broadband signal (with significant out-of-band energy) as the test case. For our purposes, we’ll use the broadband shock data in Figure 1. The signal will be corrupted by slew saturation and then low-pass filtered to emulate the data as they would be seen by a real system.
- Calculate the velocity, perform the fit operation, and examine the results. If the fit is good (see Figure 13), the error model is reasonable and we can continue. If the fit is poor, the data set is not amenable to this procedure and we should try other recovery techniques.
- Once we have a good fit, we’ll need to assess the validity of the corrected result. For this exercise, we’ll see how well the SRS of the corrected version of the corrupted data set agrees with the right answer.
 Figure 13. The suggested approach is to fit a low-order polynomial to the velocity error. |
In Figure 14, showing the results of the correction on the SRS, the curves are:
- Black, the “true” maximax (solid) and minimax (dotted) SRS values
- Red, the SRS results for the corrupted data set
- Green, the “corrected” results
 Figure 14. When the polynomial fit is differentiated to acceleration and subtracted from the raw data, the resulting SRS is very close to the original (correct) value. |
As we can see, the corrected data are very close to the “true” value, so the correction technique appears to work very well for this data set despite the fact that the data are badly corrupted. The result is typical of the many examples of clipping of this data set that have been tried. When the fit to the velocity curve is good, the SRS results are also greatly improved.
The process has also been evaluated for a wide variety of analytically generated data sets, and the basic concept has been extended to curve fitting of the displacement (double-integrated acceleration) data, a technique that has been demonstrated to be advantageous in certain cases [3].
Summary
Data correction strategies can often salvage corrupted measurements, but we must carefully evaluate the validity of the correction from a physics, common sense, and usefulness standpoint. Just because the technique makes the final result look better doesn’t mean that it’s a good tactic. The experimental engineer might want to try the approach presented here, for it appears to provide better results than filtering and wavelet correction for a particular set of shock data errors.
The techniques described here and in [3] have been implemented in the DSPCon “Shockana” Shock Response Spectrum analyzer.
References
1. Smallwood, David O., and Jerome S. Cap, “Salvaging Pyrotechnic Data with Minor Overloads and Offsets”, J Institute of Environmental Sciences and Technology, Vol. 42, No. 3, pp. 27-35.
2. Smith, Strether, and Bill Hollowell, “Techniques for the Normalization of Shock Data,” Proc 62nd Shock and Vibration Symposium, Springfield VA, 1991.
3. Smith, Strether, “Shock Instrumentation Saturation Effects and Compensation,” Proc 74th Shock and Vibration Symposium, San Diego, CA. 2003.
Strether Smith is Senior Scientist, DSPCon, Bridgewater, NJ; 908-722-5656. He can be reached directly at 408-255-3650 or ss@dspcon.com.
|
|
|
|