More Than You Ever Wanted to Know About Calibrations, Part 8 – Low Level Replicates

After Part 6 of this blog, someone reached out to ask, “In our lab, it’s been typical practice for decades to perform extra replicates near the LOQ to offset the effects of variation in the RIAs (notably non-linear calibration curves). I wonder now how beneficial it would be to the calibrations you’ve presented here. Does changing from duplicate to triplicate (or from triplicate to quadruplicate) determination in the highest variation region improve the calibration performance?” My assumption was that this is basically adding more calibration points to the low end, so adding replicates would have a similar effect to simply adding more discrete points. I did a quick check that seemed to confirm it, but let’s dig into this more deeply here.

The general idea behind this seems to be that for points that may be inaccurate, running them in replicate would average them out closer to the true value. On paper this makes sense, but the potential to add more error by measuring more high-uncertainty points is also there. To check this, I did my tried-and-true method of modeling calibrations by using Excels random number generator and playing with the variance, similar to what I’ve done in previous posts. I tried both an equal spaced calibration and a calibration following the EPA method 1633 levels, with both higher variance at the low end and with equal variance at all points. A summary of the levels and variance is shown in Table I below. 

Table I: Modified Calibration Levels and Variance

To evaluate how the calibrations performed, I modeled 5 calibrations for each parameter and looked at how the % relative standard error (%RSE) compared between a calibration with single points, and one with 3 replicates at the lowest calibration point. Table II shows the results using the method spacing and the higher variance at the low end.

Table II: %RSE results for high variance calibrations with single points and replicate low points using 1633 calibration spacing.

To evaluate how the calibrations performed, I modeled 5 calibrations for each parameter and looked at how the % relative standard error (%RSE) compared between a calibration with single points, and one with 3 replicates at the lowest calibration point. Table II shows the results using the method spacing and the higher variance at the low end.

Table II: %RSE results for high variance calibrations with single points and replicate low points using 1633 calibration spacing.

The linear equal weighted curves that ignores 0 shows a slight improvement on average, with the %RSE being 8.26% lower on average when using replicates. However, the %RSE is still generally very high, with some calibrations having over 100% RSE on both types, so adding replicates won’t turn a poor calibration into a good one. The %RSE on all other types of curves is slightly better when not using replicates, but the difference is still very small. From the results above, weighting a curve with no replicates is a much better choice than adding replicates at the low end.

Table III shows the same results, but with an equal spaced curve.

Table III: %RSE results for high variance calibrations with single points and replicate low points using equal spacing.

The results are very similar, with a slight improvement on average for the linear equal weighted ignore 0 fit, but overall high %RSE that would make the fits unacceptable. The quadratic equal weighted ignore 0 fit shows significant improvement in average %RSE, but only one of the five calibrations has a %RSE result under 60%, so it’s still better to move to weighted fits instead of adding replicates.

In the interest of space, I won’t show the tables for the calibrations with 10% variance at all points, but the results match what Tables II and III show, with some improvement on %RSE on average for unweighted curves, but the best results are still weighted curves without replicates. 

While %RSE is a good overall metric, it’s not quite an apples-to-apples comparison in what I’m showing here, especially for the models with higher variance at the low end. By adding replicates to the low end I was adding more inaccurate measurements to the calculation, which could drive the %RSE higher even if the calibration fit was similar. To check this, I took 20 modeled points at the low calibration level with high variance and plugged them into the 5 replicate calibrations for each type. From there I calculated the residual error for each measurement and averaged the results across the 5 replicate calibrations to see how accurate on average each calibration type is. I also calculated the standard deviation to see if any of the calibration types were more precise. Again, in the interests of space I only show results for the high variance linear equal weighted and linear 1/x2 weighted ignore 0 fits, but the quadratic and force 0 fits followed similar trends.

Table IV: Average Accuracy and Precision at Low Calibration Level

Table IV shows that on average there is no significant difference in accuracy or precision when adding replicate points. The best improvement in accuracy and precision comes from weighting the calibration curve appropriately. The results show that my general recommendation of weighting the curve when possible is the best way to properly define the calibration curve at the low end, and that adding replicates does not add any value.

View all of the posts in the “More Than You Ever Wanted to Know About Calibrations” series.