{"id":51808,"date":"2024-11-14T00:00:00","date_gmt":"2024-11-14T00:00:00","guid":{"rendered":"https:\/\/discover.restek.com\/uncategorized\/the-comprehensive-calibration-calculator-version-2\/"},"modified":"2026-01-23T21:13:13","modified_gmt":"2026-01-23T21:13:13","slug":"the-comprehensive-calibration-calculator-version-2","status":"publish","type":"post","link":"https:\/\/discover.restek.com\/fr\/blogs\/gnbl4813\/the-comprehensive-calibration-calculator-version-2","title":{"rendered":"The Comprehensive Calibration Calculator Version 2"},"content":{"rendered":"\n

A while back I introduced a fancy calibration spreadsheet that allowed for easy comparisons of different curve fits, but mentioned that it wasn\u2019t complete. It didn\u2019t offer force through zero options, and there were some disagreements on r2<\/sup> values when compared to MSD Chemstation. While working on some completely unrelated spreadsheets I had a flash of inspiration on how to force through zero, so I rushed to try it out before my Excel muse abandoned me. You can find the new version here<\/a>. I also have access to a new chromatography software from Agilent, OpenLab CDS, so I could do some additional comparisons, and the results from that ended up being very interesting.<\/p>\n\n\n\n

Before we dig into the software comparisons, let\u2019s talk about what\u2019s new with the spreadsheet. Entering data works the same, but when you look at the ISTD and ESTD templates you\u2019ll see some additional results, as shown in Fig. 1. The linear and quadratic fits now have two sections, Ignore 0 and Force 0, each with equal, 1\/x, and 1\/x2<\/sup> weighting, so you can compare all options at a glance. There are also two calculations for %RSD on average RF fits, but we\u2019ll get into that later.<\/p>\n\n\n\n

\"Screenshot<\/figure>\n\n\n\n

Fig. 1 \u2013 Curve fit parameters<\/p>\n\n\n\n

The residual error section also shows results for all the curve fits, so you again have all results at a glance.<\/p>\n\n\n\n

\"Screenshot<\/a><\/figure>\n\n\n\n

Fig. 2 \u2013 Residual error results<\/p>\n\n\n\n

The sample results also have all 13 curve fit types, which can be a bit overwhelming. When I use the spreadsheet, I usually highlight the column of the curve fit I\u2019ve chosen as best, just to keep it clear which results to use. Don\u2019t delete the other columns, since that will break a lot of the calculations.<\/p>\n\n\n\n

\"Screenshot<\/a><\/figure>\n\n\n\n

Fig. 3 \u2013 Sample results<\/p>\n\n\n\n

I\u2019ve also added some graphs that show all the curve fits to give a visual idea of how the fits deviate from the calibration points.<\/p>\n\n\n\n

\"Screenshot<\/a><\/figure>\n\n\n\n

Fig. 4 \u2013 Graphs of curve fits<\/p>\n\n\n\n

That covers all the new features of the spreadsheet, but the most interesting thing I found was the result comparisons between it, MSD Chemstation, and OpenLab CDS. I\u2019ve added a tab called \u201cValidation\u201d that shows the results, but let\u2019s go over the key points.<\/p>\n\n\n\n

First, the %RSD calculations for average RF fits don\u2019t agree between Chemsation and OpenLab. To understand why you have to understand the difference between the standard deviation for populations and standard deviation for samples.<\/p>\n\n\n\n

\"Screenshot<\/a><\/figure>\n\n\n\n

Fig. 5 \u2013 Average RF and %RSD comparison<\/p>\n\n\n\n

The standard deviation for a population, stdev.p in Excel, uses the formula:<\/p>\n\n\n\n

\"Formula<\/figure>\n\n\n\n

where \"\" is each data point, \"\"is the average of the data set, and N is the total data size. This is the calculation to use if you\u2019re calculating the standard deviation for a complete data set. If I measure the height of everyone in a room, this would give me the standard deviation of heights of people in the room, since by measuring everyone I have a complete population.<\/p>\n\n\n\n

The standard deviation for samples, stdev.s in Excel, uses the formula<\/p>\n\n\n\n

\"Formula<\/figure>\n\n\n\n

where \"\" is each data point, \"\" is the average of the data set, and n is the number of samples. This is the calculation to use if you\u2019re measuring an incomplete data set and want to extrapolate to a greater population. If I measure the height of everyone in a single room and want to estimate the standard deviation of heights for the total building, this is the equation to use. Note that this will always give you a greater result from the same data set, since you\u2019re dividing by a smaller number.<\/p>\n\n\n\n

Going back to our results, we find that OpenLab uses stdev.p, while Chemstation uses stdev.s. A quick check of some EPA methods finds that SW-846 uses stdev.s, 1633 uses stdev.p, and TO-15A, OTM-45, and a few 500 series methods I checked don\u2019t specify, just telling you to calculate the standard deviation. As for technical reasons to use one or the other, I can see it going either way. If you consider that you\u2019re using every calibration point measured to calculate %RSD, then you could say by using all the calibration points you\u2019ve used a complete population. You could also argue that the calibration points are just a selection of points across the calibration range, so it\u2019s a subset of the infinite number of points across the curve. I\u2019ll leave that up to those more knowledgeable about statistics, but since EPA methods differ in their interpretation, and different software packages from the same company also disagree, I included both calculations so you can choose the most appropriate for your application.<\/p>\n\n\n\n

Looking at the ignore 0 curve fits, we see agreement between everything on the curve fits themselves, though Chemstation only reports 4 significant figures, so you have to take into account rounding for that. What\u2019s interesting though is the r2<\/sup> results. My spreadsheet and OpenLab both agree on all r2<\/sup> values, but MSD Chemsation gives higher values for the 1\/x and 1\/x2<\/sup> weightings, as shown in Figure 6.<\/p>\n\n\n\n

\"Screenshot<\/a><\/figure>\n\n\n\n

Fig. 6 \u2013 Ignore 0 curve fit comparison<\/p>\n\n\n\n

It gets even more interesting when we look at the force 0 curves, where there is no agreement at all, regardless of weighting, as shown in Figure 7.<\/p>\n\n\n\n

\"Screenshot<\/a><\/figure>\n\n\n\n

Fig. 7 \u2013 Force 0 curve fit comparison<\/p>\n\n\n\n

I\u2019ve mentioned before that r2<\/sup> is a bad metric to use, and the fact that 2 software platforms from the same vendor can\u2019t agree on how to calculate it really drives home how problematic it can be. I\u2019ve kept my r2<\/sup> calculation in the spreadsheet, but I strongly encourage everyone to move to either %RSE or residual error evaluations for curve fits.<\/p>\n\n\n\n

I\u2019ll note that if you look out to 8 or 9 significant figures on the curve fit coefficients my spreadsheet and OpenLab start to disagree, but that\u2019s likely just rounding differences between OpenLab and Excel, and it\u2019s so far out that it has no practical effect on the results.<\/p>\n\n\n\n

So there you have it, the now very comprehensive calibration calculator, which I hope can help people improve their calibrations and overall data quality. At this point, it does pretty much everything I want it to, so I don\u2019t anticipate any major updates unless there\u2019s strong feedback on something I\u2019m missing. <\/p>\n\n\n\n

<\/div>\n","protected":false},"excerpt":{"rendered":"

Improve your calibrations and overall data quality with the new and updated comprehensive calibration calculator spreadsheet.<\/p>\n","protected":false},"author":27,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","_kadence_starter_templates_imported_post":false,"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"footnotes":""},"categories":[9],"tags":[2660],"industries-application":[],"post-badge":[],"resource-type":[],"product-library":[],"resource-technique":[],"hf_cat_post":[623],"ppma_author":[447],"class_list":["post-51808","post","type-post","status-publish","format-standard","hentry","category-blogs","tag-calculators"],"acf":[],"taxonomy_info":{"category":[{"value":9,"label":"Blogs"}],"post_tag":[{"value":2660,"label":"Calculators"}]},"featured_image_src_large":false,"author_info":{"display_name":"Jason Hoisington","author_link":"https:\/\/discover.restek.com\/fr\/author\/jason-hoisington\/"},"comment_info":0,"category_info":[{"term_id":9,"name":"Blogs","slug":"blogs","term_group":0,"term_taxonomy_id":9,"taxonomy":"category","description":"","parent":0,"count":442,"filter":"raw","cat_ID":9,"category_count":442,"category_description":"","cat_name":"Blogs","category_nicename":"blogs","category_parent":0}],"tag_info":[{"term_id":2660,"name":"Calculators","slug":"calculators","term_group":0,"term_taxonomy_id":2660,"taxonomy":"post_tag","description":"","parent":0,"count":6,"filter":"raw"}],"authors":[{"term_id":447,"user_id":27,"is_guest":0,"slug":"jason-hoisington","display_name":"Jason Hoisington","avatar_url":{"url":"https:\/\/discover.restek.com\/wp-content\/uploads\/people-hoisington-jason-01.jpg","url2x":"https:\/\/discover.restek.com\/wp-content\/uploads\/people-hoisington-jason-01.jpg"},"0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/posts\/51808","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/users\/27"}],"replies":[{"embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/comments?post=51808"}],"version-history":[{"count":5,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/posts\/51808\/revisions"}],"predecessor-version":[{"id":84364,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/posts\/51808\/revisions\/84364"}],"wp:attachment":[{"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/media?parent=51808"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/categories?post=51808"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/tags?post=51808"},{"taxonomy":"industries-application","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/industries-application?post=51808"},{"taxonomy":"post-badge","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/post-badge?post=51808"},{"taxonomy":"resource-type","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/resource-type?post=51808"},{"taxonomy":"product-library","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/product-library?post=51808"},{"taxonomy":"resource-technique","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/resource-technique?post=51808"},{"taxonomy":"hf_cat_post","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/hf_cat_post?post=51808"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/discover.restek.com\/fr\/wp-json\/wp\/v2\/ppma_author?post=51808"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}