Measuring RSSAC047 Conformance

RSSAC047 - a document from the Root Server System Advisory Committee proposed a set of metrics to measure DNS root servers, and the DNS root server system as a whole. the document was approved in 2020, and ICANN worked on an implementation of the metrics as code, and a deployment into 20 points of measurement distributed worldwide. ISC and Verisign, two of the root server operators proposed a review of this measurement and retained SIDN Labs (who are part of the Dutch body operating .NL as a CountryCode Top-Level Domain or ccTLD) to look into how well the measurement was performing. In this episode of PING, Moritz Mullër from SIDN Labs and Duane Wessels from Verisign respectively, discuss this "measurement of the measurement" exercise, what they found out, and what it may mean for the future of metrics at the DNS Root. It's an interesting "meta conversation" about measuring things which themselves are measurements. We see this all the time in the real world, for example diagnostic imaging machines designed to measure bone density (for osteoporosis checks) require calibration, and when you want to compare a baseline over time that calibration and the specific machine become questions the clinician may want to check, assessing the results. Change machine, you get different sensitivity. So how do you line up the data? Moritz's investigations show that in some respects, the ICANN implementation of RSSAC047 was incomplete, and didn't tell an entirely accurate story about the state of the DNS Root Server System. Also, there are questions of scale and location which means a re-implementation or future improvement is worth discussing.