Comment on Revised Comparative Evaluation Scoring
In our DAG v2comment on comparative evaluation scoring, we noted that ICANN stated in its Expression of Interest documentation (used to invite evaluators to do the comparative evaluations) that successful candidates are expected to exercise considerable subjective judgment. The document states that "the scoring process requires that the evaluators exercise considerable subjective judgment concerning the extent to which each community Applicant meets or fails to meet the standards defined for each of the four criteria." As long as subjectivity is the basis for evaluator conclusions, ICANN is compelled to factor in the potential of an applicant losing a point as a result of simple evaluator uncertainty in the evaluation scoring. Until all subjectivity has been removed from the process, ICANN's responsibility is to provide a fair process for applicants. Therefore, ICANN must allow one point for human error on the part of its evaluators. Otherwise, it is holding community-based applicants to a standard much higher than the standard of subjectivity detailed for the evaluators in the EOI (noted above). Returning the minimum score needed to demonstrate nexus to a margin of three points i.e., 13 of 16, maintains the rigor, but removes the risk of false negatives. We have stated this in our last comment on this topic, but it is worthy of repeating. ICANN will never be able to remove all subjectivity from the evaluation of new gTLD applications, so staff needs to understand this and factor it in to the scoring evaluation. In this way ICANN ensures that community-based applicants go through a rigorous, but fair and equitable process. Respectfully, Ron Andruff RNA Partners, Inc. Disclosure: RNA Partners intends to be an applicant for a new gTLD.