3/5 The need for comparative evaluation - the limits of absolute scoring
[This is the third of a series of five comments on Community Priority Evaluation] With DAGv4, the word “comparative” was entirely removed from the evaluation mechanism. The currently proposed logic for contending community-based applications is merely to determine for each application individually whether it’s claim to represent a community can be upheld. On other words, the contending applications are not compared to one another. This opens the process to an entirely avoidable danger of destructive gaming. It would also exacerbate governance problems that can naturally exist in any well-established community. It is relatively easy to define a community-based application for the purpose of winning the community-based designation. Of course this would probably not lead to good TLD proposal. But it would easily establish a rogue TLD proposal at “parity” against a well-balanced, truly community based one. With the currently proposed logic, evaluators would be powerless to stop such abuse. If the use of a scoring system (with all its undesirable side effects) cannot be avoided, it is best add an supplemental scoring mechanism to deal with contending community-based applications. This mechanism need not be designed to establish a priority in all cases, but should allow to eliminate contenders that do not deserve to be treated at par (such as cases of blatant gaming). Criteria would be designed to compare the contending community-based applicants between each other and would include: - relative relevance of the supporting community institutions- relative relevance of the community (in cases of contention between communities) - relative ability for Internet users outside of the community to understand the purpose of the TLD - the degree to which the proposed TLD’s governance model ensures accountability and transparency to the underlying community Werner Staub CORE Internet Council of Registrars |