Comments on the consultant's report

[Eric Brunner-Williams <ebw@xxxxxxxxxxxxxxxxxxxx>]

First, there is a question of method.

The consultant elected to interview "a significant number of people 
about the RSSAC" at a single meeting of ICANN (-33, Cairo) and a single 
meeting of the IETF (-73, Minneapolis), in a three week period, November 
2008.
We don't know who the consultant interviewed, if the planned targets of 
the interview process were interviewed, nor the questions asked or if 
the targets knew the breadth and depth of the planned interview.
Similarly, we don't know the details of the "telephone interviews", who 
was interviewed, or from whom "feedback and comments" were received.
In a nutshell, this part of the study method is less than transparent.

The consultant also elected to review "as much of the RSSAC’s 
publicly-available written record as possible". Forensic analysis has 
value, however the consultant does not offer to have observed directly 
the RSSAC during the pendency of the study.
This is a peculiar choice, and taken as a whole, individual interviews 
and forensic document review of group interaction, of necessity it 
cannot offer the understanding of the RSSAC function as a body that 
direct observation would provide, nor can it be informed by the RSSAC 
directly and collectively on the central question: what advice on the 
operation of the root name servers is useful to the ICANN board?
Additionally, the consultant invited comment from any interested party. 
Again, the lack of specific questions to which interested parties might 
address necessarily detracts from the breadth and depth of the 
responses. The text is unclear whether the consultant received any 
response arising from the invitations for comment published on the ICANN 
website or sent to the RSSAC mailing list.
Further, the consultant "drew on their experience in governance roles in 
commercial and non-profit organizations". Unfortunately, there are few, 
if any, comparable instances of a resource such as the root zone being 
collaboratively, and correctly published, or advising the board of a 
501(c)(3) on operational issues, to draw experience from. This is not to 
say no experience in governance is applicable, but that as ICANN is sui 
generis, and RSSAC is also sui generis, comparison with generic 
institutional models necessarily looses information, substituting the 
general for the specific as that which is to be understood.
Finally, there was intentional contamination with the SSAC review. 
Having submitted a bid in response to the request for tenders for a 
reviewer of the SSAC, there is simply no way I would have risked the 
integrity and correctness of a review of the SSAC with any other review 
"in order to ensure as far as possible that there were no 
inconsistencies between our respective conclusions and recommendations."
Group think is not an acceptable alternative to correctness, and the 
point of the study is not to give aid and comfort to a momentary 
institutional elite. "Towards the end of our research, when some themes 
had emerged, interviewing several senior members of ICANN management" 
risks substituting customer satisfaction for the review mandated by 
Article IV, Section 4, Paragraph 1 of the ICANN Bylaws of a functional 
responsibility mandated by Article VII, Section 3(b) of those Bylaws.
Had the consultant published their standard question set, assuming there 
was a questionnaire designed before the interview process began, or the 
format in which interviews were conducted (terms of reference and 
questionnaire to interviewee prior to interview, or questionnaire 
withheld from interviewee until interview, and disclosed only one 
question at a time, etc.) and other method problems eliminated, some of 
the above concerns would be adequately addressed.
Second, there is the question of time and events.

The text of Article VII, Section 3(b) was written before the PSO ceased 
to exist, before a significant geographical expansion of the root 
servers outside of North America, and before anycast. To conclude, as 
the consultant does in 4.1 "The purpose the RSSAC is currently 
fulfilling is therefore at odds with the role assigned to it in the 
Bylaws" substitutes the 1998 fact situation when the JPA was first 
drafted, for the present. The consultant could just as well have 
concluded that Article VII, Section 3(b) of the ICANN Bylaws had been 
made moot by the passage of time and events, which would have been the 
more useful answer to the question posed in section 3.2, "What is 
RSSAC's role?" page 20.
This choice allows the consultant to observe, at page 34, when answering 
the slightly rhetorical question "How effective is the RSSAC?"
   "The RSSAC has not fulfilled part of the role as set out in the
   Bylaws, in that it
   has not provided advice to the Board about detailed operational
   matters such
   as operating systems for root servers. Root Server Operators see these
   issues as falling within their domain, rather than issues for ICANN
   to be
   concerned about: they question whether it is ICANN’s role even to be
   asking
   about these issues."


Note well the "has not fulfilled part of the role as set out in the 
Bylaws", which assumes no time has elapsed, no events have transpired, 
since 1998, or that the RSSAC hold the editorial pen over ICANN Bylaws, 
and simply failed to blot out the lines mooted by time and event.
This is a significant failure by the consultant.

Another area where time and event are absent from the critical framework 
of the consultant is present in section 5.1, page 36, which asks "Does 
the RSSAC have a continuing purpose? The consultant infers that "when 
the JPA was drafted, the Department of Commerce wanted to be able to 
hold ICANN to account for the performance of the root server system", 
and therefore "that ICANN is an accountability mechanism for the Root 
Server Operators," a conclusion which the consultant works his way out 
of, having spent some of the previous 10 pages on the details of RSSAC 
minute keeping and the proactive vs reactive question, towards the close 
of section 5.2 on the following page. When the JPA was drafted, RFC 2826 
had not been published, and formally, the question of root operator 
behavior had not yet been advised by the IAB.Was the DoC's primary 
interest in ICANN accounting for performance, or the root retaining its 
globally coherent property of uniqueness?
One of those two things was, and remains, much more important than the 
other. It is the Unique DNS Root, not "performance", that was the most 
likely reason for the JPA to have tasked ICANN with agreements with the 
root server operators. The alternative, that the DoC was, and remains, 
indifferent to the uniqueness of root, and simply wants some bits of it 
run in a performant fashion, is highly unlikely.
The crux of the consultant's work product is the transition between 
section 5.2. "Is the RSSAC the most suitable vehicle?", on pages 39 and 
40, and section 5.3 "The RSSAC’s purpose going forward" through the end 
of section 5.
In 5.2 consultant opens "As currently constituted, the RSSAC is not a 
suitable vehicle ..." citing overlapping membership with the Root Server 
Operators, that minutes, and therefore discussion, is not accessible to 
non-operators (I just finished writing the minutes for the IDNAbis WG 
meeting at IETF-74, and neither the meeting, nor the minutes, are 
accessible to non-specialists, so this "reason" seems to me to be a 
claim that RSSAC work should cease unless conducted in language even the 
least technical member of the ICANN Board can understand, and only about 
issues that least technical member of the ICANN board can grasp), the 
consultant couldn't find "evidence ... related to coordination", RSSAC 
is composed of volunteers, it doesn't meet at ICANN (and just what would 
the ccNSO find sufficiently compelling as to necessitate meeting with 
the RSSAC?), and the RSSAC work product isn't generally accessible via 
anonymous HTTP.
Overlaps with a technical body. Acts as a technical body, keeping 
technical minutes for specialist use. Volunteers. Avoids distractions 
and keeps strictly to _root_server_ issues, not country-code or generic 
TLD issues, let alone GAC issues or ALAC issues or ... and doesn't have 
nice web pages.
This is underwhelming as a motive to remove an independent structural 
element in the set of understandings, memoranda, contracts, and 
operational dependencies we call "ICANN".
The vision statement in 5.3 is rosy, but realistically, will "an 
ICANN-supported body" actually "provide a source of unbiased strategic 
advice to ..." anyone? Where did the consultant come up with such a 
profound misunderstanding as to venture in 5.4 "the introduction of 
IDNs" as an example of something that may have the slightest impact on 
root server performance? The scenario is fairly fantastic, given a 
decade of experience with ICANN as an institution, compare for instance 
the whois:43/80 problem, and wildly uninformed as to the real problems 
that ICANN faces "at the root", issues going back to RFC 2826 and 
uniqueness and correctness, not performance.
The consultant then considers five options, and manages to arrive at the 
conclusion that it would be useful to replace some of the current RSSAC 
members with people with no operational involvement in the root servers, 
with a sprinkling of "strategy" verbiage to sparkle things up.
I hope the Board will have the good sense to pass on the opportunity to 
change some part of ICANN that doesn't function in a political fashion, 
and make it more like the rest of ICANN.
So which four root servers are necessary and sufficient to advise ICANN 
on root server operations? Will all of them be legally US resident 
corporate entities? Which root server operators want to do nothing 
unless four non-operators are at the table, and by the way, involve 
riding circuit with the ICANN crowd, and hold meetings in public? 
Wouldn't the correct formulation be "Verisign plus three"? Why should 
ICANN encourage some root operators to participate, and discourage other 
root operators? Is incoherence the desired outcome?
I also hope the Board will keep the RSSAC separate from the SSAC In my 
opinion the SSAC has been compromised by an obsession with retail level 
cops-and-robbers problems, leaving the "security and stability" mission 
vacant, and capture of one functional box should not compromise the 
other. Even if the SSAC review is a glowing of endless wonderfulness, 
compartmentalization along functional lines is sound management of risk 
and responsibility.
As Daniel K. observes, having the IANA (L) in the mix is good, but 
casting it as neutral is problematic. Of course, Daniel thinks the 
consultant's work is excellent and I think that it is not.
Eric Brunner-Williams,
in a personal capacity

P.S. It is starkly apparent that the consultant doesn't know what 
"security through obscurity" means, or why meeting "at ICANN" is 
different from meeting "at IETFs".