Re: [gnso-consumercci-dt] Final Version - CTCCC Advice Leter

[Evan Leibovitch <evan@xxxxxxxxx>]

On 3 December 2012 14:06, Steve DelBianco <sdelbianco@xxxxxxxxxxxxx> wrote:

  But Evan, we did not *eliminate* measurements that would be potentially
> embarrassing.
>

Certainly, any metrics that would lay transparent the fact that closed TLDs
play by different rules from open ones have been eliminated.

Contrary to Jeff's assertion, measuring (in a yes/no objective audit)
whether certain information is relayed to the public does not constitute
new regulation or requirements. It simply is intended to indicate the
extent to which all TLDs have provided that information. That the provision
of such information is not mandatory in some instances does not mean we
ought to withhold measuring whether it's universally done. The public does
not know the difference between open and closed TLDs; heck, it still
doesn't understand that .com and .co are under completely different sets of
rules even though they are marketed as equivalents.

The main point is this: Those inside the ICANN bubble are painfully aware
of the regulatory and policy distinctions between g and cc, between closed
and open, between community and standard, thick and thin, etc. The world
outside this bubble does not know, and should not be expected to care.
Measurements of PUBLIC attitudes ought not to discriminate between
different kinds of TLDs, because the public itself does not. While other
individuals are free to disagree, I'm not sure I can be any clearer about
this PoV.



>  In this draft, we have separate metrics for registrants and for users,
> where we directly measure their "understanding" as a result of plain
> language disclosure or other ways that foster understanding of a TLD's
> restrictions, such as advertising, registrar websites, etc.
>


In previous lives I've run global skills certification programs; as such I
am quite familiar with the massive psychometric gulf between measuring the
existence of something and measuring understanding of it. You can measure
awareness of facts with  multiple choice questions; to measure
"understanding" takes essay questions or interviews (and highly skilled
people to read and grade the answers). Not only is such measurement (at any
useful level of quality) orders of magnitude more expensive than an audit,
it is incredibly biased by the nature of the questions, the translations,
size and location of the samples taken, and interpretations of the answers.

In other words.... while "measuring understanding" sounds full of good
intentions, in reality it punts the related questions into oblivion, given
the inevitable complexity and expense. Either it will never happen given
the huge expense, or it will be done so poorly as to be subject to
widespread ridicule. So I choose not to pretend that this is a viable
alternative to the original.


In my view, that's not significantly different from the draft you had
> previously agreed to.
>


As I said in my previous email, the shift from auditing to measuring
understanding -- just a few words -- is massive, and undermines the work
taken to get us this far. It is an unworkable workaround, designed to avoid
objective audits that industry refuses to do because it suspects the
results might be embarrassing.



> Help me understand your angst, Evan!
>

Did that help?

- Evan