gTLD Marketplace Health Index Proposal: work needed

["Greg Aaron" <greg@xxxxxxxxxxxxxx>]

Below are notes regarding the gTLD Marketplace Health Index Proposal.  Some
of the proposed metrics (KPIs) -- and the assumptions that underlie them --
need more examination.  I offer the following as constructive comments:

 

The proposal states that "ICANN must be able to efficiently and
cost-effectively collect and analyze data underlying these KPI."  I observe
that some of the proposed KPIs are crafted around data that is currently
available. That makes them cost-effective to collect, but it does not mean
that the data is always fit for the designated purpose.  In some cases ICANN
may need to develop new sources of data.

 

I.(b) Average number of registrars offering each gTLD:  The proposal
assumption is "A larger number might indicate greater technical,
operational, legal, etc., accessibility of gTLDs by registrars."  

This assumption is flawed, because many gTLDs are not intended for wide
accessibility.  An example are the .BRAND TLDs, many of which are
deliberately accessible via one registrar only.  The KPI does not take into
account the diversity of the market and the incentives of the varied
registries and registrars.

 

I.(f) Innovation and introduction of new services:  RSEP data is not
indicative of innovation or new services.  The overwhelming number of RSEP
requests are for non-innovative and old services, like release of
two-character domains, standard IDN sets, and country & territory names.

 

I.(g) gTLD renewal rates: The proposal assumption is "A greater ratio of
renewals to deletions of second-level gTLDs might reflect greater actual use
or intent to use domains and a greater perception of those domains'
intrinsic value by registrants."  

Actual use is different from intent to use.  Neither is demonstrated by
renewal rate, and intent to use cannot be gauged reliably.  Actual usage can
be measured by more accurate and objective means, including spidering sites
to see if they have non-parking content on them.

 

II.(c) Relative incidence of ICANN breach notices issued to registries and
registrars.  The proposal assumption is that "A smaller number of breach
notices could indicate fewer noncompliant registries and registrars, and
therefore, a healthier and more trustworthy marketplace."

This KPI needs to be re-thought for the following reasons:

One, the Compliance Department decides when to send breach notices.  So
ICANN controls this KPI and the metrics.

Two, the number of breach notices does not tell us anything about how many
registries or registrars are actually out of compliance.   ICANN does not
send out a breach notice when a registrar or registry is reported to be --
or is actually even found to be -- out of compliance.  A formal, public
breach letter goes out only after ICANN makes multiple private
communications to the contracted party and the contracted party fails to fix
the problem after several attempts.

 

II.(d) Quantity and relative incidence of complaints regarding inaccurate,
invalid, or suspect Whois records.  The assumption is that "A smaller number
of Whois accuracy complaints could indicate greater reputation and trust of
gTLDs."

Use of the term "complaints" indicates that  ICANN wishes to use the data
from its Whois Inaccuracy Complaint Form. 

The number of inaccuracy complaints has no demonstrable correlation with the
public's perception.  Clearly there are many, many times more invalid WHOIS
records than are reported to ICANN.  Also, the language used here is
confusing -- what is the difference between "inaccurate" and "invalid," and
what is considered "suspect"?

 

Instead, statistics should be provided by the WHOIS Accuracy Reporting
System (ARS) that ICANN has been building
(http://whois.icann.org/en/whoisars).  ARS is designed to objectively
measure the incidence of inaccuracy across and within TLDs, and is based on
clear definitions of what is "inaccurate."

 

III. (b) Total number of unique phishing reports, as measured by
Anti-Phishing Working Group reports.  I am the editor of the APWG's
quarterly trends reports, and am the co-author of the APWG's semi-annual
Global Phishing Survey Reports.  The APWG's quarterly trends reports do not
distinguish between phishing on gTLD domains versus ccTLD domains and
therefore are not appropriate for the proposed use.  The Global Phishing
Survey Reports provide break-downs by TLD, and so the stats can be separated
into gTLD and ccTLD.  More crucially, the Global Phishing Survey Reports
distinguish between domain names registered by phishers ("malicious
domains") and domains that were broken into on compromised servers.  Domain
names registered by phishers are more relevant to the trust issue.  Domains
that were broken into on compromised servers is really a measure of hosting
(IT) security and so are not as relevant to stability is the gTLD space.  I
also note that phishing is only one kind of security or abuse issue, and it
is one that consumes relatively few domain names compared to other types of
problems.  

 

Terminology also needs some examination.  For example, note that III(a) and
III(b) are measures of security, not of "stability".  III(c) is a measure of
technical stability, and is an important metric, but the Health Index is
proposed to measure "non-technical stability."

 

The health of the marketplace can also be measured by:

.         the total number of registrars and the number accredited and
de-accredited

.         the number of registry failures, the number of registry contracts
that change hands,  and/or the number of registries sent to Emergency
Transition.

 

Sincerely yours,

--Greg Aaron

President, Illumintel Inc.