Some thoughts on improving public comment periods

[Kieren McCarthy <kierenmccarthy@xxxxxxxxx>]

Hello,

I'm pleased to see some progress being made with respect to improving the
public comments process - something that has been a bone of contention for
all stakeholder groups for a number of years now.

I read the Focus Group report with interest and I have a few responses which
I hope ICANN will reflect on and consider.

I have been a follower and observer of ICANN for many years and have spent a
lot of time considering how to involve people more in the organisation's
processes, particular in my role as general manager of public participation
for ICANN for nearly three years.


1. The current suggestions look unlikely to solve many of the problems that
sparked this review in the first place. You may well find that months of
hard work amount to only a small incremental improvement and continued
complaints if you don't reflect on the underlying issues that have led to
suggested specific changes.



2. The suggested topic categories are a good example. They are all extremely
ICANN-focussed and are focussed on issues that are current ICANN
priorities.

Those priorities will change in just a few years, necessitating the need for
more categories, and causing some categories to fall into disrepair.

The categories also act as a barrier to engagement - they are too precise.
Someone who isn't already highly versed in the ICANNese is unlikely to
browse them, or understand what they mean. You are making it less likely
that people will comment.

The idea of specific or exclusive categories is also likely to be a problem,
especially when there is a crossover on a subject - which happens
frequently.

I would strongly suggest that ICANN consider the use of GENERIC TAGS to help
identify different comment periods. Tags would enable you to flag a given
comment period as being relevant in one or more areas. And using more
generic subject areas will help with a broader understanding and
identification of what a particular comment period is about.

This approach will work better, mean more and last longer that the current
category suggestions. Here is a suggested list that would cover just every
about comment period ICANN produces:

* Top-level domains

* Second-level domains

* Security

* IP addressing

* Internet governance

* Reviews & Reports

* Policy processes

* Internal issues

* Legal issues

* Events & Conferences

* Openness, Accountability and Transparency

* Finances


3. It is very disappointing to see the issue of prioritization raised and
then rapidly dismissed as "being too subjective".

This is a dangerous way of making improvements to a system - to put out a
specific suggestion and then walked away from it if there isn't agreement.

What that approach fundamentally misses is the *reason* why the idea of
prioritization cropped up in the first place. It is because everyone has a
difficult time seeing and understanding the relevance of a particular
comment period to them.

People just see a list of comment periods, and then even if they dig into
each one, it is hard to tell how important it really is. Or at what stage in
the process it is. Or if this is the best time to comment (are people still
looking for new ideas, or comments on specific ideas?)

The simple fact is that some comment periods are of greater relevance to a
larger group than others. And some comment periods have significant
implications that can be easily overlooked. Until there is a determined
effort to address that, the problem will keep coming up again and again.

Yes, rankings may be subjective but that doesn't mean they aren't enormously
helpful. Something can often be better than nothing.

And if you went through a list of current comment periods, it's likely that
nearly everyone would agree with the level of importance for any given
comment period - except in special cases when people would also be able to
say "well, this one is of particular interest to me because..."

There are other options too. Some suggestions:

* Put an initial rating of importance on a comment period and then allow
community members to add their own rankings. That way the community decides
its own level of importance.

* Use stars rather than words (low, high etc)

* Use highlighting tags e.g. if a "G" is added it means it is of particular
importance to a General audience. If an "Ry" appears, it is of particular
interest to a Registry, and so on.



4. The same problem appears with the Reply Cycle idea - don't get bogged
down in finding problems with every possible configuration. And try not to
look at it through the eyes of the current system. If you do that - and
there is plenty of evidence in the report that that is exactly what happened
- then you'll never made any useful progress.

What needs to be looked at is the fact that the comment periods are
unnecessarily static. A document is posted. Comments are sent in. A summary
is produced. Changes are made. A new document is put out.

In the Internet era we are all used to jumping in with comments and
interacting in near real time on particular issues. The ATRT suggested a
reply cycle because it was the simplest method of introducing a concept -
that comment periods need to be more interactive.

The most important aspect with this Reply Cycle idea is to make sure that
what you end up with does *not* include in it arbitrary rules. The idea is
to allow people to go back-and-forth a little bit; not to force people to
respond in the correct way at the correct time, or to create a
one-size-fits-all solution.



5. Re: technical improvements to the software/approach. Some good thoughts
here. But beware the use of wikis.

There are those who think wikis are wonderful. And those who will have
nothing to do with them. I think you should experiment for some months
before going any particular direction to avoid future arguments.

As a sidenote: the report says: "Due to the fact that a threaded discussion
environment has never been deployed at ICANN for Public Comments..."

In fact I ran a successful test of a dynamic forum for ICANN which broke a
comment period out into different components and had threaded conversations
about each part. Ask the ICANN tech team what the software was called.
Ultimately, it proved popular and looked promising but no one followed
through with the idea.

Part of the reason for that - and this is what I am warning about in general
here - was that people were quick to think up and run through potential and
future concerns about changing the current system. This is especially common
in the ICANN environment.

It is very easy to weigh a list of possible problems too heavily against all
the future benefits that will accrue. And it is all too easy to overlook
significant shortcomings in the existing system just because people have
grown used to it.

Especially in something as moving and transitory as a comment period, it is
an ideal opportunity to try out different things. Unless you manage to
create a system that actively prevents people from commenting, then you have
already hit the current comment period baseline and while people may gripe
about changes, they will be able to achieve exactly the same, so don't let
fear of change guide efforts to improve the system.

One suggestion - you need to find a way to allow people to note they have
commented with one click (this is now extremely common with software and
posts on Twitter and Facebook). This would draw in others. And there needs
to be a quick and easy way to see what others have said.



5. What is missing from the work done so far is any changes to how comment
periods are actually run by staff.

At the moment, staff  - who usually have the broadest knowledge of the issue
that is out for public comment - have a very passive role. They post the
document, then summarise comments at the end, and then try to figure out how
to make changes to the document as a result.

Seeing as public comment is *the* key point where work and ideas start to
gain broader acceptance and awareness, it will most likely be in everyone's
interests if staff took on more of a facilitator role wrt public comments.

It is not hard to drum up interest. Staff (and the GM of Public
Participation) could easily use email, Twitter, Facebook etc to highlight
that a comment period will soon be opening, and highlight aspects of that
comment period that would spark interest.

You could give a particular comment period its own hashtag - that would
spark debate. You could run a discussion forum on the ICANN Facebook page -
everyone already understands how that software works and won't blame ICANN
for it. Think positive engagement. Staff could email and actively engage the
people that they know and meet at ICANN meetings.

Staff could also elicit questions and provide answers while the comment
period is going. And encourage people who are focussed on the same point to
go away and come back with a short summary of their discussions before the
comment period closes.

This facilitation role would have a huge positive impact on any comment
period. But it does need to be an active consideration. Having a slightly
improved comment period system will not tackle the root problem - which is
that people are busy and they aren't sufficiently encouraged or enthused to
bother to comment on every ICANN document.

This would be a huge - but positive! - shift in staff behaviour so it would
need to be carefully and professionally introduced as it would inevitably be
met with suspicion and defensiveness. But I have no doubt whatsoever that
the rewards would be enormous.


6. Another thing that ICANN really needs to consider is explicitly giving
greater priority to particular respondents. Or at least breaking out
different respondents.

It really is a no-brainer that if a supporting organization or advisory
committee sends in a response then it should be taken more seriously. This
is for the simple reason that these are the structures of ICANN itself that
are designed to filter and raise issues.

Likewise, a party that is directly impacted by a decision can expect greater
consideration than a party which is not.

If someone posts anonymously, that is fine, but they should also generally
expect it to be given less weight than someone who is prepared to state who
they are and who they work for. ICANN is deciding policy rules, not holding
a music festival.

As to the obvious concern that good material or intelligent responses may be
lost through this process (which they already are being through the current
system); again, a simple system of community rating - allowing the community
to identify its own priorities and valued responses - would not only help
flag up particular posts but it may also people to reach a general consensus
in public and through a comment period - which is what the whole point of
comment periods should be in the first place.


I hope this feedback helps. And I hope you will start making positive
changes straight away by responding to this comment period response and/or
explaining where and when the ideas were considered and accepted (or not
accepted) and the reasons why either way.

That would make me feel that my efforts had been worthwhile and it would
encourage me to respond more in future. If you set up a system that does
that for every poster before you know it, people may get some real value
from commenting on ICANN's work.


Thanks


Kieren McCarthy