RE: [gnso-wpm-dt] Thoughts About Step 6 (In Progress)

["Gomes, Chuck" <cgomes@xxxxxxxxxxxx>]

Ken,
 
Thanks to you and Jaime for all the thought you put into this.
 
I inserted a few comments below.
 
Chuck


________________________________

        From: owner-gnso-wpm-dt@xxxxxxxxx
[mailto:owner-gnso-wpm-dt@xxxxxxxxx] On Behalf Of Ken Bour
        Sent: Monday, February 08, 2010 2:32 PM
        To: gnso-wpm-dt@xxxxxxxxx
        Subject: [gnso-wpm-dt] Thoughts About Step 6 (In Progress)
        
        

        WPM Team Members:

         

        I apologize for the lateness and length of these ensuing notes,
but I have been struggling with how to facilitate and assist the team
given the discussion that took place during our last session on 2
February 2010.  

         

        I will try to summarize where I think we are and offer a few
thoughts that have finally congealed in my mind over the last couple of
days.  

         

        The 4-Quadrant Model:  Value/Benefit vs. Resources Needed

         

        In our last teleconference, we spent most of the time talking
about various issues that were contained in the paper I wrote (29 Jan
2010) concerning how to derive a prioritized list from the
two-dimensional model that rates Value/Benefit (Y) against Resources
Needed (X).  

         

        Because we used identical scales for both X and Y, I argued
that, at least arithmetically, we end up with ambivalence between
Quadrants 2 (High Value, Low Resources) and Quadrant 3 (Low Value, High
Resources).  I acknowledged that team members might interpret these two
dimensions as unequal and, indeed, during our last discussion, everyone
seemed to agree that Value/Benefit is, indeed, more important to the
model than Resources Needed.  That acknowledgement raised a question as
to how we might weight Value/Benefit so that it would be the dominant or
primary factor in deriving a prioritization. 

         

        A second element in our discussion was how to provide more
granularity in the model so that projects within a quadrant, for
example, might be discretely ranked.  

         

        One suggestion was to independently rank the 15 projects by
Value/Benefit and Resources Needed, then place them on a chart/graph.
As team members will recall, we considered ranking both variables (from
1 to n) early in our deliberations and, because of its perceived
difficulty, chose instead to rate them using a 7-point scale.  We could
go back and try an ordinal ranking process; however, I still believe
that we will end up with a similar problem in that Value and Resources
would still be on par with each other relatively.  Said another way, how
would we determine that a project ranked 14 (out of 15) on Resources
should be prioritized lower than one ranked 14 (out of 15) on Value?  To
make Value/Benefit worth more in the model, we would have to come up
with a weighting system such as I proposed in my paper.  As I attempt to
explain in the material that follows, I am not recommending that we head
down that path...  

         

        Given the difficulty of assessing Resources Needed and its high
correlation to Value/Benefit (statistically and psychologically), the
team began to consider alternatives for that variable while retaining a
two dimensional construct.  One idea was to map Urgency against Value,
perhaps as a first step, followed by a second one that would incorporate
Resources Needed [Note: how that might be done was not fleshed out and,
in my mind, is not trivial since it potentially adds a 3rd dimension to
the model and, thereby, increasing its complexity].  

         

        As promised, Jaime sent me a paper he authored concerning the
inherent difficulties in relating Urgency to Importance (or Value) in a
model.  Although the following synopsis does not do adequate justice to
Jaime's entire article, I think it captures the salient points for our
purposes:  

        (1)         Definition:  Urgency relates to matters of impending
time (deadlines/dates) while Importance should signify only value.  

        (2)         For a variety of reasons (e.g. anxiety/dread, sense
of gain/loss, perceived/real consequences, varying interpretations of
deadlines), people have an exceptionally difficult time evaluating
Urgency and Importance independently from each other, which results in a
similar cross-correlation problem that we have already seen between
Value and Resources (or Cost); therefore,

        (3)         To utilize Urgency in a meaningful way, it should be
objectively (vs. subjectively) determined. 

         

        After pondering Jaime's persuasive arguments concerning Urgency,
I find myself in complete agreement; furthermore, I think there are
additional challenges with the use of this dimension. 

         

        The first challenge is:  Accepting Jaime's thesis as legitimate,
how would we determine Urgency on an objective basis?  In my 18 months'
experience at ICANN, I have not seen many actual dates imposed and, for
projects that do/did have deadlines, they are/were almost always missed
(and not renegotiated formally).  Such a reality would cause most, if
not all, of our projects to be rated "Not Urgent."  If that result were
obtained, how would it help our prioritization task?  We already have 12
out of 15 projects rated 4 or higher on Value/Benefit; thus, we would
likely have an even tighter bunching of projects than we have at
present.  

         

        I conveyed the above thought to Jaime who wrote back, "Your
provoking thoughts made me advance that Fatal Deadlines are practical
and real, often related to linked events (a decision, the signing of a
contract, etc) and not to dates.  So, we should ask, for each of our
projects, if there is a linked event after which the work would be
pointless or would lose much of its value.  If this is not the case, I
agree with you that assessing urgency would not add anything and would
be a loss of time."  

         

        Perhaps we can take up the question that Jaime raises as to
whether we can objectively identify an EVENT (vs. date) for each project
that, once it occurs, would render the projects valueless.  If that is
possible, we still have the challenge of assigning it some numerical or
rating significance.  For example, a project that is approaching one
fatal event must still be rated higher/lower on Urgency than another one
facing a different fatal event.  If the events have dates associated
with them, the one closest could have the higher rating; but, if the
events are tied to other phenomena (not date-sensitive), as Jaime
hypothesizes, how would we choose between them without resorting to
value, importance, or consequences?  

         

        A second challenge is that I think of Urgency as being more of
an OUTCOME of the prioritization process than an INPUT to it.  If the
GNSO Council is to become the "Manager" of the Policy Development
Process, shouldn't one of its responsibilities be the establishment and
continual reevaluation of dates/deadlines and, thereby, Urgency - based
on a project's overall priority?  Given that few, if any, deadlines are
established from outside ICANN (I consider the Board "inside"), they are
largely treated as self-imposed and somewhat arbitrary.  As a result,
perhaps the Council should step into the managerial role of not only
prioritizing the work, but also establishing when results are needed.  

         

        Thoughts About How to Proceed?  

         

        Based upon the foregoing discussion, I suggest that the team not
introduce the concept of Urgency into the model.  After deeper
consideration, I do not think that it will add any new information and
would likely leave us in a similar position that we find ourselves with
the current model - unable to derive a useful prioritization.  

         

        Secondly, I suggest that the team consider dropping the
4-quadrant model and simplifying the prioritization process to just a
single dimension:  Value/Benefit.[Gomes, Chuck]   As you could probably
tell last week, I was starting to come to the same conclusion.  Once all
projects have been prioritized based on their overall value to
ICANN/GNSO, then a set of management questions should be undertaken that
include, but are not limited to:  

        (1)         Which projects should have dates/deadlines imposed,
thus establishing urgency?  Can any real consequences be identified (or
manufactured) that will cause the dates to be perceived and treated as
critical/urgent? 

        (2)         Which projects should have resources assigned
including types, skills, and quantities?

        (3)         Which projects, if any, should be accelerated,
stopped, or postponed?  

         

        Another reason that I think we should drop Resources Needed from
the model is that we defined it to include both Staff and Community.  In
order to make effective priority decisions (e.g. do this instead of
that), there must be real capacity constraints which make it
impractical, if not impossible, to simultaneously work on ALL projects
that are identified.  Although the level of Community involvement is
admittedly not infinite, I cannot think of any way to reasonably
constrain it.  Arguably, there are thousands of Internet users connected
to ICANN in one way or another through its various stakeholder groups.
If one were considering only Community volunteers (and not Staff for the
present), on what basis would one decline a 16th or 17th or 25th
project?  We acknowledge that existing volunteers are stretched thin,
which recognition led to this team being formed, but should we consider
that resource pool fixed and, if so, how would we assess its total
capacity?  On the other hand, if there is always the possibility of
involving more people, then the amount of work to be undertaken should
dictate the number of resources recruited and not the other way
around...  

         

        Having teased out the above conclusion, I do think that there is
a resource constraint that is measurable - Staff!  If one accepts the
practical reality that no project can be undertaken in the ICANN
Community without Staff involvement, then Staff resources (quantifiably
fixed in the short run) present a real and measurable constraint to how
much work can be actively pursued within the GNSO.  If a new project
comes up and all existing Staff resources (including consultants) are
maximally engaged with no slack capacity, then that project cannot be
initiated until a management re-prioritization decision is made. 
        [Gomes, Chuck] I am not sure that we should only consider Staff
resources. Before embarking on a project, I think we need to also ask
whether SGs will commit to providing resources without negatively
impacting other projects.  

         

        I'm not sure if I have made a convincing case; but, as I see it,
the Resources Needed problem can be simplified to consider only the GNSO
Policy Staff, which then functions as the effective capacity limiter to
total GNSO project workload.  Given this understanding, one way to
proceed might be as follows:

        1)      The Council prioritizes all projects on a
one-dimensional basis considering Value/Importance/Benefit/Criticality
as related to ICANN/GNSO's overall mission.  Prioritizations should be
recalibrated on a periodic basis (e.g. quarterly?) to reflect the
dynamics inherent in this environment.  

        2)      Once all projects are prioritized, the Council actively
"manages" the workload considering such factors as deadlines, Staff
resource availability, and consequences.  It commissions and
de-commissions teams as necessary to effectively utilize the aggregate
resource pool to achieve its goals and objectives.  

        3)      To facilitate its management responsibilities, projects
should be documented and tracked on an on-going basis using a
feature-rich web-based application (several open source options
available) that will assign Staff and Community resources to
projects/tasks complete with time tracking and collaborative toolsets
(e.g. integrated blogs, wikis, calendar, document repositories) that
enable work to be performed efficiently, effectively, and transparently.


         

        To achieve the prioritization in Step 1 above, we could utilize
our Value/Benefit definition and the 7-point scale as already tested.
For example, using our Delphi-based ratings for Y only, the following
prioritization would emerge:  [Gomes, Chuck]  Note that a project like
IRTB may not be able to be deleted because to do so would be to ignore a
requirement that is way overdue for reviewing a previously implemented
consenus policy; in addition, there appear to be sufficient resources to
continue it.  Also, a project like GEO is only requiring to GNSO members
and not taking up a lot of other GNSO time. Clearly it still requires
Staff time.  I think what it all comes down to, is in the end we will
have to evaluate each project individually for other factors and the
prioritization groupings as shown below are simply a guide. 

         

Project

Value

Rank

STI

6.0

1

WG

6.0

1

PDP

6.0

1

RAA

6.0

1

JIG

5.0

5

CCT

5.0

5

GCOT

5.0

5

IRD

5.0

5

CSG

5.0

5

IDNF

4.0

10

PED

4.0

10

ABUS

4.0

10

IRTB

3.5

13

GEO

2.0

14

TRAV

2.0

14

         

         

        There are several tied positions in the above ranking; however,
if it were determined by the Council that the GNSO could only handle 12
projects, hypothetically, then IDNF, PED, and ABUS (all tied at 10th)
would have to be evaluated as a small group to decide which one joins
IRTB, GEO, and TRAV to be stopped or postponed.  That tie-break could be
accomplished by using a simple majority vote.  

         

        If you managed to read this far, I thank you for your
forbearance and likewise apologize for the length of this treatise.
This prioritization task represents a complex problem set and I am
trying to offer it as much considered thought, including supporting
rationale, as I can manage.  

         

        I look forward to our session tomorrow and, of course, any
feedback that you may have in the meantime...  

         

        Ken Bour