[gnso-wpm-dt] FW: Idea for GNSO Work Prioritization Model

[Glen de Saint Géry <Glen@xxxxxxxxx>]

Forwarded On Behalf Of Olga Cavalli

Stéphane,
the idea is to propose a certain methodology for prioritizing our work, and 
then have feedback from the council.
Regards
Olga
2009/11/14 Stephvangelder
Olga,

So does that mean the group will then put together a short list of proposals 
(after your call), which can then be submitted to the full Council?

Stéphane

Envoyé de mon iPhone

Le 14 nov. 2009 à 13:28, Olga Cavalli a écrit :
Hi,
Chuck, thank you very much for your comments.

Glen is working on preparing the email list for this group.

I will send an email to the GNSO council list asking if there are others 
interested in participating.

As a first step I suggest that we try to make a first proposal of categories 
related to the action items list, to have a first starting point. These 
categories can then be modified, reviewed, changed.

Lets try to have a teleconference and exchange ideas of how to organize 
categories.

Glen, Gisella, can we try to make a doodle for this week?

Best regards and thanks

Olga


2009/11/13 Gomes, Chuck
Please see my comments below.

Chuck

________________________________
On Behalf Of Olga Cavalli November 13, 2009 7:31 AM


Subject: Re: Idea for GNSO Work Prioritization Model

Hi,
thanks Liz and Ken for your emai, sample chart and example data.

The 4 quadrant model looks very good, we have also tried with an xy chart or a 
buble chart, which essentially have the same concept.

In my opinion the key issue is the rating of the action items and the 
definition of the categories. In Ken´s example he considers certain raters, and 
I think that this rating should be an excercise of the whole council.

The Delphi could be also useful but again the key element is the definition of 
the provided rankings, in this case made by experts. (and your suggestion of 
the council being the experts is ok with me)

In the definition of the priorites I have some questions. Most appealing 
projects are the ones with low cost and high value? or priorities are related 
with other variables, like need or due dates for example? Who establishes these 
priorities and categories?
[Gomes, Chuck] The Council members, with input from their respective groups 
will set the categories and the priorities.


I think the next step should be to defining the categories, and then group the 
action items under these categories and after that prioritize them.
[Gomes, Chuck] Agree. The Action Items list is a start.  I also suggested an 
approach to reduce the number of catefories and thereby eventually making the 
prioritization  exercise more managable.


In the definition of the categories my opinion is that we must  involve all the 
GNSO Council.
[Gomes, Chuck] Untimately the full Council has to be involved in all of the 
decisions whether that be categories, priorities or methodologies.  But the the 
decision on Thursday in Seoul was to have the small group develop a proposed 
approach for Council consideration.  We have various ideas on the table but we 
have not yet developed a concrete approach that we can propose to the Council.  
If we involve the full Council at this point, we will pretty much be back where 
we were in Seoul.


Glen, it could be very useful if you can help us preparing an email list with 
all of us listed in this emai, I also think that we should ask if other GNSO 
councillors may join this excercise.
[Gomes, Chuck] New volunteers are always welcome and it is fine to invite 
others but let's not delay our work waiting for additional volunteers.  I 
suggest that Olga send a message to the Council list reminding Councilors that 
the small group is working on the task and invite new participants if they want 
to join.  In the meantime, we should continue our work with the goal of 
developing a fairly concrete methodology that can be used now and in the future 
as new tasks are considered by the Council.  I see our tasks as follows: 1) 
recommend an approach to defining the categories; 2) recommend a methodology 
that can be used to prioritize existing work as well as to help us make 
decisions when new tasks are considered in the future.


I am arriving in Egypt tonight and will be in meetings during the IGF until 
Friday, so I could participate in a conference call early morning or late 
evening Egypt time. Perhaps we can try with a doodle.
[Gomes, Chuck] Agree.


Best regards and thanks.
Olga



2009/11/9 Liz Gasster
Olga and All,

Attached is a possible approach that staff has developed to address work 
prioritization using a 4 quadrant model  (see spreadsheet named "GNSO Work 
Prioritization draft").  We came up with this based on Chuck's description of a 
method he recently used, adapting for our use in the GNSO context.  It can be 
modified or completely replaced based on your input, we just suggest it to get 
this group started with some ideas.   Note that the second spreadsheet also 
attached, entitled "Priorities for GNSO 4 Nov 09" is a test example, using this 
methodology, that Chuck prepared.  In our proposed example, we enabled 2 
dimensions:  Difficulty/Cost and Value/Benefit.  The Cost variable is placed on 
X with Y being used for relative Value. In Chuck's application, he switched the 
quadrants in his version and added the current work groups, so it would be 
useful for you to look at both of these examples.

Since we probably do not have reliable absolute scales for the variables we 
used, one approach would be to ask the community to rank all group projects 
from 1 to n on each of these dimensions (Ken used 8 groups in this 
illustration).  Then, we develop a composite average ranking from the 
individual survey results in order to build the scatter chart.   See Sheet2 for 
an example of how we might develop the average rankings.  In this example, Ken 
created 4 raters P1-P4 and permitted ranking ties (e.g. 1-1-3-3-5-6-6-8).  For 
plotting the groups, we could use the actual computed average rankings, or as 
Ken did in this case, we could assign ordinal values from 1 to 8 based on the 
averages, that is, make the lowest average rank = 1, next = 2, and so on...  
Unless the raters tend to agree on their relative rankings, the plot points 
will pile up on one another toward the center, which is an artifact of using 
averages.  Depending on the total number of raters, we could also explore using 
median or mode once we see the actual survey data.  Since the objective is 
prioritization, there is really no right/wrong answer.

In looking at the Summary chart of staff's example, quadrant Q1 will the most 
appealing projects with relatively low cost and high value.  Q4 shows the least 
desirable.  Which projects in Q2 and Q3 are more desirable will depend upon how 
important value is to cost.

Another option for establishing the rankings is to use a modified Delphi 
approach.  Delphi is a systematic, interactive method which relies on a panel 
of experts (e.g. the Council).  The experts provide individual rankings in two 
or more rounds.  After each round, a facilitator provides an anonymous summary 
of the experts' rankings as well as the reasons they provided for their 
judgments.  Experts are then encouraged to revise their earlier answers in 
light of the replies of other members of their panel.  It is believed that 
during this process the range of the answers will decrease and the group will 
converge towards the "best" answer.  Finally, the process is stopped after a 
pre-defined stop criterion (e.g. number of rounds, achievement of consensus, 
stability of results) and the mean<http://en.wikipedia.org/wiki/Mean> or 
median<http://en.wikipedia.org/wiki/Median> scores of the final rounds 
determine the results.  The Delphi method is based on the assumption that group 
judgments are more valid than individual judgments.

Please let us know how you would like to proceed and whether you would like us 
to arrange a conference call to discuss.

Thanks, Liz