[gnso-wpm-dt] FW: Idea for GNSO Work Prioritization Model
- To: "gnso-wpm-dt@xxxxxxxxx" <gnso-wpm-dt@xxxxxxxxx>
- Subject: [gnso-wpm-dt] FW: Idea for GNSO Work Prioritization Model
- From: Glen de Saint Géry <Glen@xxxxxxxxx>
- Date: Sun, 15 Nov 2009 08:35:33 -0800
Forwarded On Behalf Of Olga Cavalli
the idea is to propose a certain methodology for prioritizing our work, and
then have feedback from the council.
So does that mean the group will then put together a short list of proposals
(after your call), which can then be submitted to the full Council?
Envoyé de mon iPhone
Le 14 nov. 2009 à 13:28, Olga Cavalli a écrit :
Chuck, thank you very much for your comments.
Glen is working on preparing the email list for this group.
I will send an email to the GNSO council list asking if there are others
interested in participating.
As a first step I suggest that we try to make a first proposal of categories
related to the action items list, to have a first starting point. These
categories can then be modified, reviewed, changed.
Lets try to have a teleconference and exchange ideas of how to organize
Glen, Gisella, can we try to make a doodle for this week?
Best regards and thanks
2009/11/13 Gomes, Chuck
Please see my comments below.
On Behalf Of Olga Cavalli November 13, 2009 7:31 AM
Subject: Re: Idea for GNSO Work Prioritization Model
thanks Liz and Ken for your emai, sample chart and example data.
The 4 quadrant model looks very good, we have also tried with an xy chart or a
buble chart, which essentially have the same concept.
In my opinion the key issue is the rating of the action items and the
definition of the categories. In Ken´s example he considers certain raters, and
I think that this rating should be an excercise of the whole council.
The Delphi could be also useful but again the key element is the definition of
the provided rankings, in this case made by experts. (and your suggestion of
the council being the experts is ok with me)
In the definition of the priorites I have some questions. Most appealing
projects are the ones with low cost and high value? or priorities are related
with other variables, like need or due dates for example? Who establishes these
priorities and categories?
[Gomes, Chuck] The Council members, with input from their respective groups
will set the categories and the priorities.
I think the next step should be to defining the categories, and then group the
action items under these categories and after that prioritize them.
[Gomes, Chuck] Agree. The Action Items list is a start. I also suggested an
approach to reduce the number of catefories and thereby eventually making the
prioritization exercise more managable.
In the definition of the categories my opinion is that we must involve all the
[Gomes, Chuck] Untimately the full Council has to be involved in all of the
decisions whether that be categories, priorities or methodologies. But the the
decision on Thursday in Seoul was to have the small group develop a proposed
approach for Council consideration. We have various ideas on the table but we
have not yet developed a concrete approach that we can propose to the Council.
If we involve the full Council at this point, we will pretty much be back where
we were in Seoul.
Glen, it could be very useful if you can help us preparing an email list with
all of us listed in this emai, I also think that we should ask if other GNSO
councillors may join this excercise.
[Gomes, Chuck] New volunteers are always welcome and it is fine to invite
others but let's not delay our work waiting for additional volunteers. I
suggest that Olga send a message to the Council list reminding Councilors that
the small group is working on the task and invite new participants if they want
to join. In the meantime, we should continue our work with the goal of
developing a fairly concrete methodology that can be used now and in the future
as new tasks are considered by the Council. I see our tasks as follows: 1)
recommend an approach to defining the categories; 2) recommend a methodology
that can be used to prioritize existing work as well as to help us make
decisions when new tasks are considered in the future.
I am arriving in Egypt tonight and will be in meetings during the IGF until
Friday, so I could participate in a conference call early morning or late
evening Egypt time. Perhaps we can try with a doodle.
[Gomes, Chuck] Agree.
Best regards and thanks.
2009/11/9 Liz Gasster
Olga and All,
Attached is a possible approach that staff has developed to address work
prioritization using a 4 quadrant model (see spreadsheet named "GNSO Work
Prioritization draft"). We came up with this based on Chuck's description of a
method he recently used, adapting for our use in the GNSO context. It can be
modified or completely replaced based on your input, we just suggest it to get
this group started with some ideas. Note that the second spreadsheet also
attached, entitled "Priorities for GNSO 4 Nov 09" is a test example, using this
methodology, that Chuck prepared. In our proposed example, we enabled 2
dimensions: Difficulty/Cost and Value/Benefit. The Cost variable is placed on
X with Y being used for relative Value. In Chuck's application, he switched the
quadrants in his version and added the current work groups, so it would be
useful for you to look at both of these examples.
Since we probably do not have reliable absolute scales for the variables we
used, one approach would be to ask the community to rank all group projects
from 1 to n on each of these dimensions (Ken used 8 groups in this
illustration). Then, we develop a composite average ranking from the
individual survey results in order to build the scatter chart. See Sheet2 for
an example of how we might develop the average rankings. In this example, Ken
created 4 raters P1-P4 and permitted ranking ties (e.g. 1-1-3-3-5-6-6-8). For
plotting the groups, we could use the actual computed average rankings, or as
Ken did in this case, we could assign ordinal values from 1 to 8 based on the
averages, that is, make the lowest average rank = 1, next = 2, and so on...
Unless the raters tend to agree on their relative rankings, the plot points
will pile up on one another toward the center, which is an artifact of using
averages. Depending on the total number of raters, we could also explore using
median or mode once we see the actual survey data. Since the objective is
prioritization, there is really no right/wrong answer.
In looking at the Summary chart of staff's example, quadrant Q1 will the most
appealing projects with relatively low cost and high value. Q4 shows the least
desirable. Which projects in Q2 and Q3 are more desirable will depend upon how
important value is to cost.
Another option for establishing the rankings is to use a modified Delphi
approach. Delphi is a systematic, interactive method which relies on a panel
of experts (e.g. the Council). The experts provide individual rankings in two
or more rounds. After each round, a facilitator provides an anonymous summary
of the experts' rankings as well as the reasons they provided for their
judgments. Experts are then encouraged to revise their earlier answers in
light of the replies of other members of their panel. It is believed that
during this process the range of the answers will decrease and the group will
converge towards the "best" answer. Finally, the process is stopped after a
pre-defined stop criterion (e.g. number of rounds, achievement of consensus,
stability of results) and the mean<http://en.wikipedia.org/wiki/Mean> or
median<http://en.wikipedia.org/wiki/Median> scores of the final rounds
determine the results. The Delphi method is based on the assumption that group
judgments are more valid than individual judgments.
Please let us know how you would like to proceed and whether you would like us
to arrange a conference call to discuss.