Re: [gnso-wpm-dt] WPM-DT: "Red Team" Idea

[Stéphane Van Gelder <stephane.vangelder@xxxxxxxxx>]

Le 24 déc. 2009 à 20:18, Gomes, Chuck a écrit :

> Ken,
>  
> Thanks for the added red team detail.  Very helpful.  I would just qualify 
> your suggestions in this way:  I don't think we have time to have the red 
> team do an exhaustive review or to test the process like we have done.  
> Rather I think a fairly quick review of the process focusing on the questions 
> you suggest below should suffice. 

I agree. Stéphane.

>  
> Chuck
> 
> From: owner-gnso-wpm-dt@xxxxxxxxx [mailto:owner-gnso-wpm-dt@xxxxxxxxx] On 
> Behalf Of Ken Bour
> Sent: Thursday, December 24, 2009 1:36 PM
> To: gnso-wpm-dt@xxxxxxxxx
> Subject: [gnso-wpm-dt] WPM-DT: "Red Team" Idea
> 
> WPM-DT Members:
>  
> I have been thinking about Adrian’s suggestion and Olga’s question concerning 
> how we might take advantage of the “Red Team” concept. 
>  
> According to “Red Teams: An Audit Tool, Technique and Methodology for 
> Information Assurance,” the typical functions of such teams are to:  
> ·         Provide a surrogate adversary to "sharpen skills, expose 
> vulnerabilities that adversaries might exploit and increase the understanding 
> of the options and responses available to adversaries and competitors." The 
> red team may accomplish this by emulating the adversary.
> ·         Play "devil's advocate." The red team can offer different 
> alternatives to current plans, operations, processes and assumptions.
> ·         Offer sources of judgment that are external to the organization and 
> act as a "sounding board" for new ideas that may arise from red team 
> engagements
> Thus far, our first product is a two-dimensional rating approach (still in 
> development/test phase), but there may be others as we begin tackling how the 
> resultant charts/graphs can be used for project prioritization -- the 
> ultimate goal.   Although we are making excellent progress, we are still some 
> distance from drafting a complete package that can be recommended to the 
> Council. 
>  
> Once we reach that end-state, it might be useful to subject the ultimate 
> solution set to an exhaustive independent test just before going “live.”   We 
> have seen in our own testing that it is through exercising the processes that 
> we have uncovered potential defects (e.g. adding project acronyms instead of 
> sequence numbers; definition for Y to include GNSO more specifically).   
> Since the WPM-DT has been intellectually close to the development from the 
> outset, it is possible that we have overlooked certain fundamental elements 
> that might cause problems in production.   One way to minimize that 
> eventuality is to bring in another set of objective evaluators (or Red Team) 
> and ask them to execute the entire rating/prioritization process -- start to 
> finish -- as laid out.   By exercising the methodology in this way, a Red 
> Team could offer a fresh perspective including asking naïve questions and 
> probing the underlying rationale in ways that might not have been adequately 
> challenged.  
>  
> Following the above outline, when the time is right, we might ask Adrian to 
> lead a small group (size will depend on the ultimate solution) to actually 
> perform the entire set of procedures that are packaged into the final 
> recommendation.   The Red Team can check to ensure that the DT’s original 
> goals have been met and that the process:   
> ·         is user-friendly, unambiguous, and straightforward to execute;
> ·         produces realistic outputs that will enable the Council to make 
> effective prioritization decisions; and
> ·         is structured not only as a one-time exercise, but considers the 
> inclusion of new projects as they are proposed in the future. 
>  
> In addition, other potential evaluative questions might include: 
> ·         Are the objectives clear?  Have we satisfactorily answered the 
> obvious question, “Why are we doing this activity?”   
> ·         Does the process make sense in terms of the leading to the stated 
> objectives?
> ·         Does the methodology require accepting any assumptions that have 
> not been disclosed?
> ·         Are the instructions, guidelines, and definitions clear and 
> sensible?
> ·         If tools are provided, do they work as described? 
> ·         Others? 
>  
> To summarize, once the DT has reached closure on a final Work Prioritization 
> Model and recommendation, it could engage Adrian (and others?) to provide a 
> neutral and objective critique -- “emulating the adversary” -- or, in this 
> case, simulating how other GNSO Councilors might react to and interact with 
> the final proposed solution.    
>  
> Regards,
>  
> Ken Bour
>

Attachment: smime.p7s
Description: S/MIME cryptographic signature