Re: [gnso-ff-pdp-may08] Mannheim score concerns (minority view)

[Dave Piscitello <dave.piscitello@xxxxxxxxx>]

Interesting perspective. I don't entirely agree with the conclusion but am 
happy to see this included.

Having spent too many hours flogging through event logs from poorly tuned 
systems in the early days of IDS, I'll confess that I am not an advocate of 
automation without human oversight, George. I also find it hard to imagine I 
think it would be helpful if you take a look at Joe's proposed text and see 
where you might describe the means and value of human "participation". Perhaps 
there will be broader support if we see the text:-)

On 9/17/08 12:23 PM, "George Kirikos" <fastflux@xxxxxxxx> wrote:



Hi folks,

Just to followup on today's call, on the use of the Mannheim fast flux
score formula (page 8, #16) that the majority supported.

I'm concerned that applying any mechanical formula will inevitably
lead to more and more false positives, and that if the formula's score
is applied automatically without human oversight, many innocent
bystanders will be negatively affected.

In particular, just as malevolent virus authors ("the bad guys") today
purchase anti-virus software to pre-test their creations against the
signatures provided by anti-virus vendors, malevolent agents using
fast flux techniques can certainly test their networks to see whether
their score is at an "acceptable" level. In other words, they'll
adapt. Thus, the formula begins to lose its power to discriminate
between good and bad over time due to this adaptation. Bad guys are
certainly creative and have resources to adapt. I wouldn't be
surprised to see some of them buying registrars, or even TLD
registries, to further their goal of not being shut down.

The second reason that the rate of false positives will change over
time is due to the adoption of beneficial fast flux techniques by a
growing number of organizations, as leading edge techniques move from
"early adopters" into the mainstream. I brought up this issue before
in relation to Bayes' theorm, at:

http://forum.icann.org/lists/gnso-ff-pdp-may08/msg00425.html

"The rarer the condition for which we are testing, the greater the
percentage of positive tests that will be false positives."

As more beneficial fast flux uses occur, the "malevolent" fast flux
becomes a rarer condition, and thus the percentage of false positives
will increase.

If these two factors lead to forced revisions over time to the
Mannheim fast flux score, I'm concerned that it becomes a losing "arms
race", just like signature-based anti-virus techniques.

There also didn't seem to be data on malevolent fast flux networks
that already might exist but that aren't caught by the Mannheim fast
flux score (i.e. false negatives), which goes to the same issue of how
often this Mannheim fast flux score formula might need to be revised
in the future.

I hope the above can be added as a "minority view" (or a majority if
many rethink the issue!).

Sincerely,

George Kirikos
www.LEAP.com