Merit Network
Can't find what you're looking for? Search the Mail Archives.
  About Merit   Services   Network   Resources & Support   Network Research   News   Events   Home

Discussion Communities: Merit Network Email List Archives

North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

NOC notification policies

  • From: Troy Davis
  • Date: Sat Jun 12 06:40:19 1999

This is half rant and half hope that it'll make someone, somewhere check
their emergency policies who otherwise wouldn't.

We're seeing a ~15 mbps flood, most of which is sourced from 
ernie.eecs.uic.edu.  Traceroutes to that machine show it to be ~800 ms
slower than the hop before it, so the flood isn't spoofed.  I did a whois on
uic.edu and called the contact.  I was greeted by voicemail, where I
promptly left a message.  uic.edu has a /16 and is not SWIPped out of anyone
else's ARIN space; the same person (voicemail during off hours) is the
contact.  A different voicemail number is the contact for AS 6200, the
University of Illinois at Chicago's AS.  Their web site didn't cover
networking.

If you can't have either your IP block or domain whois point folks to a
24/7 contact, get your uplink to be the point of contact.  We'll see in a 
second why that wouldn't have helped here, but in cases where the uplink 
has > 10% of a clue, it would.  So I moved up the food chain.

uic.edu receives transit from Nap.net (single-homed, according to
route-server.cerf.net and nitrous.digex.net).  After checking ARIN whois for
AS 6200 and Jared Mauch's (excellent, BTW) NOC contact list, I came up with
1-800-801-3920 (the other number, (414) 747-8747, refers to this number for
network emergencies).

I call it.  "Teresa" answers, representing GTE and Nap.net.  I tell Teresa
my problem and she politely requests I call back during business hours when
their staff is available.  I tell her how big this flood is and request she
page their engineer or her boss.  She refuses.  As she puts it, we're not
Nap.net customers.  I inform her of what the flood is doing to my network
and she says that they only serve customers, as if I was looking for
technical support.  She's unwilling (unable, as she puts it) to investigate, 
we trade arguments, it goes nowhere, we hang up.  Her wording sounded very 
much like an official Nap.net policy - no pages except for customers.

There is at least 1 "backbone" (cough) peering at multiple major NAPs who is
completely unreachable during the off hours.  I doubt my experience today
is as rare as we'd all like it to be.  I'm thoroughly disgusted.

Please test your 24/7 monitoring from the perspective of a third party who
needs immediate attention.  Follow what I did - see if whoises on your domain 
name, IP block, and/or ASN reveal the NOC number (or one which can refer you 
there).  Call it at 2 AM once or twice in the next couple weeks and see what 
happens.

Good night.

-- 
Troy Davis





Discussion Communities


About Merit | Services | Network | Resources & Support | Network Research
News | Events | Contact | Site Map | Merit Network Home


Merit Network, Inc.