So my question is...what's out there that will allow us to check
thresholds on traffic, and notify us if needed?
RMON alarms and events for one. These are available on pretty much all
recent versions of IOS. You can set a rising or falling threshhold on
any MIB variable you like, and period of time between polls. This will
generate a trap to a network management station, and you can choose to
do what with you will the alarms.
If you want to tie this stuff into scripts you can use the net-snmp trap
daemon to call various trap handlers that could do something keep
track of the duration of the spike or send an alert.
Another thing that is out there in later releases is the EVENT MIB.
This is probably overkill for what you want, and the only way to
configure it is through SNMP.
For all of this stuff there is documentation on CCO.