Contact Us
+1.831.460.7010

Most network devices -- switches, routers, servers -- have multiple interfaces and use counters and gauges to track performance and network activity.   Certain statistics pertaining to network traffic are collected and stored.  These statistics include the total number of incoming packets and outgoing packets,  the total number of discarded packets, the total number of packets containing errors that prevent further processing, the total number of ICMP incoming and outgoing packets, etc.

Management Depends on Accurate Counters

{rokzoom}images/stories/silvercreek2/mib-table.gif{/rokzoom}

Management applications use these counters to derive statistics on the performance of the network and the device. For example, on a high performance router, if the value of the ifInOctets counter was less than 100, this would indicate that the router had just rebooted, or that something was wrong because such a low number of octets (a measure of network traffic) were flowing into the router. The management applications correlate and compute the values of many variables, such as the values of the counters, the speed of the interface, the operational status, etc., to identify and pinpoint problems. Thus, it is vitally important that the counter and interface information be accurate and correct. If it is not correct, the management application will be using bad information resulting in false positives, false negatives, or incorrect reports. Network administrators who depend on this information would be unable to troubleshoot and correct network problems.

Carriers, Operators, and ISPs Bill their Customers Based on SNMP Counter and Interface Data

Carriers, operators, and ISPs invoice their customers based on various usage models.  The billing systems make use of  the SNMP counters and interfaces as this is the "lingua franca" of the hardware devices.   Thus, it is vitally important that the counter and interface information be accurate and correct. If it is not correct, the carriers, operators and ISPs will incorrectly bill their customers and violate their Service Level Agreements.  The resulting damage to customer good will and customer relationships could be extremely serious.

The Challenge of Implementing Correct Counters and Interfaces

Whether or not they utilize SNMP, development engineers must implement counters and interfaces in their hardware devices. The values of the counters and interfaces are maintained in operating system logs and directories. Once implemented, these engineers  must comprehensively verify that all the statistics and values produced by the counters and interfaces are correct. Special purpose traffic generators will produce traffic at line speed to test the upper limits of the counters and interfaces to determine what level of load they can handle before dropping packets.  This type of testing determines if the device can "keep up" with line speed. If not, then the device must be re-designed or re-engineered, or the specifications for the device must be downgraded.

Three Layers

When this hardware load testing is complete, the next step is to test the SNMP implementation of the counters and interfaces. The SNMP agent in the hardware device will read the values of the counters and interfaces that are produced in the system logs. These values will then populate the corresponding SNMP MIB objects. The testing focus is now to verify that the SNMP counters and interfaces are operating correctly, within a margin of error (plus or minus 5%), under normal traffic conditions.

Under very heavy load conditions, such as a Denial of Service attack, or an unprovisioned and now saturated link, it is not expected that the SNMP counters will keep up. In this conditions, the SNMP traffic is generally discarded while the device attempts to perform its core function, e.g. route packets.

Note the difference between the testing task of the hardware test engineer and the SNMP test engineer:

The hardware engineer or device firmware developers are responsible for testing the accuracy of the counters, interfaces, and network statistics under all conditions, including under a heavyload.

The SNMP engineer is testing SNMP MIB counter and interface accuracy, with a margin of error, under a normal network load.

InterWorking Labs developed 97 tests for insuring SNMP MIB counter and interface accuracy, with a margin of error, under a normal network load . These tests extend SilverCreek, the SNMP Test Suite, which provides syntax, semantic, conformance and compliance testing for SNMP agents and Management Information Bases.

How Does the SNMP Counter and Interface Testing Work?

First, SilverCreek reads the value of a particular counter or interface MIB object.  

Next, SilverCreek sends a request to the traffic source to generate and send a certain type of traffic at a certain speed to the Device Under Test.

Then, SilverCreek  reads the values of the counters or  interface MIB object in the Device Under Test.  

Next, SilverCreek evaluates whether or not the counter or interface contains the appropriate value based on the amount of traffic that was sent.  

Finally, SilverCreek determines the result -- PASS, FAIL, etc. The test results are incorporated with the results from the SNMP syntactic, semantic, conformance, and compliance tests in a database or file for further examination and processing.

What About the Traffic Source?

Using the SilverCreek Developer's Guide, almost any traffic source that can be controlled with Tcl commands, can  be integrated with the SilverCreek Counter and Interface Tests.  This includes very simple software programs as well as commercial traffic generators.  

SilverCreek Diagram

Test Coverage

The SilverCreek Counters and Interfaces extension contains 97 sample tests written in Tcl.  The tests serve as sample scripts for further development for other counters.

The tests cover almost all inbound counters and some outbound counters for the Interface/IP/ICMP/TCP/UDP counters defined in RFC1213/ RFC2011/ RFC2012/ RFC2013/ RFC2863/ and RFC2665 for Ethernet like interfaces (ifType is either 6, 7, 11, 62, 69 or 117).

General Test Environment Configuration

Each test contains a detailed description on how to configure the environment in order to run that particular test.

The tests generally assume:

  1. A traffic source has been identified, configured properly to send traffic to the appropriate port on the Device Under Test.
  2. No discontinuities in the value of the counter occur during the execution of the test.
  3. Unless explicitly stated, the interface to be tested should not be put into promiscuous mode.
 

Test Suite Coverage Details

This is a summary of the SilverCreek Counter and Interface Tests. These tests are available to SilverCreek customers. They can be integrated with a packet generator and customized for testing a device under tests. There are a total of 97 tests.

52.if.ifInOctets.x
Verify ifInOctets can count packets correctly for Ethernet-like interfaces.

52.if.ifInUcastPkts.x
Verify ifInUcastPkts can count packets correctly for Ethernet-like interfaces.

52.if.ifInNUcastPkts.x
Verify ifInNUcastPkts can count packets correctly for Ethernet-like interfaces.

52.if.ifInBroadcastPkts.x
Verify ifInBroadcastPkts can count packets correctly for Ethernet-like interfaces.

52.if.ifInMulticastPkts.x
Verify ifInMulticastPkts can count packets correctly for Ethernet-like interfaces.

52.if.ifInErrors.x
Verify ifInErrors can count packets correctly for Ethernet-like interfaces.

52.if.ifInUnknownProtos.x
Verify ifInUnknownProtos can count packets correctly for ethernet-like interfaces.

52.if.ifHCInOctets.x
Verify ifHCInOctets can count packets correctly for Ethernet-like interfaces.

52.if.ifHCInUcastPkts.x
Verify ifInUcastPkts can count packets correctly for ethernet-like interfaces.

52.if.ifHCInBroadcastPkts.x
Verify ifHCInBroadcast can count packets correctly for ethernet-like interfaces.

52.if.ifHCInMulticastPkts.x
Verify ifHCInMulticastPkts can count packets correctly for ethernet-like interfaces.

52.dot3.dot3StatsAlignmentErrors.x
Verify dot3StatsAlignmentErrors can count packets correctly for ethernet-like interfaces.

52.dot3.dot3StatsFCSErrors.x
Verify dot3StatsFCSErrors can count packets correctly for ethernet-like interfaces.

52.dot3.dot3StatsFrameTooLongs.x
Verify dot3StatsFrameTooLongs can count packets correctly for ethernet-like interfaces.

52.dot3.if.ifInErrors.x
Check ifInErrors counter consistency with dot3StatsAlignmentErrors, dot3StatsFCSErrors, dot3StatsFrameTooLongs, dot3StatsInternalMacReceiveErrors and dot3StatsSymbolErrors.

52.dot3.dot3ControlInUnknownOpcodes.x
Verify dot3ControlInUnknownOpcodes can count packets correctly for ethernet-like interfaces.

52.dot3.dot3InPauseFrames.x
Verify dot3InPauseFrames can count packets correctly for ethernet-like interfaces.

52.ip.ipInReceives.x
Verify ipInReceives can count packets correctly.

52.ip.ipInHdrErrors.x
Verify ipInHdrErrors can count packets correctly.

52.ip.ipInAddrErrors.x
Verify that ipInAddrErrors can count packets correctly.

52.ip.ipForwDatagrams.x
Verify ipForwDatagrams can count packets correctly.

52.ip.ipOutNoRoutes.x
Verify ipOutNoRoutes can count packets correctly.

52.ip.ipInUnknownProtos.x
Verify ipInUnknownProtos can count packets correctly.

52.ip.ipInDelivers.x
Verifies ipInDelivers can count packets correctly.

52.ip.ipReasmReqds.x
Verify ipReasmReqds can count packets correctly.

52.ip.ipReasmOKs.x
Verify ipReasmOKs can count packets correctly.

52.ip.ipReasmFails.x
Verify ipReasmFails can count packets correctly.

52.icmp.icmpInMsgs.x
Verify icmpInMsgs can count packets correctly.

52.icmp.icmpOutMsgs.x
Verify icmpOutMsgs can count packets correctly.

52.icmp.icmpInErrors.x
Verify icmpInErrors can count packets correctly.

52.icmp.icmpInDestUnreachs.x
Verify icmpInDestUnreachs can count packets correctly.

52.icmp.icmpInTimeExcds.x
Verify that icmpInTimeExcds can count packets correctly.

52.icmp.icmpInParmProbs.x
Verify icmpInTimeExcds can count packets correctly.

52.icmp.icmpInSrcQuenchs.x
Verify icmpInSrcQuenchs can count packets correctly.

52.icmp.icmpInRedirects.x
Verify icmpInRedirects can count packets correctly.

52.icmp.icmpInRedirects.x
Verify that icmpInRedirects can count packets correctly.

52.icmp.icmpInEchos.x
Verify that icmpInEchos can count packets correctly.

52.icmp.icmpOutEchoReps.x
Verify that icmpOutEchoReps can count packets correctly.

52.icmp.icmpInEchoReps.x
Verify that icmpInEchoReps can count packets correctly.

52.icmp.icmpInTimestamps.x
Verify that icmpInTimestamps can count packets correctly.

52.icmp.icmpOutTimestampReps.x
Verify that icmpOutTimestampRep can count packets correctly.

52.icmp.icmpInTimestampReps.x
Verify that icmpInTimestampRep can count packets correctly.

52.icmp.icmpInAddrMasks.x
Verify that icmpInAddrMasks can count packets correctly.

52.icmp.icmpOutAddrMaskReps.x
Verify that icmpOutAddrMaskReps can count packets correctly.

52.icmp.icmpInAddrMaskReps.x
Verify that icmpInAddrMaskReps can count packets correctly.

52.tcp.tcpInSegs.x
Verify that tcpInSegs can count packets correctly.

52.tcp.tcpInErrs.x
Verify that tcpInErrs can count packets correctly.

52.udp.udpInErrors.x
Verify that udpInErrors can count packets correctly.

52.if.ifInOctets.x
Verify ifInOctets can count packets correctly for Ethernet-like interfaces.

52.if.ifInUcastPkts.x
Verify ifInUcastPkts can count packets correctly for Ethernet-like interfaces.

52.if.ifInNUcastPkts.x
Verify ifInNUcastPkts can count packets correctly for Ethernet-like interfaces.

52.if.ifInBroadcastPkts.x
Verify ifInBroadcastPkts can count packets correctly for Ethernet-like interfaces.

52.if.ifInMulticastPkts.x
Verify ifInMulticastPkts can count packets correctly for Ethernet-like interfaces.

52.if.ifInErrors.x
Verify ifInErrors can count packets correctly for Ethernet-like interfaces.

52.if.ifInUnknownProtos.x
Verify ifInUnknownProtos can count packets correctly for ethernet-like interfaces.

52.if.ifHCInOctets.x
Verify ifHCInOctets can count packets correctly for Ethernet-like interfaces.

52.if.ifHCInUcastPkts.x
Verify ifInUcastPkts can count packets correctly for ethernet-like interfaces.

52.if.ifHCInBroadcastPkts.x
Verify ifHCInBroadcast can count packets correctly for ethernet-like interfaces.

52.if.ifHCInMulticastPkts.x
Verify ifHCInMulticastPkts can count packets correctly for ethernet-like interfaces.

52.dot3.dot3StatsAlignmentErrors.x
Verify dot3StatsAlignmentErrors can count packets correctly for ethernet-like interfaces.

52.dot3.dot3StatsFCSErrors.x
Verify dot3StatsFCSErrors can count packets correctly for ethernet-like interfaces.

52.dot3.dot3StatsFrameTooLongs.x
Verify dot3StatsFrameTooLongs can count packets correctly for ethernet-like interfaces.

52.dot3.if.ifInErrors.x
Check ifInErrors counter consistency with dot3StatsAlignmentErrors, dot3StatsFCSErrors, dot3StatsFrameTooLongs, dot3StatsInternalMacReceiveErrors and dot3StatsSymbolErrors.

52.dot3.dot3ControlInUnknownOpcodes.x
Verify dot3ControlInUnknownOpcodes can count packets correctly for ethernet-like interfaces.

52.dot3.dot3InPauseFrames.x
Verify dot3InPauseFrames can count packets correctly for ethernet-like interfaces.

52.ip.ipInReceives.x
Verify ipInReceives can count packets correctly.

52.ip.ipInHdrErrors.x
Verify ipInHdrErrors can count packets correctly.

52.ip.ipInAddrErrors.x
Verify that ipInAddrErrors can count packets correctly.

52.ip.ipForwDatagrams.x
Verify ipForwDatagrams can count packets correctly.

52.ip.ipOutNoRoutes.x
Verify ipOutNoRoutes can count packets correctly.

52.ip.ipInUnknownProtos.x
Verify ipInUnknownProtos can count packets correctly.

52.ip.ipInDelivers.x
Verifies ipInDelivers can count packets correctly.

52.ip.ipReasmReqds.x
Verify ipReasmReqds can count packets correctly.

52.ip.ipReasmOKs.x
Verify ipReasmOKs can count packets correctly.

52.ip.ipReasmFails.x
Verify ipReasmFails can count packets correctly.

52.icmp.icmpInMsgs.x
Verify icmpInMsgs can count packets correctly.

52.icmp.icmpOutMsgs.x
Verify icmpOutMsgs can count packets correctly.

52.icmp.icmpInErrors.x
Verify icmpInErrors can count packets correctly.

52.icmp.icmpInDestUnreachs.x
Verify icmpInDestUnreachs can count packets correctly.

52.icmp.icmpInTimeExcds.x
Verify that icmpInTimeExcds can count packets correctly.

52.icmp.icmpInParmProbs.x
Verify icmpInTimeExcds can count packets correctly.

52.icmp.icmpInSrcQuenchs.x
Verify icmpInSrcQuenchs can count packets correctly.

52.icmp.icmpInRedirects.x
Verify icmpInRedirects can count packets correctly.

52.icmp.icmpInRedirects.x
Verify that icmpInRedirects can count packets correctly.

52.icmp.icmpInEchos.x
Verify that icmpInEchos can count packets correctly.

52.icmp.icmpOutEchoReps.x
Verify that icmpOutEchoReps can count packets correctly.

52.icmp.icmpInEchoReps.x
Verify that icmpInEchoReps can count packets correctly.

52.icmp.icmpInTimestamps.x
Verify that icmpInTimestamps can count packets correctly.

52.icmp.icmpOutTimestampReps.x
Verify that icmpOutTimestampRep can count packets correctly.

52.icmp.icmpInTimestampReps.x
Verify that icmpInTimestampRep can count packets correctly.

52.icmp.icmpInAddrMasks.x
Verify that icmpInAddrMasks can count packets correctly.

52.icmp.icmpOutAddrMaskReps.x
Verify that icmpOutAddrMaskReps can count packets correctly.

52.icmp.icmpInAddrMaskReps.x
Verify that icmpInAddrMaskReps can count packets correctly.

52.tcp.tcpInSegs.x
Verify that tcpInSegs can count packets correctly.

52.tcp.tcpInErrs.x
Verify that tcpInErrs can count packets correctly.

52.udp.udpInErrors.x
Verify that udpInErrors can count packets correctly.

 

Summary of the SilverCreek Counter and Interface Tests

SilverCreek is an entire test suite designed to test your agent implementation, check the conformance to the MIB definitions and SNMP protocols to ensure interoperability and conformance to the latest standards from the IETF.  The Counters and Interfaces Extension provides a mechanism for a quality assurance engineer or tester to verify that the counters and interfaces correctly reflect accurate values based on the amount and or type of traffic sent.  This benefits operators and carriers who bill their customers based on their utilization of capacity.  In addition, network administrators and network managers can reliably use this information to monitor their networks and quickly discover and correct network problems.



Want to Know More?