Course Goals:

  • Understanding and using SNMP, IPMI and custom checks with Zabbix
  • Creating Templates for trending utilization and notification of network devices both OS based and Appliance based
  • Working with Triggers and Actions to create notifications (Polling and Traps)
  • Understanding and configuring ITIL service monitoring with Key Performance Indicators rolled up from server, process, service and dependency checking using Zabbix

Simple Network Management Protocol "SNMP": is a network protocol used to exchange server and application management data.  * There are 3 versions of SNMP v1, v2c and v3 with varying levels of security, controls and support by vendors.

  • The machine that recieves traps or sends queries to clients is called the SNMP Management Station.
  • There are 2 methods of exchanging data between the SNMP management station and the SNMP client - TRAP and POLLING (sometimes called discovery)
  • Data that can be retrieved from devices, applications and operating systems varies greatly between vendors but there is some standardization.  Most data is in the form of Attribute/Value Pairs or Tables that are made availble to SNMP by the vendor. 
  • A/V Pairs are defined by the Internet Assigned Numbers Authority (IANA) and are assigned to institutions for private use in what is defined as an ARC generally at an enterprise level.  Rice's defined ARC in OID format is 1.3.6.1.4.1.134 which translates to .iso.org.dod.internet.private.enterprises.Rice.
  • Management Information Base (Mibs) are a heirarchical representation of SNMP data, its data type definitions, OID and syntax.  MIBs provide the OID to Mib map and are provided by vendors or are based on IETF standards for standards based resources such as TCP/IP networking, printing, SMTP email, LDAP and other resources.
  • A open source client suite of software CLI tools is provided called net-snmp that is availbale for Windows, Mac and Linux at http://www.net-snmp.org/download.html
    • snmpget - used to querey hosts for information.
    • snmpwalk - used to traverse the SNMP OID tree that is supported by a client.
    • snmptable  - used to querey host for table information.
    • snmptranslate - used to translate between mibs and oids.

Intelligent Platform Management Interface (IPMI) is a standards based specification initiated by Intel to define a common set of interfaces for monitoring of computer hardware such as CPU, Fans, Power Supplies, Temperatures and other Iron related resources.

  • Similar to SNMP, IPMI if supported on the hardware communicates to a management station but can also be configured to communicate directly via email and SMS to managers.  Each vendor implementation may vary however.  As an example, SUN provides IPMI data via SNMP as well as IPMI directly. 
  • Zabbix supports the polling and trending of IPMI data in the same manner that SNMP is supported.  The only difference 

Where to Begin

  1. Install and/or configure an snmp client on the device that will be used as an SNMP client.  Most hardware and some software come with built in SNMP capabilities.
    1. Linux Installation - "yum install net-snmp" this will download and install all of the snmp client software, you will need to configure it yourself. We suggest a simple configuration with a read only community string of public.  If the machine is public facing, restrict SNMP to the monitoring station access and any equipment that you will test from.
    2. Windows Installations - Installation is from the Add or Remove Software/Hardware section of the Control Panel and choose Windows Components, Management and Monitoring Tools, SNMP with configuration done in Control Panel / Services.
    3. Once installed and configured you can use CLI tools to test your access controls and data.
  2. There are a number of resources used to find out what data is available from an operating system, a hardware component, a network resource or an application.
    1. Use snmpwalk to traverse the entire tree of available A/V Pairs.  "snmpwalk -v2c -cpublic <hostname or IP of client>
    2. Use snmpget to  querey for a specific oid for its value.  "snmpget -v2c -cpublic <hostname or IP of client> 1.3.6.1.4.1
    3. Vendor supplied mibs provide specific OID and ARC information regarding data that can be used for monitoring, trending and notification
    4. MIB repositories Mib Depotand Byte Sphereare online searchable databases where vendor mibs can be analyzed and downloaded.

The SNMP / IPMI Monitoring, Trending and Notification Process

  • Once clients are installed or configured to be accessible to and by the Zabbix server (monitor.rice.edu), the process of identifying, classifying and utilizing data for monitoring, trending and notification can begin. 
  • Begin by answering the following questions
    1. What do I need to know from the server or service(s) that will help me manage the service delivery?
      1. Do I need to track scalability?
      2. Do I want to be able to diagnose problems in the service delivery?
      3. What characterizes the service hardware and software requirements (disk space/performance, CPU cycles, Networking, memory)?
      4. How should the service or server be classified with regards to support (critical, business day only support, non-critical service)?
      5. When and how do I want to be notified if there is a problem?
      6. What should constitute a problem?
      7. Will baseline service data need to be gathered and if so, what data?
  • Determine if there is a way to get the data you need from the Zabbix agent, SNMP, IPMI or custom checks and implement.  Process status checks generally provide a fixed vocabulary of choices represented by integer values while performance data is generally real numbers from 0 to a defined maximum.
  • Configure Zabbix to poll or trap data required to monitor, trend or send alerts.
  • Define the trigger thresholds that will satisfy your needs based on an initial guess, trend data to date or other means.
  • Create actions that fulfill your notification requirements - set notifications to just yourself while in the initial stages.
  • Test the notifications, graphs and track over time for tuning
  • Rinse and repeat as necessary.
  • Define the IT Service monitoring that you would like and include maintenance windows and KPI.  Some additional A/V pairs may need to be added for KPI.  Status checks that are used for monitoring are good for KPI.

Example 1:  SNMP Printer Ink and Fuser polling

  1. Identify the printer make and model :  HP Laserjet 4000 series
  2. Use the web or telnet interface on the printer to configure it for SNMP
    1.  On a 4250, you can use the web interface under Networking Tab, Network Settings, SNMP
    2. Use telnet alternatively to set HP printer snmp settings Telnet <IP/Hostname> 
      1. snmp-config 1
      2. default-get-cmnty 1
      3. (to enable trap sending) auth-trap 1
      4. trap-dest 10.129.92.52 (IP of zabbix server)
    3. Download the vendors MIB or a close proximity from MIB Depot and evaluate
      1.  http://mibdepot.com
      2. Click the Search Option
      3. Choose Vendors on the left hand panel
      4. Locate and Select HP
      5. Search the web page for "Laserjet"
      6. Find the closest match  "Laserjet Series 4200-MIB" and note the SNMP version is V1, however we will attempt to use Version 2c and will see that it works.  If you don't get a return value try this trick.
      7. Browse the MIB looking for consumables such as INK and  FUSER information.  Fusers in HP Mibs are called Maintenance Kits for some models and Ink is sometimes called Toner or Marker Supplies so knowing the lingo helps.
      8. Identify the oid for a catagory such as prtMarkerSuppliesEntry  .1.3.6.1.2.1.43.11.1.1.9.1.1 and note the tree structure (how the oid numbering is aligned and what other values are close to this one)
      9. Test the oid against the printer  snmpwalk -v2c -cpublic -Of <ip of printer/hostname>  1.3.6.1.4.1.11.2.3.9.4.2.2.11.1.1  to see what is returned and disect the data
      10. bash-3.2# snmpwalk -v2c -cpublic  -Of  10.64.19.249 .1.3.6.1.2.1.43.11.1.1
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.2.1.1 = INTEGER: 1
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.2.1.2 = INTEGER: 1
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.3.1.1 = INTEGER: 0
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.3.1.2 = INTEGER: 0
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.4.1.1 = INTEGER: 3
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.4.1.2 = INTEGER: 3
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.5.1.1 = INTEGER: 21
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.5.1.2 = INTEGER: 15
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.6.1.1 = STRING: "Black Cartridge HP Q5942A"  This element describes the Toner as relative to item 1.1
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.6.1.2 = STRING: "Maintenance Kit HP 110V-Q5421A, 220V-Q5422A" This element describes the Fuser as relative to item 1.2
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.7.1.1 = INTEGER: 7
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.7.1.2 = INTEGER: 7
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.8.1.1 = INTEGER: 10000
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.8.1.2 = INTEGER: 225000
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.9.1.1 = INTEGER: 7300
        .iso.org.dod.internet.mgmt.mib-2.43.11.1.1.9.1.2 = INTEGER: 128665
      11. The Supplylevels for the Black Toner and the Fuser are defined in ...9.1.1 and 9.1.2, we know the values match the description because of the tree structure defined in the mib information found on midDepot states that the Printer Marker Supplies data is found under 1.3.6.1.4.1.11.2.3.9.4.2.2.11.1.1 and that sub elements description and supply levels are found under 6.1.1, 6.1.2 and 9.1.1 and 9.1.2 respectively as found in the snmpwalk.
      12. .iso.org.dod.internet.private.enterprises.43.11.1.1.9.1.1 = INTEGER: 7500
        .iso.org.dod.internet.private.enterprises.43.11.1.1.9.1.2 = INTEGER: 128833
      13.  Per the information on this OID at Mib Depot, the values indicate the amount of toner and fuser levels remaining .  The first number should be the Toner, the second is the fuser.  This is determined by looking at another A/V Pair under the prtMarkerSuppliesEntrydescription 1.3.6.1.4.1.11.2.3.9.4.2.2.11.1.1.6.0.  The values 8.1.1 and 8.1.2 are the maximum capacity values when new toner and fuser are installed.
      14. This may be able to be confirmed from the web interface on the printer depending on the make, model and firmware on the printer.



Toner Verification Calculation:  Data from snmpwalk shows current use at 7300 and max of 10,000  7300/10000 ~ 74% which is equivalent to the information from the gui so we are on the right track.  Data from snmp also shows the current fuser use at 128665 and a mx of 225000 which calculates 128665/225000 ~ 57% again matching the gui.  Now that we have confirmed that the data we are polling against is the appropriate, we can move to Zabbix.
Note that on the gui, the estimated number of pages for the toner is 5758 this correlates to 73% of the toner cartriage and calculates to ~8000 pages and not 10,000 set by the max value in the printer.  This is a fudge factor provided by the manufacturer to provide a reserve in a toner low operation.  So we will want to consider this when setting the notification threshold.  The manufacturer sets the low toner alert when the printer is a 2000 of 10000 printed pages per cartriage for this printer or 20% so we may want to do the same or lower if we have the supplies on hand.  Sheldon recommends 500 pages as the notification if the toner is on site and 100 for the fuser.

Defined Request:  I want to get an email when the toner level or the fuser needs replacement on an HP printer.

Move to Zabbix to start work on a Template

  1. First check to see if an existing Template has been set up by someone else.  If it exists, clone it for your use, if not create a new template and send a note to the list so that others will know it is there.
  2. An existing HP Printer Template exists so we will use Full Clone to make a copy for our use.
  3. Create a new snmp v1 item based on the OID identified in 2.c.xii  above and set the polling interval for once per day or 86,400 seconds.  For testing set this number low 60 seconds initially and change it later.
  4. Create 2 new triggers based on the OID to set an event threshold of < 500 pages for an action.
  5. Create a new action or adjust an existing action to send an alert when the threshold logic is true.
  6. Associate the template to a host that you wish to monitor
  7. Test the trigger by setting the comparison value to a one greater than the current value from 2.c.xii (7300) for Toner and validate that a page or email has been sent and that a status change is identified in the item.
  8. A graph can  be created for the host to track the data that you are polling.  Graphs can be configured to show triggers as well and both of these together can be helpful in diagnosing problems.
  9. Check your email or whatever notification service you set up to see if the trigger is working.
  10. If all the test work out, go back and set you polling interval in the template and the test value in the trigger to your permenant values. (86400 seconds polling current values and 500 pages for toner)
  • No labels