Zabbix notifications by phone via Asterisk

One night at the station, the air conditioners turned off and the temperature started to rise, naturally, I received email notifications on the phone, but since it was night, I only saw them in the morning, so it was necessary for such an emergency to quickly make the possibility of phone call notifications.

I will give an example of the variant of Zabbix alerts using a phone call through Asterisk.

Asterisk should have a module loaded, to do this, open the module configuration file, for example, in the nano editor (Ctrl+X to exit, y/n to save or cancel changes):

sudo nano /etc/asterisk/modules.conf

Continue reading Zabbix notifications by phone via Asterisk

Installing and Configuring Zabbix Server on Linux Ubuntu

Zabbix — system monitoring, tracking servers and network equipment.

Below I will give an example of installing Zabbix Server and Zabbix Agent from the official Ubuntu repository, if you need to install the latest version, you must first install Zabbix repositories, for example, see my article – Installing Zabbix Server 3.4 on Ubuntu 16.04

Install the server, web interface and agent:

sudo apt-get install zabbix-server-mysql zabbix-frontend-php zabbix-agent traceroute fping

If during the installation did not configure the mysql parameters, then manually create the MySQL user and complete the rights to the database:

mysql -uroot -p
create database zabbix character set utf8 collate utf8_bin;
grant all privileges on zabbix.* to zabbix@localhost identified by 'PASSWORD';
exit

We import tables into the database:

cd /usr/share/zabbix-server-mysql
sudo gunzip schema.sql.gz
sudo gunzip images.sql.gz
sudo gunzip data.sql.gz
mysql -uroot -p zabbix < schema.sql
mysql -uroot -p zabbix < images.sql
mysql -uroot -p zabbix < data.sql

Configs are located at /etc/zabbix/

The web interface configuration is located at /etc/apache2/conf-available/zabbix
If it does not exist, you can copy the template and configure:

sudo cp /etc/zabbix/apache.conf /etc/apache2/sites-enabled/zabbix.conf

Or we will make a symbolic link to the web server directory on the zabbix files:

sudo ln -s /usr/share/zabbix/ /var/www/html/

Open the configuration file PHP for example in the editor nano (CTRL+X for exit, y/n for saving or canceling the changes):

sudo nano /etc/php5/apache2/php.ini

And change some of the minimum parameters:

date.timezone = Europe/Kiev
max_execution_time 300
memory_limit 128M
post_max_size 16M
upload_max_filesize 2M
max_input_time 300

For the changes to take effect, you must restart the web server:

sudo /etc/init.d/apache2 restart

Open the web interface at http://SERVER/zabbix/ and follow the prompts for further installation.
By default, the user name for logging in through the web interface is admin, the password is zabbix.
When saving the configuration file zabbix.conf.php there will be a message that there is no write permission, so we will temporarily allow them:

sudo chmod 777 /etc/zabbix

When the configuration file is saved back:

sudo chmod 755 /etc/zabbix

Also, in the second configuration file zabbix_server.conf, the login and password for connecting to the MySQL database (same as in zabbix.conf.php):

sudo nano /etc/zabbix/zabbix_server.conf

If the message appears in the web interface that the zabbix server is not running, then you need to change the START=no parameter to START=yes in the /etc/default/zabbix-server file and execute the start command:

sudo service zabbix-server start

After installation, I recommend changing the password to admin and disabling the guest user.

Installing Zabbix Server 3.4 on Ubuntu 16.04

On the test I will install Zabbix Server 3.4 in Ubuntu Server 16.04.

First of all, we will install a repository with Zabbix 3.4 version for Ubuntu 16.04, as the old version is available in the official Ubuntu repository:

sudo wget http://repo.zabbix.com/zabbix/3.4/ubuntu/pool/main/z/zabbix-release/zabbix-release_3.4-1+xenial_all.deb
sudo dpkg -i zabbix-release_3.4-1+xenial_all.deb
sudo apt update

Now run the Zabbix Server and Zabbix Agent installation command:

sudo apt install zabbix-server-mysql zabbix-frontend-php zabbix-agent

Connect to MySQL:

mysql -uroot -p

Create the database and the user:

create database zabbix character set utf8 collate utf8_bin;
grant all privileges on zabbix.* to zabbix@localhost identified by 'PASSWORD';
quit;

I import the tables into the created database:

zcat /usr/share/doc/zabbix-server-mysql*/create.sql.gz | mysql -uzabbix -p zabbix

Open the configuration file in a text editor and where “DBPassword” is the password for the database:

sudo nano /etc/zabbix/zabbix_server.conf

Open the configuration file in a text editor and where “DBPassword” write the password to the database:

systemctl restart zabbix-server zabbix-agent
systemctl enable zabbix-server zabbix-agent

To restart it is still possible so:

sudo service zabbix-server start

In php.ini, specify the date.timezone, for example Europe/Kiev, or uncomment it in /etc/zabbix/apache.conf if it is copied to the web server configuration:

sudo cp /etc/zabbix/apache.conf /etc/apache2/sites-enabled/zabbix.conf

Or we will make the link where we need:

sudo ln -s /usr/share/zabbix/ /var/www/html/

Open the web interface at http://SERVER/zabbix/ and follow the prompts for further installation.
By default, the user name for logging in through the web interface is admin, the password is zabbix.

See also my article:
Installing and Configuring Zabbix Server on Linux Ubuntu

Monitoring the web site in Zabbix

Here is an example of setting up monitoring of a web site in Zabbix:

1) Select or add a network node for which we will observe.

2) Add to the Web site “Web Scenario”, specifying in the first tab:
Name: any
Agent: for example, Internet Explorer 11.0
in the “Steps” tab, add a step:
Name: any
URL: for example http://example.com
Required string: any string that is present on the site
Required status codes: 200

3) Add a graph for the created “Web Scenario” to see the download speed by selecting Download speed for scenario “example.com check”.

4) We add a trigger that will work after 3 unsuccessful attempts to access the site:
Name: any, for example “Site example.com is not available”
Importance: any
Expression:

{example.com:web.test.fail[example.com check].last(#3)}<>0

Done.

Solution of error with Detect operating system in Zabbix

The standard “Detect operating system” script does not work in Zabbix, when trying to execute it, the “sudo: no tty present and no askpass program specified” error appears, so for its operation it is necessary to take several actions.

The first is to install nmap:

apt-get install sudo nmap

The second is to allow the user of Zabbix to run nmap, for this open the file /etc/sudoers in any editor, for example nano:

sudo nano /etc/sudoers

and add the following line to it:

zabbix  ALL=(root) NOPASSWD: /usr/bin/nmap

or you can not open it in the editor, but execute the command:

sudo echo "zabbix  ALL=(root) NOPASSWD: /usr/bin/nmap" >> /etc/sudoers

After these manipulations, the script should work, determine the operating system, show the list of open ports, etc.

SQL queries for Zabbix

I will write some useful examples of sql queries for the Zabbix database:

Search for a host by name:

SELECT * FROM hosts WHERE host like '%name%';
SELECT * FROM hosts WHERE name like '%name%';

Find the data items of the specified host:

SELECT * FROM items WHERE hostid = '10105';

Find the history of the values for the specified data item:

SELECT * FROM history WHERE itemid = '24526';

Delete the whole history of the data element until 01.11.2014 (the time is specified in Unix format, converters can be found through the search engine):

DELETE FROM history WHERE itemid = '24526' AND clock < '1414800000';

Delete all data history until 01.11.2014:

DELETE FROM history WHERE clock < '1414800000';

Massively changed the interval and dynamics of changes to the data elements of templates and hosts, I have the following queries (the first identifies the template ID or host, the second changes the intervals):

SELECT * FROM `hosts` WHERE host="Template ICMP Ping";
UPDATE items SET delay=3600 WHERE hostid=10105 AND delay=600;
UPDATE items SET trends=180 WHERE hostid=10047 AND trends=365;

Monitoring of Apache2 parameters in Zabbix

Install the necessary components:

sudo apt-get install curl

Activate the module information about apache2 (usually it is activated initially):

sudo a2enmod info

Open the configuration file of the module and specify the IP address of the zabbix server to allow it to view information about apache2 (if apache2 is on the local machine, then access is usually allowed by specifying local or 127.0.0.1):

sudo nano /etc/apache2/mods-enabled/status.conf

In the nano editor, CTRL+X is used to exit and y/n to save or discard changes. Example of specifying IP:

<Location /server-status>
    SetHandler server-status
    Require local
    Require ip 192.168.1.5
</Location>

Restart the web server to apply the changes:

sudo service apache2 restart

Now the information about apache2 is available through the browser by the link http://HOST/server-status

Let’s move on to Zabbix. Create a folder for scripts if it does not exist:

mkdir /etc/zabbix/scripts/
chown root:zabbix -R /etc/zabbix/scripts/
chmod 750 /etc/zabbix/scripts/

In the zabbix server configuration file, we specify the path to this folder:

sudo nano /etc/zabbix/zabbix_server.conf
ExternalScripts=PATH

Now I will give the content of the script:

#!/bin/bash
if [[ -z "$1" || -z "$2" || -z "$3" ]]; then
  exit 1
fi
##### PARAMETERS #####
RESERVED="$1"
METRIC="$2"
URL="$3"
STATSURL="${URL}?auto"
#
CACHE_TTL="55"
CACHE_FILE="/tmp/zabbix.apache2.`echo ${URL} | md5sum | cut -d" " -f1`.cache"
EXEC_TIMEOUT="2"
NOW_TIME=`date '+%s'`
##### RUN #####
if [ -s "${CACHE_FILE}" ]; then
  CACHE_TIME=`stat -c"%Y" "${CACHE_FILE}"`
else
  CACHE_TIME=0
fi
DELTA_TIME=$((${NOW_TIME} - ${CACHE_TIME}))
#
if [ ${DELTA_TIME} -lt ${EXEC_TIMEOUT} ]; then
  sleep $((${EXEC_TIMEOUT} - ${DELTA_TIME}))
elif [ ${DELTA_TIME} -gt ${CACHE_TTL} ]; then
  echo "" >> "${CACHE_FILE}" # !!!
  DATACACHE=`curl -sS --insecure --max-time ${EXEC_TIMEOUT} "${STATSURL}" 2>&1`
  echo "${DATACACHE}" > "${CACHE_FILE}" # !!!
  echo "URL=${URL}"  >> "${CACHE_FILE}" # !!!
  chmod 640 "${CACHE_FILE}"
fi
#
if [ "${METRIC}" = "accesses" ]; then
  cat "${CACHE_FILE}" | grep -i "accesses" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "kbytes" ]; then
  cat "${CACHE_FILE}" | grep -i "kbytes" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "cpuload" ]; then
  cat "${CACHE_FILE}" | grep -i "cpuload" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "uptime" ]; then
  cat "${CACHE_FILE}" | grep -i "uptime" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "avgreq" ]; then
  cat "${CACHE_FILE}" | grep -i "ReqPerSec" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "avgreqbytes" ]; then
  cat "${CACHE_FILE}" | grep -i "BytesPerReq" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "avgbytes" ]; then
  cat "${CACHE_FILE}" | grep -i "BytesPerSec" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "busyworkers" ]; then
  cat "${CACHE_FILE}" | grep -i "BusyWorkers" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "idleworkers" ]; then
  cat "${CACHE_FILE}" | grep -i "idleworkers" | cut -d':' -f2 | head -n1
fi
if [ "${METRIC}" = "totalslots" ]; then
  cat "${CACHE_FILE}" | grep -i "Scoreboard" | cut -d':' -f2 | sed -e 's/ //g' | wc -c | awk '{print $1-1}'
fi
#
exit 0

Let’s make the script file executable:

chown root:zabbix /etc/zabbix/scripts/apache2-status.sh
chmod 550 /etc/zabbix/scripts/apache2-status.sh

Script validation example:

sudo -u zabbix /etc/zabbix/scripts/apache2-status.sh none accesses http://HOST/server-status

Open the zabbix agent configuration file in the editor:

sudo nano /etc/zabbix/zabbix_agentd.conf

And we will specify the following parameters:

UserParameter=apache2[*],/etc/zabbix/scripts/apache2-status.sh "none" "$1" "$2"

Some commands allow you to see the number of apache2 processes and connections to port 80, for this you do not need a script, for example, you can specify:

UserParameter=apache2.count_processes,ps aux | grep apache | wc -l
UserParameter=connections_on_80_port,netstat -na | grep :80 | wc -l

Restart the zabbix agent to apply the changes:

sudo /etc/init.d/zabbix-agent restart

Let’s check:

zabbix_get -s 127.0.0.1 -k "apache2[accesses,http://HOST/server-status]"

Now we will create a template and add data elements, an example of created data items:

apache2[KEY,http://HOST/server-status]

Exported ready template – apache2-status

We also need to add a macro to the monitored host:

Macro: {$APACHE_STATS_URL}
Value: http://HOST/server-status

Done.

The number of apache2 processes can be obtained from the Zabbix agent by creating on the Zabbix server a data element with a key:

proc.num[apache2]

Monitoring nf_conntrack in Zabbix

I fixed an error on one server once, read my article about it How to fix the error “nf_conntrack: table full, dropping package”
And there was an idea to control nf_conntrack in Zabbix.

View the current value and the maximum can be commands:

cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max

Opened the configuration file Zabbix agent in a text editor:

nano /etc/zabbix/zabbix_agentd.conf

And he added a couple of lines at the end:

UserParameter=nf_conntrack_count, cat /proc/sys/net/netfilter/nf_conntrack_count
UserParameter=nf_conntrack_max, cat /proc/sys/net/netfilter/nf_conntrack_max

Restart the Zabbix agent to apply the changes:

sudo /etc/init.d/zabbix-agent restart

Now you can create a template on the Zabbix server (for example, with the name “TemplateName”), create the data elements in it: nf_conntrack_count and nf_conntrack_max.
Create a graph for the created data items.
You can create a trigger such as an expression (it will work when the current value of nf_conntrack exceeds 3100000):

{TemplateName:nf_conntrack_count.last(0)}>3100000

Apply the template to the desired nodes of the network.

Done.

How to increase the number of recent events in the Zabbix panel

I noticed that the Zabbix panel displays a maximum of 20 recent events, and the rest should be viewed in groups of network nodes.

To increase the number of recent events displayed, open the file ../zabbix/include/defines.inc.php in any editor, for example, in nano:

sudo nano /usr/share/zabbix/include/defines.inc.php

and approximately on line 537 we find the following:

define('DEFAULT_LATEST_ISSUES_CNT', 20);

Let’s change the number 20 to 150, for example, and save the changes, then 150 recent events will be displayed in the Zabbix panel.

By the way, in the nano editor, the keyboard shortcut Ctrl+W is used for searching, Ctrl+O and Enter to save, and Ctrl+X to exit.

Monitoring Linux ISG in Zabbix

Today I wanted to monitor Linux ISG sessions in Zabbix.

By entering the command on one of the servers:

/opt/ISG/bin/ISG.pl show_count

Saw the following:

Approved sessions count: 2021
Unapproved sessions count: 2

The Zabbix agent on the server has already been installed, so it opened its configuration file (in the nano editor, the Ctrl+X keys for the exit, and y/n for saving or canceling the changes):

nano /etc/zabbix/zabbix_agentd.conf

Invented and added the following code:

UserParameter=isg.approved, /opt/ISG/bin/ISG.pl show_count | grep "Approved sessions count:" | awk '{print $4}'
UserParameter=isg.unapproved, /opt/ISG/bin/ISG.pl show_count | grep "Unapproved sessions count:" | awk '{print $4}'

We will allow Zabbix agent to work as root with the user specifying:

AllowRoot=1

Restart the Zabbix agent to apply the changes:

sudo /etc/init.d/zabbix-agent restart

On the Zabbix server, create an ISG template, add the data elements to it, specifying the type – Zabbix agent, and the keys: isg.approved, isg.unapproved.
Create graphics for the created data items.

Apply the template to the desired nodes of the network.

Done.