Installing the Ministra TV Platform (Stalker Middleware) in Ubuntu

On the test, I will install the Ministra TV Platform (formerly called Stalker Middleware) in Ubuntu Server 14.04 LTS.

First we will make a request for download by visiting the official website www.infomir.eu.
Unpack the downloaded archive:

sudo apt-get install unzip
unzip stalker_portal-5.3.0.zip

Copy the files to the web server directory:

cp -a infomirgroup-stalker_portal* /var/www/stalker_portal

Install the necessary components:

sudo apt-get update
sudo apt-get -y upgrade
sudo apt-get install -y -u apache2 nginx memcached mysql-server php5 php5-mysql php-pear nodejs upstart 
sudo pear channel-discover pear.phing.info
sudo pear install phing/phing

Create MySQL database and user:

mysql -u root -p
CREATE DATABASE stalker_db;
CREATE USER 'stalker'@'localhost' IDENTIFIED BY 'PASSWORD';
GRANT ALL PRIVILEGES ON stalker_db.* TO stalker@localhost IDENTIFIED BY '1' WITH GRANT OPTION;
FLUSH PRIVILEGES;
exit

If you need to change the settings in the /var/www/stalker_portal/server/config.ini configuration file, create a custom.ini file and add the parameters that you need to change:

sudo nano /var/www/stalker_portal/server/custom.ini

For example, the password for the database:

[database]
mysql_pass = PASSWORD

Perform phing (this process can take a long time, also in my case the MySQL user root password was requested):

cd /var/www/stalker_portal/deploy/
sudo phing

In /etc/mysql/my.cnf we will specify:

max_allowed_packet = 32M

We activate short PHP tags in /etc/php5/apache2/php.ini:

short_open_tag = On

The libapache2-mod-php5filter package should not be installed on the system:

sudo apt-get purge libapache2-mod-php5filter

Let’s make a backup copy of the standard apache2 web server configuration and clear the file:

sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/000-default_backup.conf
sudo nano /etc/apache2/sites-available/000-default.conf

Add to it:

    <VirtualHost *:88>
            ServerAdmin webmaster@localhost
            DocumentRoot /var/www
            <Directory /var/www/stalker_portal/>
                    Options -Indexes -MultiViews
                    AllowOverride ALL
                    Require all granted
            </Directory>
            ErrorLog ${APACHE_LOG_DIR}/error.log
            CustomLog ${APACHE_LOG_DIR}/access.log combined
    </VirtualHost>

Let’s change the port to Listen 88:

sudo nano /etc/apache2/ports.conf

Restart apache2 to apply the changes:

sudo service apache2 restart

Let’s make a backup copy of the standard nginx web server configuration and clear the file:

sudo mv /etc/nginx/sites-available/default /etc/nginx/sites-available/default_backup
sudo nano /etc/nginx/sites-available/default

Add to it:

    server {
        listen       80;
        server_name  localhost;
     
        location / {
            proxy_pass http://127.0.0.1:88/;
            proxy_set_header Host $host:$server_port;
            proxy_set_header X-Real-IP $remote_addr;
        }
     
        location ~* \.(htm|html|jpeg|jpg|gif|png|css|js)$ {
            root /var/www;
            expires 30d;
        }
    }

Restart nginx to apply the changes:

sudo service nginx restart

Now you can open the browser interface administrator interface http://SERVER/stalker_portal, login – admin, password – 1.
And the client – http://SERVER/stalker_portal/c/

The reason for increasing RX overruns on the network adapter

I noticed once on one of the servers that the RX overruns value is growing slightly.

I executed several commands (where p2p1 and p2p2 are the names of network interfaces):

ifconfig p2p1
ifconfig p2p2

Only the value of RX overruns increased by 10 packets every second, with traffic at about 2Gb/s (100,000 packets per second).
The server was equipped with the network adapter “HP NC552SFP 10Gb 2-Port Ethernet Server Adapter” with a network controller from Emulex.

Having looked the size of the maximum and current buffer:

ethtool -g p2p1
ethtool -g p2p2

Found that the buffers are set to maximum, TX buffer is 4096, and the maximum possible RX buffer was only 512.
See also my article – Changing TX and RX network interface buffers in Linux

Having looked at the correct distribution of network card interrupts over the processor cores:

grep p2p1 /proc/interrupts

Found that the network adapter has 4 IRQ interrupts maximum, and irqbalance accordingly allocated them to 4 cores, and the processor cores are 24.

The problem was solved by replacing the network adapter with a more expensive one – “665249-B21 HP 10Gb 2-port 560SFP + Adapter” with an Intel 82599 network controller.
After that, an error was not observed, RX and TX buffers were 4096, and IRQ was distributed to all 24 cores.
After a couple of days, the error counters remained at zero:

p2p1      RX packets:62535001155 errors:0 dropped:0 overruns:0 frame:0
          TX packets:36343078751 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:77395016742081 (77.3 TB)  TX bytes:10991051263063 (10.9 TB)

p2p2      RX packets:35672087256 errors:0 dropped:0 overruns:0 frame:0
          TX packets:58598868464 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:10996254475480 (10.9 TB)  TX bytes:73378418623349 (73.3 TB)

Probably the previous network adapter was some kind of truncated version, since the price was two times lower, and in general for serious purposes it is better to use network adapters with a controller from Intel.

See also my article – Configuring the Network in Linux

How to make a bootable USB flash drive with CAINE

To make a bootable USB flash drive with CAINE, perform several actions:

1) Download the image CAINE from the official site www.caine-live.net

2) Download Universal USB Installer http://www.pendrivelinux.com/universal-usb-installer-easy-as-1-2-3/

3) We connect the USB flash drive to the computer and run the Universal USB Installer. In it, we agree with the license agreement by clicking “I Agree“, in the second window where “Step 1:” choose “CAINE Linux“, where “Step 2” click “Browse” and point to the downloaded image of CAINE (for example, caine9.0.iso). In “Step 3:” select the letter of the flash drive, tick the “Format” (this will re-partition the file system table and erase all the data on the USB flash drive).

4) Click “Create” and wait for the process to complete.

IPTables rules for ntopng

First of all, let’s look at the current IPTables rules:

iptables -nvL

To open the ntopng port, add the rule:

sudo iptables -A INPUT -m tcp -p tcp --dport 3000 -j ACCEPT

To open the ntopng port for a specific network or IP only:

sudo iptables -A INPUT -m tcp -p tcp --dport 3000 -s 10.0.0.0/24 -j ACCEPT

See also my articles:
Configuring IPTables
Install and configure ntopng

IPTables rules for nprobe

First of all, let’s look at the current IPTables rules:

iptables -nvL

In order for nprobe to accept NetFlow data, open the port for it:

sudo iptables -A INPUT -p udp --dport 2055 -j ACCEPT

In order for nprobe to accept NetFlow data only from a particular network or IP:

sudo iptables -A INPUT -s 10.0.0.0/24 -p udp --dport 2055 -j ACCEPT

See also my articles:
Configuring IPTables
Install and configure nprobe

Install and configure nprobe

Suppose we installed ntopng as I described in this article – Install and configure ntopng
That is, they selected the necessary package at http://packages.ntop.org/apt-stable/ and downloaded it:

wget wget http://apt-stable.ntop.org/16.04/all/apt-ntop-stable.deb
sudo dpkg -i apt-ntop-stable.deb

Install nprobe if it is not installed:

sudo apt-get clean all
sudo apt-get update
sudo apt-get install nprobe

To receive NetFlow data and transfer it to ntopng, create a file (in the nano editor, press CTRL+X to exit, y/x to save or discard changes):

sudo nano /etc/nprobe/nprobe-anyname.conf

Add to it:

--zmq="tcp://*:5556"
-3 2055
--flow-version 9
-n=none
-i=none

2055 is the port on which you want to receive NetFlow data, and port 5556 is used to transmit it to ntopng.
See my articles on configuring NetFlow on switches:
Configuring NetFlow on Cisco
Configuring sFlow on D-Link Switches
Setting up and using Traffic Flow in Mikrotik

Now it remains to open the ntopng configuration in a text editor:

sudo nano /etc/ntopng/ntopng.conf

And add a line at the end (thereby adding the nprobe interface to collect statistics):

--interface="tcp://127.0.0.1:5556"

It remains to restart ntopng to apply the changes:

sudo service ntopng restart

Check whether everything works:

sudo netstat -tulpen | grep 2055
sudo netstat -tulpen | grep 5556
sudo /etc/init.d/nprobe status

I noticed that nprobe does not always shut down after the command:

sudo /etc/init.d/nprobe stop

Therefore, if necessary, you can stop it like this:

sudo killall nprobe
sudo kill -9 PID

In the free version of nprobe, I had a message:

NOTE: This is a DEMO version limited to 25000 flows export.

The full version can be purchased at the official site of shop.ntop.org.

See also:
IPTables rules for nprobe