Monitor Network data transfer using Vnstat

If you want to monitor and manage your Internet bandwidth, perhaps to make sure your ISP is not overbilling you, try vnStat, an open source, Linux-based application that gives you a clear picture of your bandwidth usage. This utility got the command-line options and also got the UI part which give the output in form of a graph and is simple to install and easy to use.

vnStat is a network traffic monitor for Linux that keeps a log of daily network traffic for the selected interface(s). vnStat isn’t a packet sniffer. The traffic information is analyzed from the /proc filesystem. That way vnStat can be used even without root permissions

Installation/Configuration

1) Download the source – here
2) Untar – # tar -zxvf vnstat-1.6.tar.gz
3) Compile/Install the source

# ./configure
# make
# make install

4) To create a database for vnstat type in vnstat -u -i interface (ie, eth1), after the database is created, which is the first thing to do after a fresh install of vnstat, allow some time for the database to populate with information. After some time just type in vnstat to get a summary of your daily traffic. The database is stored in a file in the /var/lib/vnstat/ directory in a small binary file, you can sets up a cron job that refreshes the bandwidth consumption data on the eth0 port every five minutes

# vnstat

If you’re not satisfied with the appearance of vnStat’s command-line interface, you can install  PHP-based Web front end for vnStat, which takes the data collected by the command-line vnStat and displays it in tables and graphically in your browser. The front end requires a Web server configured with PHP.

Download the tarball, extract the files into a subdirectory of your Web server’s document root directory — most likely /var/www/ or /var/www/html — and give it the right permissions so the Web server can write to it:

# mkdir /var/www/vnstat
# tar -zxvf vnstat_php_frontend-1.4.tar.gz -C /var/www/vnstat/
# chown -R www-data.www-data /var/www/vnstat

Launch your browser and visit the vnStat page at http://localhost/vnstat to see something like …

 

Convert Gmail account into Web based file server

PhpGmailDrive (PGD) turns your Gmail account into a Web based file server. You may add more than one Gmail account, arrange attachments in multiple folders, and apply themes to the interface. It can be embedded into any HTML page.

PhpGmailDrive is a new type of file sharing utility. Unlike typical file servers (say xDrive) it uses Gmail as backend file server. You can enjoy it simply by hosting a small PHP script in your web site without any database.

Gmail file space is more than 2GB, so you can imagine you already have such a big space on internet. However, you will require a Gmail account (If you don’t have any Gmail address, go GmailSwap or ask your friends.).

PGD automatically connects Gmail server and fetches list of all attached files and generate downloadable links in a tree like view. If you are looking for a file server to upload your music or notes without any hassle, PGD is ideal for you. Technically speaking PGD is wrapper of Gmailer .
Thought there is no need to know hecks of PHP or Gmailer. Just you have to check that your PHP hosting service provider has curl extension of PHP to handle HTTP/HTTPS traffic.

Here are some key features of “Php Gmail Drive”:
* Successfully connects to Gmail and only grabs list of Attachments in Gmail messages.
* List of attachments are displayed with inbuilt Javascript engine. You can customize output to any other formats.
* 100% compliant to GMAILFS. So you can upload with Windows Gmail Shell extension or Linux GmailFS utilities..
* Supports Multiple Gmail accounts in the same script .
* Supports all browsers.
* Total size is less than 100 KB.

Requirements:
* GMailer should works well with PHP >= 4.
* Also it requires the curl extension.
* Because GMailer ALWAYS connects to GMail via SSL, you may need OpenSSL for curl to talk SSL.

Tools for creating TCP/IP packets

hping (http://www.hping.org/)

hping is a command-line oriented TCP/IP packet assembler/analyzer. The interface is inspired to the ping(8) unix command, but hping isn’t only able to send ICMP echo requests. It supports TCP, UDP, ICMP and RAW-IP protocols, has a traceroute mode, the ability to send files between a covered channel, and many other features

Features include:
* Firewall testing
* Advanced port scanning
* Network testing, using different protocols, TOS, fragmentation
* Manual path MTU discovery
* Advanced traceroute, under all the supported protocols
* Remote OS fingerprinting
* Remote uptime guessing
* TCP/IP stacks auditing
* hping can also be useful to students that are learning TCP/IP

Hping works on the following unix-like systems: Linux, FreeBSD, NetBSD, OpenBSD, Solaris, MacOs X, Windows.

Nemesis (http://nemesis.sourceforge.net/)

Nemesis is a command-line network packet crafting and injection utility for UNIX-like and Windows systems. Nemesis, is well suited for testing Network Intrusion Detection Systems, firewalls, IP stacks and a variety of other tasks. As a command-line driven utility, Nemesis is perfect for automation and scripting.

Nemesis can natively craft and inject ARP, DNS, ETHERNET, ICMP, IGMP, IP, OSPF, RIP, TCP and UDP packets. Using the IP and the Ethernet injection modes, almost any custom packet can be crafted and injected.

Features include:
* ARP/RARP, DNS, ETHERNET, ICMP, IGMP, IP, OSPF, RIP, TCP and UDP protocol support
* Layer 2 or Layer 3 injection
* Packet payload from file
* IP and TCP options from file

Scapy (http://www.secdev.org/projects/scapy/)

Scapy is a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more.

It can handle most classical tasks like scanning, tracerouting, probing, unit tests, attacks or network discovery (it can replace hping, 85% of nmap, arpspoof, arp-sk, arping, tcpdump, tethereal, p0f, etc.).

Features include:
* Port Scanning
o SYN Scan
o Other TCP Scans
o UDP Scans
o IP Scan
* Host Discovery
o ARP Ping
o ICMP Ping
o TCP Ping
o UDP Ping
* OS Fingerprinting
o ISN
o nmap_fp
o p0f
o queso
* Sniffer – includes powerful facilities for traffic capture and analysis
* Wireless – can not only sniff and decode packets but also inject arbitrary packets
* Traceroute – standard ICMP Traceroute can be emulated
* Firewall/IDS Testing
o TCP Timestamp Filtering
o NAT Detection
o Firewalking

Yersinia (http://www.yersinia.net)

Yersinia is a framework for performing layer 2 attacks.

It is designed to take advantage of some weakeness in different network protocols. It pretends to be a solid framework for analyzing and testing the deployed networks and systems.

Features include:
* Attacks for the following network protocols are supported
o  Spanning Tree Protocol (STP)
o Cisco Discovery Protocol (CDP)
o Dynamic Trunking Protocol (DTP)
o Dynamic Host Configuration Protocol (DHCP)
o Hot Standby Router Protocol (HSRP)
o 802.1q
o 802.1x
o Inter-Switch Link Protocol (ISL)
o VLAN Trunking Protocol (VTP)

SendIP (http://www.earth.li/projectpurple/progs/sendip.html)

SendIP is a command-line tool to send arbitrary IP packets. It has a large number of options to specify the content of every header of a RIP, RIPng, BGP, TCP, UDP, ICMP, or raw IPv4/IPv6 packet. It also allows any data to be added to the packet. Checksums can be calculated automatically, but if you wish to send out wrong checksums, that is supported too.

packETH (http://packeth.sourceforge.net/)

packETH is a Linux GUI packet generator tool for ethernet. It allows you to create and send any possible packet or sequence of packets on the ethernet.

Features:
* you can create and send any ethernet packet. Supported protocols:
o ethernet II, ethernet 802.3, 802.1q, QinQ
o ARP, IPv4, user defined network layer payload
o UDP, TCP, ICMP, IGMP, user defined transport layer payload
o RTP (payload with options to send sin wave of any frequency for G.711)
* sending sequence of packets
o delay between packets, number of packets to send
o sending with max speed, approaching the teoretical boundary
o change parameters while sending (change IP & mac address, UDP payload, 2 user defined bytes, etc.)
* saving configuration to a file and load from it – pcap format supported

Mausezahn (http://www.perihel.at/sec/mz/)

Mausezahn is a fast traffic generator which allows you to send nearly every possible and impossible packet. Mausezahn can be used, for example, as a traffic generator to stress multicast networks, for penetration testing of firewalls and IDS, for simulating DoS attacks on networks, to find bugs in network software or appliances, for reconnaissance attacks using ping sweeps and port scans, or to test network behavior under strange circumstances. Mausezahn gives you full control over the network interface card and allows you to send any byte stream you want (even violating Ethernet rules).

Mausezahn can be used for example:
* As traffic generator (e. g. to stress multicast networks)
* To precisely measure jitter (delay variations) between two hosts (e. g. for VoIP-SLA verification)
* As didactical tool during a datacom lecture or for lab exercises
* For penetration testing of firewalls and IDS
* For DoS attacks on networks (for audit purposes of course)
* To find bugs in network software or appliances
* For reconnaissance attacks using ping sweeps and port scans
* To test network behaviour under strange circumstances (stress test, malformed packets, …)

…and more. Mausezahn is basically a versatile packet creation tool on the command line with a simple syntax and context help. It could also be used within (bash-) scripts to perform combination of tests.

Commands related to Process control

Here “PID” is refer to the process ID, that you can get from command “ps -aux”

fg PID
Bring a background or stopped process to the foreground.

bg PID
Send the process to the background. Opposite to fg. The same can be accomplished with z. If you have stopped jobs, you have to type exit twice in row to log out.

any_command&
Run any command in the background (the symbol “&” means “run the proceeding command in the background”).

batch any_command
Run any command (usually one that is going to take more time) when the system load is low. I can logout, and the process will keep running.

at 17:00
Execute a command at a specified time. You will be prompted for the command(s) to run, until you press d.

kill PID
Force a process shutdown. First determine the PID of the process to kill using ps.

killall program_name
Kill program(s) by name.

lpc
(as root) Check and control the printer(s). Type “?” to see the list of available commands.

lpq
Show the content of the printer queue. Under KDE (X-Windows), you may use GUI-based “Printer Queue” available from “K”menu-Utilities.

lprm job_number
Remove a printing job “job_number” from the queue.

nice program_name
Run program_name adjusting its priority. Since the priority is not specified in this example, it will be adjusted by 10 (the process will run slower), from the default value (usually 0). The lower the number (of “niceness” to other users on the system), the higher the priority. The priority value may be in the range -20 to 19. Only root may specify negative values. Use “top” to display the priorities of the running processes.

renice -1 PID
(as root) Change the priority of a running process to -1. Normal users can only adjust processes they own, and only up from the current value (make them run slower).

c, z, s, and q also belong to this chapter but they were described previously. In short they mean: stop the current command, send the current command to the background, stop the data transfer, resume the data transfer.

Protection from malware using Squid proxy server

Malware (for “malicious software”) is any program or file that is harmful to a computer user. Thus, malware includes computer viruses, worms, Trojan horses, and also spyware, programming that gathers information about a computer user without permission.

Ignoring the threat of malware is one of the most reckless things you can do in today’s increasingly hostile computing environment. Malware is malicious code planted on your computer, and it can give the attacker a truly alarming degree of control over your system, network, and data – without your knowledge

Now, you can get complete protection from such malware by using/configuring the Malware Block List with squid so that you, your network and your users are protected.

Visit : http://www.malware.com.br for more information of the Malware Block List

To use the Malware Block List on a Squid proxy to block user access to URLs that contain Malware you need to perform the following simple steps:

* Download the block list:
wget -O – http://malware.hiperlinks.com.br/cgi/submit?action=list_squid > malware_block_list.txt

* Create an ACL in the main configuration file (squid.conf) pointing to a file which will have the list URLs:
acl malware_block_list url_regex -i “/etc/squid/malware_block_list.txt”

* Enable the ACL created previously:
http_access deny malware_block_list
deny_info http://malware.hiperlinks.com.br/denied.shtml malware_block_list

* Force Squid reconfiguration:
squid -k reconfigure

To have an up-to-date block list, create a cron job to run every 4 hours, pointing to a script like this:
#!/bin/sh
wget -O – http://malware.hiperlinks.com.br/cgi/submit?action=list_squid > /etc/squid  /malware_block_list.txt
squid -k reconfigure

* Testing  time: After reloading the squid, try to visit the following site (“www.uploadhut.com”), you should see something like ..

If you are able to see this page on your browser — relax, you are now safe from the malware 🙂

/etc/grub.conf explained

The following example shows the structure of a GRUB menu file. The example installation has a Linux boot partition under /dev/sda5, a root partition under /dev/sda7, and a Windows installation under /dev/sda1.

gfxmenu (hd0,4)/boot/message
color white/blue black/light-gray
default 0
timeout 8

title linux
root (hd0,4)
kernel /boot/vmlinuz root=/dev/sda7 vga=791 resume=/dev/sda9
initrd /boot/initrd

title windows
rootnoverify (hd0,4)
chainloader(hd0,0)+1

The first block defines the configuration of the splash screen:

gfxmenu (hd0,4)/boot/message
The background image message is located in the top directory of the /dev/sda5 partition.

color white/blue black/light-gray

Color scheme: white (foreground), blue (background), black (selection), and light gray (background of the selection). The color scheme has no effect on the splash screen, only on the customizable GRUB menu that you can access by exiting the splash screen with Esc.

default 0
The first menu entry title linux is the one to boot by default.

timeout 8
After eight seconds without any user input, GRUB automatically boots the default entry. To deactivate automatic boot, delete the timeout line. If you set timeout 0, GRUB boots the default entry immediately.

The second and last block lists the various bootable operating systems. The sections for the individual operating systems are introduced by title.

The first entry (title linux) is responsible for booting Linux. The kernel (vmlinuz) is located in the first logical partition (the boot partition) of the first hard disk. Kernel parameters, such as the root partition and VGA mode, are appended here. The root partition is specified according to the Linux naming convention (/dev/sda7/), because this information is read by the kernel and has nothing to do with GRUB. The initrd is also located in the first logical partition of the first hard disk.

The second entry is responsible for loading Windows. Windows is booted from the first partition of the first hard disk (hd0,0). The command chainloader +1 causes GRUB to read and execute the first sector of the specified partition.

The menu file can be changed whenever necessary. GRUB then uses the modified settings during the next boot.

How to setup Boot Password (Grub)

Even before the operating system is booted, GRUB enables access to file systems. Users without root permissions can access files in your Linux system to which they have no access once the system is booted. To block this kind of access or prevent users from booting certain operating systems, set a boot password.
As the user root, proceed as follows to set a boot password:

At the root prompt, encrypt the password using grub-md5-crypt:

# grub-md5-crypt
Password: ****
Retype password: ****
Encrypted: $1$lS2dv/$JOcHkLosIn7CJk9xShzzJVw/

Paste the encrypted string into the global section of the file menu.lst:

gfxmenu (hd0,4)/message
color white/blue black/light-gray
default 0
timeout 8
password –md5 $1$lS2dv/$JOcHkLosIn7CJk9xShzzJVw/

Now GRUB commands can only be executed at the boot prompt after pressing P and entering the password. However, users can still boot all operating systems from the boot menu.

To prevent one or several operating systems from being booted from the boot menu, add the entry lock to every section in menu.lst that should not be bootable without entering a password. For example:

title linux
kernel (hd0,4)/vmlinuz root=/dev/sda7 vga=791
initrd (hd0,4)/initrd
lock

After rebooting the system and selecting the Linux entry from the boot menu, the following error message is displayed:

Error 32: Must be authenticated

Press Enter to enter the menu. Then press P to get a password prompt. After entering the password and pressing Enter, the selected operating system (Linux in this case) should boot.

Tuning MySQL Performance with MySQLTuner

MySQLTuner is a Perl script that analyzes your MySQL performance and, based on the statistics it gathers, gives recommendations which variables you should adjust in order to increase performance. That way, you can tune your my.cnf file to tease out the last bit of performance from your MySQL server and make it work more efficiently.

This document comes without warranty of any kind! I do not issue any guarantee that this will work for you!

1 Using MySQLTuner

You can download the MySQLTuner script as follows:

wget http://mysqltuner.com/mysqltuner.pl

In order to run it, we must make it executable:

chmod +x mysqltuner.pl

Afterwards, we can run it. You need your MySQL root password for it:

./mysqltuner.pl

server1:~# ./mysqltuner.pl

>>  MySQLTuner 0.9.8 – Major Hayden <major@mhtx.net>
>>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
>>  Run with ‘–help’ for additional options and output filtering
Please enter your MySQL administrative login: <– root
Please enter your MySQL administrative password: <– yourrootsqlpassword

——– General Statistics ————————————————–
[–] Skipped version check for MySQLTuner script
[!!] Your MySQL version 4.1.11-Debian_etch1-log is EOL software!  Upgrade soon!
[OK] Operating on 32-bit architecture with less than 2GB RAM

——– Storage Engine Statistics ——————————————-
[–] Status: +Archive -BDB -Federated +InnoDB +ISAM -NDBCluster
[–] Data in MyISAM tables: 301M (Tables: 2074)
[–] Data in HEAP tables: 379K (Tables: 9)
[!!] InnoDB is enabled but isn’t being used
[!!] ISAM is enabled but isn’t being used
[!!] Total fragmented tables: 215

——– Performance Metrics ————————————————-
[–] Up for: 12d 18h 33m 30s (1B q [1K qps], 185K conn, TX: 3B, RX: 377M)
[–] Reads / Writes: 78% / 22%
[–] Total buffers: 2.6M per thread and 58.0M global
[OK] Maximum possible memory usage: 320.5M (20% of installed RAM)
[OK] Slow queries: 0% (17/1B)
[OK] Highest usage of available connections: 32% (32/100)
[OK] Key buffer size / total MyISAM indexes: 16.0M/72.3M
[OK] Key buffer hit rate: 99.9%
[OK] Query cache efficiency: 99.9%
[!!] Query cache prunes per day: 47549
[OK] Sorts requiring temporary tables: 0%
[!!] Temporary tables created on disk: 28%
[OK] Thread cache hit rate: 99%
[!!] Table cache hit rate: 0%
[OK] Open file limit used: 12%
[OK] Table locks acquired immediately: 99%
[!!] Connections aborted: 20%

——– Recommendations —————————————————–
General recommendations:
Add skip-innodb to MySQL configuration to disable InnoDB
Add skip-isam to MySQL configuration to disable ISAM
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
When making adjustments, make tmp_table_size/max_heap_table_size equal
Reduce your SELECT DISTINCT queries without LIMIT clauses
Increase table_cache gradually to avoid file descriptor limits
Your applications are not closing MySQL connections properly
Variables to adjust:
query_cache_size (> 16M)
tmp_table_size (> 32M)
max_heap_table_size (> 16M)
table_cache (> 64)

server1:~#

You should carefully read the output, especially the recommendations at the end. It shows exactly which variables you should adjust in the [mysqld] section of your my.cnf (on Debian and Ubuntu the full path is /etc/mysql/my.cnf). Whenever you change your my.cnf, make sure that you restart MySQL. You can then run MySQLTuner again to see if it has further recommendations to improve the MySQL performance. This way, you can optimize MySQL step by step.

How To use Automount Service to import/mount NFS File Systems?

The autofs daemon can be used to mount remote file systems automatically. To do this, add the following entry in the your /etc/auto.master file:

/nfsmounts /etc/auto.nfs

Now the /nfsmounts directory acts as a root for all the NFS mounts on the client if the auto.nfs file is completed appropriately. The name auto.nfs is chosen for sake of convenience—you can choose any name. In the selected file (create it if it does not exist), add entries for all the NFS mounts as in the following example:

localdata -fstype=nfs server1:/data
nfs4mount -fstype=nfs4 server2:/

Activate the settings with rcautofs start. For this example, /nfsmounts/localdata, the /data directory of server1, is then mounted with NFS and /nfsmounts/nfs4mount from server2 is mounted with NFSv4.

How to view Routing Table and Change your default Gateway

The netstat -nr command will provide the contents of the touting table. Networks with a gateway of 0.0.0.0 are usually directly connected to the interface. No gateway is needed to reach your own directly connected interface, so a gateway address of 0.0.0.0 seems appropriate. The route with a destination address of 0.0.0.0 is your default gateway.

Example below:

# netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
172.19.12.0     0.0.0.0         255.255.254.0   U         0 0          0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth0
0.0.0.0         172.19.12.1     0.0.0.0         UG        0 0          0 eth0

Change Your Default Gateway

you’ll need to update your /etc/sysconfig/network file to reflect the change. This file is used to configure your default gateway each time Linux boots.

NETWORKING=yes
HOSTNAME=bigboy
GATEWAY=192.168.1.1