Install Drizzle on Debian lenny

Drizzle is a Free Software/Open Source database management system (DBMS)
that was forked from version 6.0 of the MySQL DBMS. Drizzle is a re-thought and re-worked version of the MySQL kernel designed specifically for high-performance, high-concurrency environments.
Like MySQL, Drizzle has a client/server architecture and uses SQL as its primary command language.
Drizzle is distributed under version 2 of the GNU General Public License.

Drizzle is written in the C++ programming language, and stores its string data in an UTF-8 format.
It is being developed for modern Unix-like operating systems, including Linux,
Mac OS X, and Solaris – in general, any OS that conforms to POSIX.

I decided to try to get drizzle up on my Debian Lenny, normally you can install it with apt but
Drizzle is not yet present in the Debian repository.
Well, to install it please follow this steps, first make sure that all build dependencies are met:

protobuf : Protocol Buffers – Google’s data interchange format
Protocol Buffers are a way of encoding structured data in an efficient yet extensible format.

libdrizzle : is the client and protocol library that is required to build Drizzle.

libuuid : generates and parses 128-bit universally unique ids (UUIDs).
See RFC 4122 for more information.

libreadline5 : The GNU readline library aids in the consistency of user interface
across discrete programs that need to provide a command line interface.

libpcre3 : This is a library of functions to support regular expressions whose syntax
and semantics are as close as possible to those of the Perl 5 language.

You can get them with aptitude like:
$ sudo aptitude install libprotobuf-dev protobuf-compiler uuid-dev libdrizzle0 libdrizzle-dev libreadline5-dev libpcre3-dev

Download the last source code archives for the Drizzle database server:
$ wget http://launchpad.net/drizzle/trunk/aloha/+download/drizzle-2009.07.1088.tar.gz
$ tar xzf drizzle-2009.07.1088.tar.gz
$ cd drizzle-2009.07.1088
$ ./configure –with-libprotobuf-prefix=/usr/local/drizzle
$ make
$ sudo make install

Create a non-privileged group and user for the database server, as below:
$ sudo groupadd drizzle
$ sudo useradd -g drizzle drizzle

Create data directory and set the right permission on it:
$ sudo mkdir /usr/local/drizzle/data
$ sudo chown -R drizzle:drizzle /usr/local/drizzle

Then, Start up the Drizzle server by this command
$ sudo ./usr/local/drizzle/sbin/drizzled –user=drizzle –datadir=/usr/local/drizzle/data/ &

Run drizzle client, which is very similar to the MySQL client.
Start it up and access the Drizzle server as below:

$ ./usr/local/drizzle/bin/drizzle
Welcome to the Drizzle client.. Commands end with ; or \g.
Your Drizzle connection id is 2
Server version: 2009.07.1088 Source distribution (trunk)

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer.

drizzle> \h
List of all Drizzle commands:
Note that all text commands must be first on line and end with ‘;’
? (\?) Synonym for `help’.
clear (\c) Clear command.
connect (\r) Reconnect to the server. Optional arguments are db and host.
delimiter (\d) Set statement delimiter. NOTE: Takes the rest of the line as new delimiter.
ego (\G) Send command to drizzle server, display result vertically.
exit (\q) Exit drizzle. Same as quit.
go (\g) Send command to drizzle server.
help (\h) Display this help.
nopager (\n) Disable pager, print to stdout.
notee (\t) Don’t write into outfile.
pager (\P) Set PAGER [to_pager]. Print the query results via PAGER.
print (\p) Print current command.
prompt (\R) Change your drizzle prompt.
quit (\q) Quit drizzle.
rehash (\#) Rebuild completion hash.
source (\.) Execute an SQL script file. Takes a file name as an argument.
status (\s) Get status information from the server.
tee (\T) Set outfile [to_outfile]. Append everything into given outfile.
use (\u) Use another database. Takes database name as argument.
warnings (\W) Show warnings after every statement.
nowarning (\w) Don’t show warnings after every statement.

drizzle> select version();
+————–+
| version() |
+————–+
| 2009.07.1088 |
+————–+
1 row in set (0 sec)

drizzle> exit;

have fun!

cpan recent modules

This command grabs and print the URL of a plain text file that lists new files added to CPAN the past two weeks.
$ perl -MLWP::Simple -e “getprint ‘http://cpan.org/RECENT'”

You can easily make it part of a tidy little shell command, like this one that mails you the list of new SOAP::modules
$ perl -MLWP::Simple -e “getprint ‘http://cpan.org/RECENT'” | grep “/by-module/SOAP” | mail -s “New SOAP module! ” $USER

Using bash variables in awk

First you need to assign the variables with the -v option:
The option -v followed by var=value is an assignment to be done before (the awk program).

Example of use:
This example define a bash file, that look for any entry “address” into the squid’s access file.

$ cat > search
#!/bin/bash
awk -v address=”$1″ ‘$3 == address && $4 ~ /200/’ /var/log/squid/access.log

$ chmod +x search; sudo ./search 192.168.0.2

Nagios jabber notification

Nagios can be configured to send notifications by various ways email “the default notification for Nagios”, sms and jabber …
It’s very flexible enough to let you develop reel solutions.
In this tutorial we will focus on how to configure nagios to send notification to your jabber account using twisted.

First install python-twisted interface.
On Debian and derive like Ubuntu use apt like this:
$ sudo apt-get install python-twisted

You can check twisted website for more informations : http://twistedmatrix.com/

Later, append to contacts_nagios2.cfg the notifications method notify-host-by-jabber and notify-service-by-jabber
Also, set up a new paramter address1 that must contain your(s) jabber account.

$ sudo vim /etc/nagios3/conf.d/contacts_nagios2.cfg
define contact{
contact_name root
alias Root
service_notification_period 24×7
host_notification_period 24×7
service_notification_options w,u,c,r
host_notification_options d,r
service_notification_commands notify-service-by-email,notify-service-by-jabber
host_notification_commands notify-host-by-email,notify-host-by-jabber
email mezgani@nivose.securfox.org
address1 mezgani@secufox.org
pager +212669119530
}

And let’s define the command that will be interpreted when nagios call notify-host-by-jabber and notify-service-by-jabber
So change /home/mezgani/script/fdbot.py by the path of your python bot, the output of the command will be redirected
to the log file /var/log/nagios3/jabber.log
Add the following to your /etc/nagios3/commands.cfg.

$ sudo vim /etc/nagios3/commands.cfg

# ‘notify-host-by-jabber’ command definition
define command{
command_name notify-host-by-jabber
command_line /home/mezgani/script/fdbot.py “$CONTACTADDRESS1$” “$NOTIFICATIONTYPE$ : Host $HOSTALIAS$ is $HOSTSTATE$ ($OUTPUT$)” > /var/log/nagios3/jabber.log
}

# ‘notify-service-by-jabber’ command definition
define command{
command_name notify-service-by-jabber
command_line /home/mezgani/script/fdbot.py “$CONTACTADDRESS1$” “$NOTIFICATIONTYPE$ : $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ ($OUTPUT$)” > /var/log/nagios3/jabber.log
}

Well, here i define the jabber bot named fdbot, it is a python script based on twisted framework that transforms your message to a valid jabber message.
You can choose your script language it can be done with perl using ‘Net::XMPP’, with ruby using ‘xmpp4r’, and others.
my choise of python was just for my need to learn twisted framework.

You can Get fdbot by git like:
$ git clone git://github.com/mezgani/fdbot.git
set up the undefined variables in the fdbot.py script.

and restart nagios:
$ sudo /etc/init.d/nagios3 restart

The source code at Github. Go visit the Github project page for fdbot.

Install ntop on Red Hat Enterprise Linux / CentOS Linux / Debian

ntop is a network probe that shows network usage in a way similar to what top does for processes,
dnstop for dns, ptop for postgresql, iotop, atop …

In interactive mode, it displays the network status on the user’s terminal. In Web mode, it acts as a web server, creating a HTML dump of the network status. It sports a NetFlow/sFlow emitter/collector, a HTTP-based client interface for creating ntop-centric monitoring applications, and RRD for persistently storing traffic statistics.

ntop is available for both Unix and Win32-based platforms. It has been developed by Luca Deri, an Italian research scientist and network manager at University of Pisa.

On Redhat or Centos:
For installing ntop on Red Hat or Centos, please follow these instructions:

Packages needed :
– libpcap: system interface for user-level packet capture
– libpcap-devel: development library for libpcap
– rrdtool: Time-series data storage and display system (programs)
– rrdtool-devel: Time-series data storage and display system (development)
– perl-rrdtool: Time-series data storage and display system (perl interface)
– graphviz: Graph Visualization Software

First install libpcap, and libpcap-devel using yum
$ sudo yum install libpcap-devel libpcap

The rrdtool, ntop and graphviz packages you can get them like this:


$ wget ftp://rpmfind.net/linux/dag/redhat/el5/en/i386/dag/RPMS/rrdtool-devel-1.2.30-1.el5.rf.i386.rpm
$ wget ftp://rpmfind.net/linux/dag/redhat/el5/en/i386/dag/RPMS/rrdtool-1.2.30-1.el5.rf.i386.rpm
$ wget ftp://rpmfind.net/linux/dag/redhat/el5/en/i386/dag/RPMS/perl-rrdtool-1.2.30-1.el5.rf.i386.rpm
$ wget http://dag.wieers.com/rpm/packages/ntop/ntop-3.3-1.el5.rf.i386.rpm

$ wget http://www.graphviz.org/graphviz-rhel.repo
$ sudo cp graphviz-rhel.repo /etc/yum.repos.d/
$ sudo yum update
$ sudo yum install graphviz

Installing perl-rrdtool, rrdtool, rrdtool-devel and ntop with rpm package manager :
$ sudo rpm -ivh perl-rrdtool-1.2.30-1.el5.rf.i386.rpm rrdtool-1.2.30-1.el5.rf.i386.rpm rrdtool-devel-1.2.30-1.el5.rf.i386.rpm ntop-3.3-1.el5.rf.i386.rpm

Type the following command to set ntop admin password, enter:
$ ntop -A
Sat Jul 4 14:31:49 2009 NOTE: Interface merge enabled by default
Sat Jul 4 14:31:49 2009 Initializing gdbm databases
ntop startup – waiting for user response!
Please enter the password for the admin user:
Please enter the password again:
Sat Jul 4 14:32:24 2009 Admin user password has been set

The following command start ntop:
$ sudo /usr/bin/ntop -d -L -u ntop -P /var/ntop –skip-version-check –use-syslog=daemon -d

If you have many interfaces you can specify them like this :
$ /usr/bin/ntop -i “eth0,eth1” -d -L -u ntop -P /var/ntop –skip-version-check –use-syslog=daemon -d

Well we need to run ntop as services, please refer to /etc/ntop.conf:
$ sudo vim /etc/ntop.conf
–user ntop
–db-file-path /var/ntop
–interface eth0,eth1
–use-syslog
–http-server 3000
–https-server 3001
–local-subnets 192.168.1.0/20
–domain mydomain.org
–daemon

At last start your ntop service:
$ sudo service ntop start

By default ntop listen on 3000 port. You can view ntop stats by visiting following url:
http://server:3000/ or https://server:3001/
as defined in the ntop.conf.

On Debian:
The ntop package is present in the Debian repository, so you can get simply by:
$ aptitude install ntop

Personalize your ntop config file. After, you can start the service :
$ sudo invoke-rc.d ntop start

pv – Pipe viewer

pv – Pipe viewer is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.

Here there are some example of using this tool:

Example 1: Copying file
$ pv file > /path/to/dir/file

Example 2: Decompess file
$ pv data.tar.bz2 |tar xjf –

Example 3: Backing up a PostgreSQL database
$ pg_dump data | pv -t | bzip2 > data.bz2

Example 4: File remote copy
# on computer foo, with IP address 192.168.0.1
$ tar -cf – /path/to/dir | pv | nc -l -p 6666 -q 5

# on computer bar
$ nc 192.168.0.1 6666 | pv | tar -xf –

Example 5: Switching a compress file format from bzip2 to zip
$ pv -cN source < file.tar.bz2 | bzcat | pv -cN bzcat | gzip -9 | pv -cN gzip > file.tar.gz

Example 6: Generate a compressed archive of the current directory
$ (tar cvf – . 2>log | pv -n -s `du -sb . | awk ‘{ print $1 }’` | gzip > out.tgz) 2>&1 | dialog –title “Archiving” –gauge ‘Progress’ 9 60

And there are also, many useful tips with pv, for more usage please refer to manual pages.

If you want to get pv on Debian or Debian based system such as Ubuntu do the following:
$ sudo aptitude install pv

If you’re on Fedora do:
$ sudo yum install pv