Upgrading PostgreSQL

To upgrade a PostgreSQL installation start by dumping all the existing databases. You should do this before stopping the current database server
$ PGUSER=postgres /usr/bin/pg_dumpall > dump.sql
$ sudo /etc/init/d/postgresql stop
$ sudo mv /usr/local/pgsql /usr/local/pgsql.old

Build and install your new version. Be sure to create a PGDATA directory that is owned by the postgres user. Initialize the PGDATA directory with initdb
$ sudo mkdir /usr/local/pgsql/data
$ sudo chown postgres:postgres /usr/local/pgsql/data
$ sudo -u postgres /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data

Once you have created your new database, start the new database server and populate the new database with the data you dumped from your old system:

$ sudo -u postgres /usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data
$ sudo -u postgres /usr/local/pgsql/bin/psql -d postgres -f dump.sql

Cisc vs Risc

CISC (Complex Instruction Set Computers)
The primary goal of CISC architecture is to complete a task in as few lines of assembly as possible.
One of the primary advantages of this system is that the compiler has to do very little work to translate a high-level language statement into assembly. Because the length of the code is relatively short, very little RAM is required to store instructions. The emphasis is put on building complex instructions directly into the hardware.
Examples of CISC processors are the System/360 (excluding the ’scientific’ Model 44), VAX, PDP-11, Motorola 68000 family, and Intel x86 architecture based processors.

 

Characteristics of a CISC architecture:
* Emphasis on hardware;
* Includes multi-clock complex instructions;
* Small code sizes;
* Complex data types in hardware; some CISCs have byte string instructions, or support complex numbers;

RISC (Reduced instruction set computer) is a type of microprocessor architecture that utilizes a small, highly-optimized set of instructions, rather than a more specialized set ofinstructions often found in other types of architectures.
RISC processors only use simple instructions that can be executed within one clock cycle.
Well known RISC families include Alpha, ARC, ARM, AVR, MIPS, PA-RISC, Power Architecture (including PowerPC), SuperH, and SPARC.

Characteristics of a RISC architecture:
* Emphasis on software;
* Single-clock, reduced instruction only;
* Uniform instruction format, using a single word with the opcode in the same bit positions in every instruction, demanding less decoding;
* Identical general purpose registers, allowing any register to be used in any context, simplifying compiler design;
* Simple addressing modes. Complex addressing performed via sequences of arithmetic and/or load-store operations;
* Typically larger code sizes;
* Few data types in hardware;

The CISC approach attempts to minimize the number of instructions per program, sacrificing the number of cycles per instruction. RISC does the opposite, reducing the cycles per instruction at the cost of the number of instructions per program.

Sniffing USB port on Linux

The linux kernel has a facility called “usbmon” which can be used to sniff the USB bus.
So the output is really easy to collect, even from the command line shell or using libpcap parser like tcpdump or wireshark.
Let’s see how we can do that:
Simply mount debugfs and load the usbmon module, if usbmon is built into the kernel.
$ sudo mount -t debugfs none_debugs /sys/kernel/debug
$ sudo modprobe usbmon
Verify that bus sockets are present.
$ sudo ls /sys/kernel/debug/usbmon
0s 0u 1s 1t 1u 2s 2t 2u 3s 3t 3u 4s 4t 4u

Now you can choose to either use the socket ‘0u’ (to capture packets on all buses),
or find the bus used by your device, so run
$ sudo cat /proc/bus/usb/devices
or may be
$ sudo lsusb
and find the T-line which corresponds to the device.
If my device is connected to the forth bus
Start ‘cat’ like this
$ sudo cat /sys/kernel/debug/usbmon/4u
With wireshark it’s more verbose so run, wireshark and select usbmon4 usb bus 4.

Happy sniffing,

Code names of Debian

When a Debian distribution is in the development stage, it has no version number but a codename. The purpose of these code names is to make easier the mirroring of the Debian distributions.
The code names of Debian releases are names of characters from the film Toy Story. The unstable, development distribution is permanently nicknamed sid, after the emotionally unstable next-door neighbor boy who regularly destroyed toys.

Code names of Debian

Howto setup IPMI under Linux (RHEL/CentOS 5) on the HP BL460c (Intel(R) Xeon(R) E5430)

IPMI is a standardized message-based hardware management interface that provides a foundation for full monitoring of system health and management of the system, both locally and remotely in a secure manner.
The hardware chip that implements the core of IPMI is known as the Baseboard Management Controller (BMC), or Management Controller (MC).
And there are many companies that support IPMI. Dell, HP, Intel Corporation and NEC Corporation …

In our case and In practice, we will use the hp-OpenIPMI and hpasm drivers with RHEL/CentOS 5,
you must install the “kernel-devel” package and make sure its version matches the “kernel”
package you’re currently using. The “kernel-devel” package contains the necessary .config file for building kernel modules
(which is what the hp-OpenIPMI package wants to do).
$ sudo yum install kernel-devel

Fetch the HP OpenIPMI device drivers package and install it:
$ wget ftp://ftp.hp.com/pub/softlib2/software1/pubsw-linux/p402764200/v49065/hp-OpenIPMI-8.3.0-69.rhel5.i386.rpm
$ sudo rpm -ivh hp-OpenIPMI-8.3.0-69.rhel5.i386.rpm

The modules installed by the hp-OpenIPMI driver are :
ipmi_devintf.ko, ipmi_msghandler.ko, ipmi_poweroff.ko, ipmi_si.ko, ipmi_watchdog.ko

Install OpenIPMI the open source IPMI client:
$ sudo yum install OpenIPMI.i386

Create a patch file with this contenent:
$ cat > ipmi.patch

*** ipmi 2009-10-01 10:05:37.000000000 +0000
— /etc/init.d/ipmi 2009-09-30 10:53:20.000000000 +0000
***************
*** 335,341 ****
modprobe -q -r ${m} > /dev/null 2>&1
done
# delete interface node ONLY if ipmi_devintf is unloaded
! [ `lsmod | grep -c "ipmi_devintf"` -eq 0 ] &&
rm -f “/dev/ipmi${INTF_NUM}”
}

— 335,341 —-
modprobe -q -r ${m} > /dev/null 2>&1
done
# delete interface node ONLY if ipmi_devintf is unloaded
! [ `/sbin/lsmod | grep -c "ipmi_devintf"` -eq 0 ] &&
rm -f “/dev/ipmi${INTF_NUM}”
}

***************
*** 345,353 ****
modprobe -q -r ${m} > /dev/null 2>&1
done
# delete interface node ONLY if ipmi_devintf is unloaded
! [ `lsmod | grep -c "ipmi_devintf"` -eq 0 ] &&
rm -f “/dev/ipmi${INTF_NUM}”
! lsmod | egrep -q “ipmi_(poweroff|watchdog)” > /dev/null 2>&1
if [ "$?" -ne "0" ]; then
stop_watchdog_quiet
stop_powercontrol_quiet
— 345,353 —-
modprobe -q -r ${m} > /dev/null 2>&1
done
# delete interface node ONLY if ipmi_devintf is unloaded
! [ `/sbin/lsmod | grep -c "ipmi_devintf"` -eq 0 ] &&
rm -f “/dev/ipmi${INTF_NUM}”
! /sbin/lsmod | egrep -q “ipmi_(poweroff|watchdog)” > /dev/null 2>&1
if [ "$?" -ne "0" ]; then
stop_watchdog_quiet
stop_powercontrol_quiet
***************
*** 409,415 ****
— 409,424 —-

#############################################################################
start()
+
{
+ maj=`cat /proc/devices | awk ‘/ipmidev/{print $1}’`
+ if [ -c /dev/ipmi0 ]
+ then
+ rm -f /dev/ipmi0
+ /bin/mknod /dev/ipmi0 c $maj 0
+ else
+ /bin/mknod /dev/ipmi0 c $maj 0
+ fi
echo -n “Starting ${MODULE_NAME} drivers: “
load_ipmi_modules
if [ ${RETVAL} -eq 0 ]; then

To apply the patch to the ipmi file, change to the directory where the file is located:
$ cd /etc/init.d
$ sudo patch -p1 -i ~/ipmi.patch
Launch the IPMI service, which will load the kernel modules for you
$ sudo /etc/init.d/ipmi start

Fetch the HP System Health Application and Insight Management Agents package for Red Hat Enterprise Linux 5 (x86) like this:
$ wget ftp://ftp.hp.com/pub/softlib2/software1/pubsw-linux/p1925054526/v42992/hpasm-8.0.0-173.rhel5.i386.rpm
Let installed it with rpm and configure it before
$ sudo rpm -ivh hpasm-8.0.0-173.rhel5.i386.rpm
$ sudo /etc/init.d/hpasm configure

Start the hpasm agents using the command
$ sudo /etc/init.d/hpasm start

There are two ways to configure the BMC. You can configure it through the boot-time menu (Ctrl-E), where you can set the management password and IP address information. Or, you can configure it with ipmitool from the OS. We will configure it from OS so create a custom ipmi_lan.conf with this content
$ cat /etc/ipmi_lan.conf
addr 0.0.0.0
priv_limit admin
allowed_auths_callback md5
allowed_auths_user md5
allowed_auths_operator md5
allowed_auths_admin md5
user 2 true “username” “password” admin 1 md5

Start the ipmi daemon that will listen in all interfaces and will bind the 623 port by default.
$ sudo ipmilan -c /etc/ipmi_lan.conf

Add rule to the firewall if it’s running, this run tell to iptables to make the udp port 623 listening for connection coming from 192.168.0.18:
$ sudo vim /etc/sysconfig/iptables
-A RH-Firewall-1-INPUT -m state –state NEW -m udp -s 192.168.0.18 -p udp –dport 623 -j ACCEPT

Finally, to view the chassis status from a remote server, use the following command from your remote machine :
In this example from my Debian i do this :
$ ipmitool -I lan -U username -H 192.168.0.2 -a chassis status
Password:
System Power : on
Power Overload : false
Power Interlock : inactive
Main Power Fault : false
Power Control Fault : false
Power Restore Policy : always-on
Last Power Event :
Chassis Intrusion : inactive
Front-Panel Lockout : inactive
Drive Fault : false
Cooling/Fan Fault : false
Front Panel Control : none

Note the ‘-H’ option indicates the remote machine, ‘-U’ option parameter is the ‘userid’ name. The ‘-a’ option indicates to prompt for password.
This IPMItool command ‘chassis command’ result some information and the status of the machine’s system
Note that you cannot only get the status of the machine, you can also power it on and off, reset, adjust fan speeds etc …

For IPMI Problems please refer to https://twiki.cern.ch/twiki/bin/view/FIOgroup/IpmiPrb

Clone the same Debian packages from another target

Sometimes we need to clone machines, for such use we may refer to rsync, and when we talk about debian package the debian package manager dpkg comes with some famous options that make our clone procedure very easy :
Let’s take an example :
So we have to clone the machine nibiru (machine to restore) from the machine maya (machine to backup):
Well On maya run:
$ sudo dpkg –get-selections > /tmp/dpkg.txt
Copy the dpkg.txt file to the target to restore system and run:
$ scp /tmp/dpkg.txt mezgani@nibiru:~/

On nibiru:
$ sudo dpkg –set-selections < ~/dpkg.txt
$ sudo apt-get -y update
$ sudo apt-get dselect-upgradeet

How to find the PostgreSQL database size

PostgreSQL is a powerful, open source relational database system. It has more than 15 years of active development and a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness.
If you want to find the size of a database, in our example the sqview database, please tape:

sqview-# SELECT pg_database_size(’sqview’);
pg_database_size
——————
111415944
(1 ligne)

The result in octal, for a pretty print we will use the pg_size_pretty function which converts the size in bytes to human understandable format.
sqview-# SELECT pg_size_pretty(pg_database_size(’sqview’));
pg_size_pretty
—————-
106 MB
(1 ligne)

Well if we need to get the size of all databases
sqview-# SELECT pg_database.datname, pg_size_pretty(pg_database_size(pg_database.datname)) AS size FROM pg_database;
datname | size
———–+———
postgres | 3537 kB
template1 | 3537 kB
template0 | 3480 kB
freedom | 25 MB
anakeen | 6081 kB
sqview | 106 MB
rt | 8201 kB
(7 lignes)

PostgreSQL can also give the size of a table ‘users’ like this

sqview-# SELECT pg_size_pretty(pg_relation_size(‘users’));
pg_size_pretty
—————-
64 kB
(1 ligne)

This value exclude indexes and some auxiliary data.
If you want to include them use pg_total_relation_size instead of pg_relation_size as shown below.
sqview-# SELECT pg_size_pretty(pg_total_relation_size(‘users’));
pg_size_pretty
—————-
152 kB
(1 ligne)

to find the largest table in the postgreSQL database.
sqview-# SELECT relname, relpages FROM pg_class ORDER BY relpages DESC limit 1;
relname | relpages
———+———-
sqview | 9666
(1 ligne)

Plotting CPU temperature with gnuplot

Good sysadmins take a look daily to log, and they collect a lot of logs, the most of the information collected consists of time series data:
a set of numerical values associated with a sequence of discrete time values.
There are many number of tools for monitoring this data visually as it is collected, most of them are built using the exellent RRDTool, like cacti …
In this article, I’ll introduce you to Gnuplot, focusing specifically on how to plot time series data.
Gnuplot is a portable command-line driven interactive data and function plotting utility.
It has been supported and under development since 1986. Gnuplot supports many types of plots in either 2D and 3D.
It can draw using lines, points, boxes, contours, vector fields, surfaces, and various associated text. It also supports various specialized plot types.

Let’s start our example that is based on reading CPU Temperature Sensor.
We will use lm_sensor a hardware monitoring tool.
Create a cron entry that collect continuously the CPU temperature into a log file /var/log/sensors.log.
(echo `date +%b-%d-%H:%M:%S` | tr -d ‘\ 012′ ; echo -n ‘ ‘; sensors | awk ‘/CPU Temp:/{ print $3 }’ )>> /var/log/sensors.log

After create a file with this contents.
$ cat > temper.conf
set terminal png size 620,360
set xdata time
set timefmt “%b-%d-%H:%M:%S”
set output “temper.png”
# time range must be in same format as data file
set xrange ["AUG-25-00:00:00":"SEP-05-00:00:00"]
set yrange [0:60]
set grid
set xlabel “Date-Time”
set ylabel “°C”
set title “CPU temperature”
set key left box
plot “/var/log/sensors.log” using 1:2 index 0 title “aldebaran” with lines

Wait for a while for collecting data and tape the following command that will end up with a file called ‘temper.png’.
Use your favorite image viewer to take a look.
$ cat temper.conf | gnuplot

Monitoring Hard Disks with SMART on CentOS

In this article i explain how to use smartmontools’ smartctl utility and smartd dæmon to monitor the health of a system’s disks. I have a HP server that come with a hard drive HP seagate DG072BB975 unfortunately that is not yet supported by hddtemp.
The server uses SAS (Serial Attached SCSI) HP drives, so the device is mounted in /dev/cciss/c0d0

First install smartmontools.
$ sudo yum install smartmontools

Looking to our partitions system.
$ cat /proc/partitions
major minor #blocks name

104 0 71652960 cciss/c0d0
104 1 104391 cciss/c0d0p1
104 2 71545477 cciss/c0d0p2
253 0 69500928 dm-0
253 1 2031616 dm-1
That appear that there some cciss (Compaq Smart Array Controller) interface good.

So, the HP server come with 2 disks and by default there are connected to the 0 and 1 channel.
first HDD is on -d cciss,0 and the second on the -d cciss,1

Well, to get smartctl working, you need to specify device/bus type and the disk number, like this:
$ sudo /usr/sbin/smartctl -H -d cciss,0 /dev/cciss/c0d0

smartctl version 5.38 [i686-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

SMART Health Status: OK

this output shows the results of the health status inquiry. This is the one-line Executive Summary Report of disk health; the disk shown here has passed. If your disk health status is FAILING, back up your data immediately.
For a full report you can set the option –all like this:
$ sudo /usr/sbin/smartctl –all -d cciss,0 /dev/cciss/c0d0

smartctl version 5.38 [i686-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Device: HP DG072BB975 Version: HPDC
Serial number: 3NP3FWC200009917UFB3
Device type: disk
Transport protocol: SAS
Local Time is: Fri Aug 14 02:30:17 2009 GMT
Device supports SMART and is Enabled
Temperature Warning Enabled
SMART Health Status: OK

Current Drive Temperature: 26 C
Drive Trip Temperature: 68 C
Elements in grown defect list: 0
Vendor (Seagate) cache information
Blocks sent to initiator = 3368064691
Blocks received from initiator = 4000083366
Blocks read from cache and sent to initiator = 1897459483
Number of read and write commands whose size segment size = 0
Vendor (Seagate/Hitachi) factory information
number of hours powered up = 5259.38
number of minutes until next internal SMART test = 46

Error counter log:
Errors Corrected by Total Correction Gigabytes Total
ECC rereads/ errors algorithm processed uncorrected
fast | delayed rewrites corrected invocations [10^9 bytes] errors
read: 0 0 0 0 0 0.000 0
write: 0 0 0 0 0 0.000 0

Non-medium error count: 0
No self-tests have been logged
Long (extended) Self Test duration: 1070 seconds [17.8 minutes]

Configuring smartd:
Edit your smartd.conf file to add the disks you need. Ensure you edit the correct file (/etc/smartd.conf) and add the following lines:
$ cat >> etc/smartd.conf
/dev/cciss/c0d0 -d cciss,0 -H -m root@domain.tld
/dev/cciss/c0d0 -d cciss,1 -H -m root@domain.tld

Replace the email with your own so when a disk fails it will contact you directly.
and start smart daemon.
$ sudo /etc/init.d/smartd start

When you work on a group that runs a large computing cluster with many nodes and many disk drives, the use of SMART become very interesting really when it could help reduce downtime and keep the cluster operating more reliably.

Filesystems (jfs, xfs, ext3) comparison on Debian

File system is a method for storing and organizing computer files and the data they contain to make it easy to find and access them and there are a lot of type of file systems : shared file systems, database file systems, network file systems, disk file systems …
In Linux there are many disk file systems available, i felt the need to know more about some file systems performance on my old box, so i decided to do a benchmark essay.
In this studies we will look at the disk file systems and specially JFS (The IBM Journaled File System), XFS and ext3 (Third Extended File system).
This benchmark essay is based on some real-world tasks appropriate for a file server with older generation hardware (Pentium 4, IDE hard-drive).
I used an advanced automated tool named Bonnie++, a benchmark suite that is aimed at performing a number of simple tests of hard drive and file system performance. you can find more information about it at http://www.coker.com.au/bonnie++/

Description of Hardware used at tests:

processor : Intel(R) Pentium(R) 4 2xCPU 3.00GHz
RAM : 1287404 kB
Controller : 82801EB/ER (ICH5/ICH5R) IDE Controller
Hard drive:
$ sudo /usr/sbin/smartctl -i /dev/hdb

=== START OF INFORMATION SECTION ===
Model Family: Seagate Maxtor DiamondMax 20
Device Model: MAXTOR STM3802110A
Serial Number: 9LR4B9T3
Firmware Version: 3.AAJ
User Capacity: 80 026 361 856 bytes
Device is: In smartctl database [for details use: -P show]
ATA Version is: 7
ATA Standard is: Exact ATA specification draft version not indicated
Local Time is: Sat Aug 8 19:10:53 2009 UTC
SMART support is: Available – device has SMART capability.
SMART support is: Enabled

OS:
$ cat /etc/issue; uname -a
Debian GNU/Linux squeeze/sid
Linux aldebaran 2.6.29 #2 SMP Thu Apr 2 20:37:46 UTC 2009 i686 GNU/Linux

All optional daemons killed (crond,sshd,httpd,…)
In this essay i used 3 partitions with 26GB for each one, the first partitions contains the jfs file systems the second contains the xfs file systems and the third contains ext3 file systems.

$ sudo parted /dev/hdb
GNU Parted 1.8.8
Using /dev/hdb
Welcome to GNU Parted! Type ‘help’ to view a list of commands.
(parted) p
Model: MAXTOR STM3802110A (ide)
Disk /dev/hdb: 80,0GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 17,4kB 26,0GB 26,0GB jfs jfs
2 26,0GB 52,0GB 26,0GB xfs xfs
3 52,0GB 78,0GB 26,0GB ext3 ext3

I install the file systems on each partition :

$ sudo jfs_mkfs /dev/hdb1
$ sudo mkfs.xfs /dev/hdb2
$ sudo mkfs.ext3 /dev/hdb3

I make directories and i mount them :
$ mkdir jfs xfs ext3
$ sudo mount /dev/hdb1 jfs/
$ sudo mount /dev/hdb2 xfs/
$ sudo mount /dev/hdb3 ext3/

Later i rund bonnie++ this :
$ for i in *; do sudo /usr/sbin/bonnie++ -u 1000:1000 -d `pwd`/$i -q > $i.csv ; sleep 20; done
sleeping time is for freeing the busy resources, thanks bortzmyer for the tips
And i get this great result detailed

We can see that there are not big difference between these filesystem, even JFS is useless of cpu and has better output/intput usage.
JFS may not have been marketed as effectively as other alternatives, but is a solid choice in the long list of quality Linux filesystems.