Monday, November 7, 2011

Monitoring VMware ESXi and vSphere with Nagios

Requirements
  • Perl 5.8
  • Several supporting Perl modules:
    • Crypt-SSLeay (0.51) [Crypt::SSLeay]
    • Data-Dumper (2.102) [Data::Dumper]
    • MethodMaker (2.0.8) [Class::MethodMaker]
    • XML-LibXML (1.60) [XML::LibXML]
    • libwww-perl (5.805) [LWP]
This article describes how to monitor a VMWare ESXi or vSphere host with Nagios, using the OP5 Check ESX Plugin written in PERL. The plugin can monitor either a single ESXi/vSphere server or a VirtualCenter/vCenter Server and individual virtual machines. We’ll see here how to monitor an ESXi 4 host.
The following tutorial has been made on a CentOS server, you may have to adapt some paths with other distributions.

Install all the dependencies of Perl SDK.

perl -MCPAN -e shell

The above command shall bring you to the perl cpan install CLI. This will allow you to install requirements for Perl.

Example:

cpan> install
Crypt::SSLeay

This article describes how to monitor a VMWare ESXi or vSphere host with Nagios, using the OP5 Check ESX Plugin written in PERL. The plugin can monitor either a single ESXi/vSphere server or a VirtualCenter/vCenter Server and individual virtual machines. We’ll see here how to monitor an ESXi 4 host.

The following tutorial has been made on a CentOS server, you may have to adapt some paths with other distributions.

Installation

The prerequisite for the plugin to work is to install the VMWare Perl SDK available on the manufacturer website.
Download the file on your server, for example in the root directory, untar it and run the installer that way :

# cd /root # tar xvzf VMware-vSphere-Perl-SDK-4.1.0-254719.i386.tar.gz  # cd vmware-vsphere-cli-distrib/ # ./vmware-install.pl


"Creating a new vSphere CLI installer database using the tar4 format.

Installing vSphere CLI.

You must read and accept the vSphere CLI End User License Agreement to continue.
Press enter to display it."

"Read through the License Agreement"

"Do you accept? (yes/no)"

yes


"In which directory do you want to install the executable files?
[/usr/bin]"


"The following Perl modules were found on the system but may be too old to work
with vSphere CLI:

Crypt::SSLeay
Compress::Zlib

The installation of vSphere CLI 4.0.0 build-161974 for Linux
completed successfully. You can decide to remove this software from your system
at any time by invoking the following command:
"/usr/bin/vmware-uninstall-vSphere-CLI.pl".

Enjoy,

--the VMware team"

If ever you're having an issue installing the SDK and complaining about http_proxy, issue the following command:

Run commands before vmware-install.pl:

export http_proxy=

export ftp_proxy=

Download nagios check plugin check_esx3.pl from op5.com

http://www.op5.org/community/plugin-inventory/op5-projects/op5-plugins

Follow the instructions given by the script. Depending on your setup, some PERL dependencies must be installed prior for the SDK to work correctly. When it’s done, we can get the plugin here, and copy it to /usr/lib/nagios/plugins/. Make it executable :

# cd /usr/lib/nagios/plugins/ # chmod a+x check_esx

Take NOTE: change check_esx to check_esx3.pl

Configuration

Now, we can start the real configuration for Nagios. We’ll need a username and password to access the ESXi host, let’s define those Nagios variables in a safe place in /etc/nagios/resource.cfg, so that this information will be hidden from the CGIs :

$USER11$=username $USER12$=password

In this tutorial, we’ll be monitoring these resources : CPU, memory usage, net usage, runtime status and IO/read/write. But some more are available, see the references here. Below are the new commands related to ESXi to add in the /etc/nagios/objects/command.cfg file (these are the ESXi related commands only, NOT the full command.cfg, you may append this at the end of the file) :

# check vmware esxi machine # check cpu define command{         command_name check_esx_cpu         command_line $USER1$/check_esx -H $HOSTADDRESS$ -u $USER11$ -p $USER12$ -l cpu -s usage -w $ARG1$ -c $ARG2$         }   # check memory usage define command{         command_name check_esx_mem         command_line $USER1$/check_esx -H $HOSTADDRESS$ -u $USER11$ -p $USER12$ -l mem -s usage -w $ARG1$ -c $ARG2$         }   # check net usage define command{         command_name check_esx_net         command_line $USER1$/check_esx -H $HOSTADDRESS$ -u $USER11$ -p $USER12$ -l net -s usage -w $ARG1$ -c $ARG2$         }   # check runtime status define command{         command_name check_esx_runtime         command_line $USER1$/check_esx -H $HOSTADDRESS$ -u $USER11$ -p $USER12$ -l runtime -s status         }   # check io read define command{         command_name check_esx_ioread         command_line $USER1$/check_esx -H $HOSTADDRESS$ -u $USER11$ -p $USER12$ -l io -s read -w $ARG1$ -c $ARG2$         }   # check io write define command{         command_name check_esx_iowrite         command_line $USER1$/check_esx -H $HOSTADDRESS$ -u $USER11$ -p $USER12$ -l io -s write -w $ARG1$ -c $ARG2$         }

And an example of the configuration for a Nagios host called esxi01 in /etc/nagios/hosts/esxi01.cfg :

# Host esx01 define host{         use                     linux-server         host_name               esxi01         alias                   VMWare ESXi 01         address                 192.168.1.100         }   # Define a service to "ping" the local machine define service{         use                             generic-service         host_name                       esxi01         service_description             PING         check_command                   check_ping!100.0,20%!500.0,60%         }   # VMWare # check cpu define service{         use                             generic-service         host_name                       esxi01         service_description             ESXi CPU Load         check_command                   check_esx_cpu!80!90         }   # check memory usage define service{         use                             generic-service         host_name                       esxi01         service_description             ESXi Memory usage         check_command                   check_esx_mem!80!90         }   # check net define service{         use                             generic-service         host_name                       esxi01         service_description             ESXi Network usage         check_command                   check_esx_net!102400!204800         }   # check runtime status define service{         use                             generic-service         host_name                       esxi01         service_description             ESXi Runtime status         check_command                   check_esx_runtime         }   # check io read define service{         use                             generic-service         host_name                       esxi01         service_description             ESXi IO read         check_command                   check_esx_ioread!40!90         }   # check io write define service{         use                             generic-service         host_name                       esxi01         service_description             ESXi IO write         check_command                   check_esx_iowrite!40!90         }
It’s done. Restart Nagios and wait a while (or re-schedule) for the new resources to be monitored.

Tuesday, October 18, 2011

NuSOAP WSDL Complex Data

Source: http://www.koopman.me/2008/01/nusoap-wsdl-service-return-array-of-complex-data/

I’m just getting the code in the post for now… hopefully I’ll have time to come back and document it for you. I am excluding my config.php on purpose, you’ll need to define you’re own constants in there.

require_once(“config.php”);
require_once(“nusoap.php”);
/*
* This function takes an $email address and returns an array of email addresses
* that are the given email address’s recent contacts from their address book.
*
* @param string $email (the email address of the user)
* @param string $token (a password that is used for authentication for use of this function, it is NOT the email users password.)
*/
function getAutocompleteContacts( $email, $num, $token ) {
if ($token != SOAP_TOKEN)
return new soap_fault(‘Server’, ”, “Supplied token does not match our records”,”);
$email = strtolower(trim($email));
if ( ! ereg( ‘^’.email_reg.’$', $email) )
return new soap_fault(‘Client’, ”, “Must supply a valid email address: $email not valid”,”);
$aDB = DB::connect( ADDR_DSN );
if ( DB::isError($aDB) ) {
return new soap_fault(‘Server’, ”, ‘Service temporarily unavailable: could not connect to ADDR_DSN DB’,”);
}
$aDB->setFetchMode( DB_FETCHMODE_ASSOC );
// REALLY YOU SHOULD DO YOU’RE OWN QUERY, BUT FOR THE SAKE
// OF THIS EXAMPLE, I’M JUST GOING TO INJECT A COUPLE RESULTS:
$result = array();
$result[] = array( ‘contact’ => ‘Chaos Captain’, ‘email’ => ‘choas@sdfusidfousdf.com’);
$result[] = array( ‘contact’ => ‘Joe Joe’, ‘email’ => ‘choas@sdf768sdf798s7df987.com’);
return $result;
#return new soap_fault(‘Server’, ”, ‘Fallthrough error, should have faulted on invalid type above’,”);
}
$HTTP_RAW_POST_DATA = isset($HTTP_RAW_POST_DATA) ? $HTTP_RAW_POST_DATA : ”;
$server = new soap_server;
$server->configureWSDL(‘addressbook1′, ‘urn:’.$_SERVER['SCRIPT_URI']);
$server->wsdl->addComplexType(
‘Contact’,
‘complexType’,
‘struct’,
‘all’,
”,
array(
‘contact’ => array(‘name’ => ‘contact’, ‘type’ => ‘xsd:string’),
‘email’ => array(‘name’ => ‘email’, ‘type’ => ‘xsd:string’),
)
);
$server->wsdl->addComplexType(
‘ContactArray’,
‘complexType’,
array’,
”,
‘SOAP-ENC:Array’,
array(),
array(
array(‘ref’=>’SOAP-ENC:arrayType’,'wsdl:arrayType’=>’tns:Contact[]‘)
),
‘tns:Contact’
);
$server->register(‘getAutocompleteContacts’,
array(‘email’ => ‘xsd:string’, ‘num’ => ‘xsd:int’, ‘token’ => ‘xsd:string’), // input parameters
array(‘return’ => ‘tns:ContactArray’),
‘urn:’.$_SERVER['SCRIPT_URI'], // namespace
‘urn:’.$_SERVER['SCRIPT_URI'].”#getAutocompleteContacts”, // soapaction
‘rpc’, // style
‘encoded’, // use
‘Fetch array of address book contacts for use in autocomplete’); // documentation
#$server->wsdl->schemaTargetNamespace = $_SERVER['SCRIPT_URI'];
$server->service($HTTP_RAW_POST_DATA);
exit();
?>


Client Code:

require_once(‘config.php’);
require_once(‘nusoap.php’);
$client = new soapclient(SOAP_SERVER_WSDL_URL, true);
// Check for an error
$err = $client->getError();
if ($err) {
// Display the error
echo

Constructor error

’ . $err . ‘
’;
// At this point, you know the call that follows will fail
}
$result = $client->call(‘getAutocompleteContacts’, array(‘email’=>’something@asdf97s9d8f7sdf.com’, ‘num’=>5, ‘token’=>SOAP_TOKEN) );
print_r($result);
?>


Result is as follows:

Array
(
[0] => Array
(
[contact] => Chaos Captain
[email] => choas@sdfusidfousdf.com
)
[1] => Array
(
[contact] => Joe Joe
[email] => choas@sdf768sdf798s7df987.com
)
)


Tuesday, September 6, 2011

Setup SNMP on an Asterisk server

Posted on March 1, 2011 by Zeeshan A Zakaria


Today I happened to troubleshoot a server in a remote part of the world, which I had shipped with SNMP and MRTG installed. It was a very wise decision to do this at that time, as it made an important troubleshooting very simple, which otherwise could have wasted days and still yielded no results.

While going over my notes in this regard, I thought to put this information here as well if it could help someone else who is looking to do a similar setup. But you should know that setting up SNMP can be tricky and the following might not work for you.

SNMP – Simple Network Monitoring Protocol, as it name suggests, is a protocol used to monitor various properties of network equipment. These properties are identified by something called OIDs (Object Identifiers) which are long numeric strings and identify various things, like hardware version, time on the device, data flow rate, and tons of other stuff. OIDs and grouped in what is called a MIB (Management Information Base). Asterisk has a list of its own OIDs which provide various pieces of information over the SNMP when requested by a software or hardware which is interested in that information. More on SNMP you can read on the Internet.

The following assumes that you do have SNMP module installed with your asterisk installation. You select this when running the #make menuselect just before the installation of asterisk. See my blog on asterisk installation for more details.
SNMP Setup

Following are the steps which I used on this CentOS 5.3/Asterisk 1.4 machine to setup SNMP:
1 yum install net-snmp-util

Created /etc/snmp/snmpd.conf with the following:

; Add this at the bottom
master agentx
agentXSocket /var/agentx/master
agentXPerms 0660 0550 nobody asterisk

; main configuration of the snmp
com2sec local localhost public
com2sec mynetwork 192.168.1.0/24 public

group notConfigGroup v1 notConfigUser
group notConfigGroup v2c notConfigUser
group MyROGroup v1 local
group MyROGroup v2c local
group MyROGroup v1 mynetwork
group MyROGroup v2c mynetwork

view all included .1 80

access MyROGroup "" any noauth exact all all none

Going into the detail of what all these lines mean is not the goal here, but the words in bold red are the ones which will be specific to your own network. The word ‘public’ is like a password, and almost all SNMP devices come preconfigured with this password. The user is ‘asterisk’, in your setup it might be ‘root’ or some other user.
Asterisk SNMP Config

Now add the following in /etc/asterisk/res_snmp.conf. If the file doesn’t exist, create one:
1 [general]
2 subagent = yes
3 enabled = yes

And now copy the Asterisk related MIB to the SNMP folder:
1 cp /usr/share/doc/asterisk-1.4.22/digium-mib.txt /usr/share/snmp/mibs
2 cp /usr/share/doc/asterisk-1.4.22/asterisk-mib.txt /usr/share/snmp/mibs

It is possible that these files don’t exist on your system, in which case you’ll have to get them from the Internet.

Next do the following:
1 export MIBS=+ASTERISK-MIB
2 chmod 755 /var/agentx
Done

This is all. Now restart asterisk, add snmpd and snmptrapd in chkconfig and start these services as daemons:
1 chkconfig snmpd on
2 chkconfig snmptrapd on
3 service snmpd start
4 service snmptrapd start
Testing it

In order to test if SNMP is properly installed, do the following:
1 snmpwalk -On -c public -v 2c localhost asterisk

and it’ll give you a list of the asterisk related MIBS

Friday, August 12, 2011

Using Wireshark to Troubleshoot VoIP

Wireshark (also known as ethereal or tethereal) is a "network protocol analyzer. It lets you capture and interactively browse the traffic running on a computer network." If you want a definitive answer on what is causing problems with your VoIP calls, you need to learn how to use Wireshark. Thankfully, it's pretty simple for anyone with basic knowledge of networking.

There are two parts to successfully running a network trace: capturing the traffic and viewing the results. I'll go through capturing traffic on different operating systems and viewing the results on a Windows PC using Wireshark.

Capturing traffic on a Linux-based PBX

Assuming you're running CentOS, which is the default for most PBX software, you will need to install the Wireshark package:

yum install wireshark

Run the following commands on your PBX to capture all traffic, SIP (signalling) and RTP (audio), between the PBX and your provider's server into file /root/my.cap. While the capture is running, restart your PBX software so it will attempt to register with your provider. Then attempt an outgoing phone call and an incoming phone call. Type Ctrl-C after running these tests to stop the capture.

tshark host server.provider.com -w /root/my.cap

Run the following command to compress your capture file my.cap into a compressed my.cap.gz file:

gzip /root/my.cap

From your Windows PC, run the following to copy that capture file from your PBX's IP address to your Windows PC's C: drive. You will need to have the pscp program, which is available here. You will be prompted for the PBX's root password. Replace 192.168.1.2 with the IP address of your PBX.

pscp root@192.168.1.2:/root/my.cap.gz c:\

Your capture file is now ready to be viewed from your Windows PC. Run Wireshark and open the capture file (Wireshark can open compressed capture files).

Capturing traffic on a Windows PC

If you have a Windows PC on the same network as your Voice over IP adapter, you can use it to capture all traffic between the adapter and your provider. Since a network switch isolates the traffic on each port, you cannot capture another device's traffic from your PC (unless it supports port mirroring). Instead, the easiest way to capture traffic on your local network is to get a cheap Ethernet hub. A hub repeats all traffic coming into one port on all of the other ports. This means any device plugged into a hub can view the traffic to/from every other device on the same hub.

Plug your hub into your router and your PC and adapter into the hub, as shown below:

Tip: A more advanced method of capturing traffic would be to plug the hub into your cable modem, so it sits between the modem and router. However, anything else you plug into the hub (like the PC running Wireshark) will be unprotected and open to all traffic from the Internet.

Start Wireshark and click on Capture Options. This will allow us to select the proper network card to capture traffic from, the host to capture traffic to/from (your provider's gateway) and a place to save the capture file. You must run the capture in "promiscuous" mode, which means Wireshark will "look" at any traffic it sees on the network, not just traffic to/from the host it's running on (your PC).

Screenshot_1
While the capture is running, reboot your adapter so it will attempt to re-register with your provider. Also, attempt to make an incoming and outgoing call. After running these tests, click on the "Stop the running live capture" button. Your capture will already be loaded for viewing.

Reading and understanding a VoIP traffic capture

Now that you've captured the traffic between your adapter (or PBX) and your provider, you have enough information to figure out what is causing the problem. For most people, the easiest thing to do now is send the capture file to your provider. An experienced technical support agent will be able to immediately tell you what is causing the problem and how to fix it. As I mentioned in an earlier post (VoIP, no dial tone, missed calls & port forwarding), there's a 90% chance the the problem is your router.

Tip: Unless you have significant networking experience, stop here and send the capture to your provider. It will be tremendously helpful to them in troubleshooting the issue and you can save yourself a few hours of time.

If you'd like to dig into the details and see exactly what is happening, then continue on. When you open your capture file, you'll see a split-screen with each Ethernet frame on the top, a user-readable drill-down of the contents in the middle, and the raw data on the bottom.

A working SIP REGISTER conversation

Scroll down in the top panel until you reach the first REGISTER attempt from your adapter to your provider. Select the frame and in the middle window, right-click on the "Session Initiation Protocol" section and pick "Expand Subtrees". Find the line starting with "Call-ID", right-click on it, and select Apply as Filter > Selected. The Filter box above the first window pane will now show something like "sip.Call-ID == "5b1c8fdb-90c4f2a3@10.0.1.100"", meaning only the frames that contain that Call-ID will be shown. This makes it easier to filter out one conversation from a capture that may have a lot of extraneous traffic. This is what a proper REGISTER attempt will look like:

2

Your adapter will attempt to register, but it doesn't send any Authorization data the first time. The provider will respond with a 401 Unauthorized and will include information such as the realm, nonce, algorithm values. The adapter will then use these values, along with your SIP password, to form an encrypted response which it will use when re-registering. Since the second register attempt has Authorization information, the provider will accept the registration and reply with a 200 OK.

You can dig into each SIP request and response to see exactly how the registration conversation works. Now that you know what a working REGISTER conversation looks like, you can identify one that is broken.

A broken SIP REGISTER conversation

The most common cause for an adapter not being able to register is a router/firewall issue. If the provider's 401 Unauthorized response never makes it back to the device, it can never properly send Authorization information to the provider to register.

3

As you can see in the registration attempt above, the adapter never gets the 401 Unauthorized, so it keeps trying without success. If your provider was looking at a trace from their end, they'd see a 401 Unauthorized being sent, but no REGISTER with an Authorization section, clearly indicating that the 401 was being blocked by the router/firewall.

Working outgoing & incoming calls

Most providers configure their adapters to only play a dial-tone when the adapter is registered. Therefore, once you are registered, you will probably have no issues making an outbound call. Also, an outbound call is a connection initiated from your private network, so there are usually no firewall issues to deal with either.

An outgoing call is initiated by an INVITE request sent from your adapter to the provider. The provider will reply with a 100 Trying and a 407 Proxy Authentication Required. This is similar to the 401 Unauthorized during the REGISTER request, except with a Proxy-Authenticate section instead of an Authorization section. Your adapter will re-send the INVITE with your credentials. The provider will respond with a 100 Trying, 183 Session Progress (which will cause you to hear ringing), and 200 OK when the call is answered. Your adapter will acknowledge the answer and the conversation will begin. Finally, when one of the sides hangs up, a few BYE messages will be sent and acknowledged.

Tip: You may not see this exact sequence of events, depending on how your provider handles the call and if it's actually answered. A busy or invalid number will return different responses than a 200 OK.

A working incoming call looks similar to an outgoing call, with INVITEs, 100s, 183s, and 200s.

Broken outgoing & incoming call

It's rare to have an issue with outgoing calls if you have a dial-tone. However, a capture will show what's going wrong if you do.

If you're not receiving incoming calls at all, chances are you won't see anything on the capture, which is still useful information! A common problem is that you will receive incoming calls for a few minutes after your adapter is first powered on (and registers), but then your incoming calls don't ring through. If you can capture this entire sequence (register, successful incoming call, failed incoming call 15 minutes later), it will clearly demonstrate that there is a router/firewall issue.

Analyzing calls with Wireshark

Wireshark has telephony-specific features that may come in handy for troubleshooting VoIP calls. After opening your capture file, go to Telephony > VoIP Calls. Wireshark will automatically detect all of the calls in your capture.

You can then click "Prepare Filter" to easily view just the frames associated with a particular call, click "Flow" to see the conversation between your adapter and provider, or click "Player" to listen to one or both sides of the conversation in the capture.

Also, you can go to Telephony > RTP > Show All Streams. Select an RTP stream (the audio from one side of a phone call), click Find Reverse, click on Analyze. You can now see and graph statistics like how many RTP packets were lost or the max and mean jitter.

Finally, if you want to cut/paste a UDP SIP conversation into a support ticket with your provider, you can right-click on one of the frames in the conversation and select Follow UDP Stream. Change the radio button to ASCII before copying or saving the data.

Conclusion

Now that you know how to capture and analyze your Voice over IP traffic using Wireshark, you have the ability to do some troubleshooting on your own that goes beyond rebooting the adapter. However, as I've mentioned before, sometimes it's far easier and faster to let your provider take over after you've provided them with a helpful capture showing the problem.

Please don't take this as an opportunity to hassle your provider over issues that don't cause any noticeable problems in your phone calls. There will be jitter; there will be dropped RTP packets. That is the nature of voice over IP and there are numerous methods of ensuring voice quality despite these network-level problems. Your provider is focused on providing you with clear phone calls, not on optimizing the statistics generated by Wireshark.

If you're interested in the inner-workings of voice over IP and SIP, there's no better way to dig in than running a Wireshark capture and figuring out exactly what is going on.

Wednesday, July 20, 2011

Install 64 Bit Package via Yum

I had a task to install Ralus (Backup_exec agent) on linux. I had an issue where there is incompatibility issue on the c compiler. I had to install a compiler that works with the Ralus. I did yum install on a certain package as follows:

yum install libstdc++.so.5

The above command did install the compat-libstdc++-33-3.2.3-61 but still the Ralus didn't work. Until I realized that I am running 64 bit machine so I had to do force install of 64 bit of the compat-libstdc package.

yum install compat-libstdc++-33-3.2.3-61.x86_64

So the lesson here if ever there is a necessity or something happened where it didn't download and install the package in 64 bit, do as follows:

yum install packagename.x86_64

And you should be good.

Hope you find this helpful. Thanks, Cheers and God Bless!!!

Monday, June 6, 2011

Someday

I don’t know how much longer
That I have to put up with everything
I’ve been hiding all the truth inside my heart
Everytime we meet
Everytime you turn face to me
Though I look indifferent
Do you know how much i have to force myself?
Can you hear my heart calling for you, loving you?
But I can’t open my heart for anyone to know
Can you hear it?
My heart keeps waiting there for you
Waiting for you open it
and hope you will realize..
Someday

Though I love you
Though I feel
but deep down inside, I don’t dare to tell you
Everytime we meet
Everytime you turn face to me
Though I look indifferent
Do you know how much i have to force myself?

Can you hear my heart calling for you, loving you?
But I can’t open my heart for anyone to know
Can you hear it?
My heart keeps waiting there for you
Waiting for you open it
and hope you will realize..
Someday

Can you hear my heart calling for you, loving you?
But I can’t open my heart for anyone to know
Can you hear it?
My heart keeps waiting there for you
Waiting for you open it
And I hope you will realize
That this person loves you
Please I hope you will know
Someday

Friday, May 6, 2011

Compiling the Linux Kernel – CentOS

Steps:

#yum -y install kernel-devel kernel-headers

#yum -y groupinstall “Development-Tools”

#cd /usr/src

#wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.30.2.tar.bz2

#tar -jxvf linux-2.6.30.2.tar.bz2

#cd /usr/src/linux-2.6.30.2/

#make menuconfig

#make clean

*#make dep (is no more necessary)

#make bzImage

#make modules

#make modules_install

#make install

*make install will copy the files: System.map, vmlinuz and the initial ram disk initrd-2.6.30.2.img to /boot. It will also add the lines you will need on grub, now just restart your computer and choose the new kernel on the grub screen.

Have Fun !!!

Friday, April 29, 2011

Setting up SendMail as SMTP Relay Server

Here are the steps to take in order to use Sendmail as SMTP relay server:

By default relay on sendmail is disabled.

So when you do relay on it by doing telnet

220 nagios.gowireless.net ESMTP Sendmail 8.13.8/8.13.8; Fri, 29 Apr 2011 11:59:2
3 -0700
HELO
501 5.0.0 HELO requires domain address
MAIL FROM:test@test.com
250 2.1.0 test@test.com... Sender ok
relaying denied

So to allow relay on SendMail do as follows:

1. First allow the sendmail to listen to all at port 25. (Allow incoming smtp connections)

- Edit /etc/mail/sendmail.mc

Change...

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl

To..

DAEMON_OPTIONS(`Port=smtp, Name=MTA')dnl

- save it

and then do

make -C /etc/mail

- restart sendmail service

service sendmail restart

2. Allow or open port 25 on the firewall.

3. Now we will include IP based authentication

- Edit /etc/mail/access and add the IP / IP block to allow relay from

Connect:localhost.localdomain RELAY
Connect:localhost RELAY
Connect:127.0.0.1 RELAY
Connect:10.1 RELAY
Connect:1.1 RELAY

- save it

- then recompile the access.db

makemap hash /etc/mail/access.db < /etc/mail/access

- then do restart sendmail

4. To test do as follow from a different pc


telnet 10.1.10.13 25
220 nagios.gowireless.net ESMTP Sendmail 8.13.8/8.13.8; Fri, 29 Apr 2011 11:59:2
3 -0700
HELO
501 5.0.0 HELO requires domain address
MAIL FROM:test@test.com
250 2.1.0 test@test.com... Sender ok
RCPT TO:rowell.rufino@gowireless.com
250 2.1.5 rowell.rufino@gowireless.com... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
testing
.
250 2.0.0 p3TIxNH1002876 Message accepted for delivery
quit
221 2.0.0 nagios.gowireless.net closing connection

Hopefully you find this helpful. Thanks. Take Care and God Bless!!!

Friday, April 8, 2011

Setting Up Mail Server on Linux

There came a time where I was asked by my boss to setup a Mail server for a certain client. At first I was astounded because I am not sure how to do it. But thankfully I found some helpful sites that guided me to establish a complete Email Server. In this blog I wanted to share them with you to help those who would like to setup their own Mail server.

The mail server that I have setup is composed of the following components:

- a Linux OS (preferably Red Hat Enterprise or CentOS)
- Dovecot for IMAP / POP3
- Postfix for SMTP
- ClamAV for antivirus
- SpamAssassin
- MailScanner

How to install SMTP, POP3, IMAP and Webmail service

Postfix will be providing the SMTP service, Dovecot will provide the POP3 and IMAP service, while Apache and SquirrelMail will provide the Webmail service.

The following are the steps we will take on setting up the mail server:

1. How to install and setup Postfix SMTP server.
2. How to install and setup Dovecot POP3 and IMAP server.
3. Configure Postfix SMTP Authentication using Dovecot SASL.
4. How to install and setup SquirrelMail WebMail.
5. How to configure the firewall.


How to install and setup Postfix SMTP server

We need to install 2 packages here namely:

- postfix and system-switch-mail

To do so, issue the following command:

# yum install postfix system-switch-mail

By default the MTA on Linux is sendmail so we need to switch from sendmail to postfix. Use the system-switch-mail tool to switch to postfix.

Configure Postfix

1. Edit the postfix configuration file.

# vi /etc/postfix/main.cf

Find the following keys and change its values as follows
inet_interfaces = all mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain home_mailbox = Maildir/ 

Make sure that all mail_spool_directory lines are commented out. Otherwise, it will override the setting in the home_mailbox line above.

2. Restart postfix service

# service postfix restart

Test Postfix

1. For testing purposes, create a linux user.

# useradd johndoe

Sample postfix session. Replace johndoe with any valid user account. The dot after the line test is a command that should be typed in.

2. Perform testing.

[root@mail ~]# telnet localhost smtp Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. 220 mail.acme.local ESMTP Postfix ehlo localhost 250-mail.acme.local 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN mail from: 250 2.1.0 Ok rcpt to: 250 2.1.5 Ok data 354 End data with . test . 250 2.0.0 Ok: queued as 9729067C17 quit 221 2.0.0 Bye Connection closed by foreign host. [root@mail ~]#
To check if the mail indeed exists

[root@mail ~]# cd /home/johndoe/Maildir/new [root@mail new]# ls 1185669817.Vfd00I18012M795756.mail.acme.local [root@mail new]# cat 1185669817.Vfd00I18012M795756.mail.acme.local
Don’t worry, you don’t have to type in the whole filename above. Just type in the first few characters say 118 then press Tab to activate automatic completion.

From johndoe@mail.acme.local  Thu Feb 22 21:48:28 2007 Return-Path:  X-Original-To: johndoe Delivered-To: johndoe@mail.acme.local Received: from localhost.localdomain (localhost.localdomain [127.0.0.1])         by mail.acme.local (Postfix) with SMTP id 9729067C17         for ; Thu, 22 Feb 2007 21:48:26 -0500 (EST) Message-Id: <20070222134827.9729067C17@mail.acme.local> Date: Thu, 22 Feb 2007 21:48:26 -0500 (EST) From: johndoe@mail.acme.local To: undisclosed-recipients:;  test  [root@mail mail]#
If you encounter any problems, check the log file at /var/log/maillog.

How to install and setup Dovecot POP3 and IMAP server

1. To install dovecot:

# yum install dovecot

Configure Dovecot

1. Edit the dovecot configuration file

# vi /etc/dovecot.conf

Find the following keys and change its values as follows

protocols = pop3 pop3s imap imaps mail_location = maildir:~/Maildir/ pop3_uidl_format = %08Xu%08Xv imap_client_workarounds = delay-newmail outlook-idle netscape-eoh pop3_client_workarounds = outlook-no-nuls oe-ns-eoh


2. Start Dovecot server

# service dovecot start

Test Dovecot

Sample dovecot session. Replace johndoe and password with any valid user name and password.

root@mail ~]# telnet localhost pop3 +OK dovecot ready. user johndoe +OK pass password +OK Logged in. list +OK 1 messages: 1 622 . retr 1 +OK 622 octets Return-Path:  X-Original-To: johndoe Delivered-To: johndoe@mail.acme.local Received: from localhost.localdomain (localhost.localdomain [127.0.0.1])         by mail.acme.local (Postfix) with SMTP id 9729067C17         for ; Thu, 22 Feb 2007 09:06:37 -0500 (EST) Message-Id: <20070222140640.9729067C17@mail.acme.local> Date: Thu, 22 Feb 2007 09:06:37 -0500 (EST) From: johndoe@mail.acme.local To: undisclosed-recipients:; X-IMAPbase: 1172153557 1 Status: O X-UID: 1 Content-Length: 5 X-Keywords:   test . quit +OK Logging out. Connection closed by foreign host. [root@mail ~]#

Configure Postfix SMTP Authentication using Dovecot SASL

SMTP Authentication (SMTP Auth) provides an access control mechanism that can be used to allow legitimate users to relay mail while denying relay service to unauthorized users, such as spammers.

Thanks to the new SASL support in Dovecot 1.0 and the new Dovecot SASL support in Postfix 2.3, setting up SMTP authentication is now easier. Instead of setting up two separate authentication for Postfix and Dovecot, we can now just setup the authentication in Dovecot and just let Postfix talk to Dovecot.

Configure Postfix and Dovecot

1. Edit the file /etc/dovecot.conf and make sure your auth default section has the lines below.

auth default {    socket listen {     client {    path = /var/spool/postfix/private/auth    mode = 0660    user = postfix    group = postfix     }   }    mechanisms = plain login }


2. Edit /etc/postfix/main.cf, find the keys below and change its values as follows or add it at the bottom of the file if the key (the word before the = sign) cannot be found.

mynetworks = 127.0.0.0/8 smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions =  permit_mynetworks,      permit_sasl_authenticated, reject_unauth_destination broken_sasl_auth_clients = yes
The first line says trust only localhost, meaning only localhost can send email outside the network (relay). The last line is there to support old clients like Microsoft Outlook Express 4.0 and Microsoft Exchange 5.0 just in case someone is still using it.

3. Restart the Dovecot and Postfix service. But if you installed MailScanner, restart MailScanner instead of Postfix.

Test Postfix

Sample postfix session

[root@mail ~]# telnet mail smtp
Replace mail with the name of your server. We should not use localhost since localhost is a trusted client ip address. And make sure the domain name you specified does not resolve to 127.0.0.1 which is the IP address of localhost.

Trying 192.168.0.1... Connected to mail.acme.local (192.168.0.1). Escape character is '^]'. 220 mail.acme.local ESMTP Postfix ehlo localhost 250-mail.acme.local 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-AUTH LOGIN PLAIN 250-AUTH=LOGIN PLAIN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN
Note the new 250-AUTH lines.

mail from: 250 2.1.0 Ok rcpt to: 554 5.7.1 : Relay access denied
It works, now to check if we can send it after authenticating.

auth plain AGpvaG5kb2UAcGFzc3dvcmQ= 235 2.0.0 Authentication successful rcpt to: 250 2.1.5 Ok quit 221 2.0.0 Bye Connection closed by foreign host. [root@mail ~]#
The gibberish text after AUTH PLAIN is the base64 encoded value of the user name johndoe and password password. You can generate your own base64 text using the form below.

http://www.linuxmail.info/postfix-smtp-auth-dovecot-sasl/

How to install and setup SquirrelMail WebMail.

SquirrelMail is an open source standards-based webmail package written in PHP. When installed, SquirrelMail is ready out of the box. All it needs is an installed web server like Apache, SMTP server like Postfix, and IMAP server like Dovecot.

1. Get the latest stable release of Squirrelmail

# cd /usr/src

# wget http://www.squirrelmail.org/countdl.php?fileurl=http%3A%2F%2Fprdownloads.sourceforge.net%2Fsquirrelmail%2Fsquirrelmail-1.4.21.tar.gz

# tar xvzf squirrelmail-1.4.21.tar.gz

# mv squirrelmail-1.4.21 /usr/local/squirrelmail

Prepare SquirrelMail directories

Make sure to change "nogroup" to whatever group Apache will be running as. You can check what that is by looking at the value of the "Group" setting in your Apache main configuration file (probably /etc/httpd/conf/httpd.conf).

# mkdir /usr/local/squirrelmail # cd /usr/local/squirrelmail # mkdir data temp # chgrp nogroup data temp # chmod 0730 data temp

Unpack SquirrelMail

# cd /usr/local/squirrelmail # tar --bzip2 -xvf /usr/local/src/downloads/squirrelmail-1.4.17.tar.bz2 # mv squirrelmail-1.4.17 www 

Configure SquirrelMail

Run the SquirrelMail configuration utility.

# cd /usr/local/squirrelmail # www/configure 

Select the "D" option and then configure SquirrelMail with the "uw" preset. Also make sure to set the data and attachment directory settings ("/usr/local/squirrelmail/data" and "/usr/local/squirrelmail/temp" respectively) under "4. General Options". Make any other changes as you see fit, select "S" to save and then "Q" to quit.

Configure access to SquirrelMail in Apache

Modify your main Apache configuration file (typically /etc/httpd/conf/httpd.conf) by adding the following:


"Alias /squirrelmail /usr/local/squirrelmail/www    Options None   AllowOverride None   DirectoryIndex index.php   Order Allow,Deny   Allow from all     Deny from all     Allow from all     Allow from all     Allow from all     Allow from all     Allow from all     Order Deny,Allow   Deny from All   Allow from 127   Allow from 10   Allow from 192     Order Deny,Allow   Deny from All   Allow from 127   Allow from 10   Allow from 192 "

Restart apache

Reference:
http://www.linuxmail.info/squirrelmail-webmail-setup-howto-in-centos-5/
http://squirrelmail.org/docs/admin/admin-3.html#ss3.2

Thursday, January 27, 2011

How to set 100% table height

One of the suggestions I have read across the web that really works for me on table layout, is setting the table height. This is useful most specially if you are dealing with nested tables where you want the table inside to maximize the height of the cell. In order to do this, you must make use of style property of table. Here is how:

table width="100%" border="1" style="height: 100%;"

Hopefully you find this tip useful. Thank you. Take care and God Bless!!!

Tuesday, January 18, 2011

Automated Backup Script Using tar and FTP


I am posting here a script I made to automate backup via tar ang FTP. I hope that you may find this helpful. In the next post, I will try to explain each entries on the script. Thanks, take care and God Bless!!!

#!/bin/bash
# A Simple (Poor Man) disk based backup script
# Will backup all user home directories in seperate archives
# as a single /home backup file can be quite large and unwieldly.
# Does not need to be modified if users are added/deleted

# Step - 1 Create Timestamp and set up variables and functions
# BUDTSTAMP = Backup Date/Time Stamp
BUTDSTAMP=$(date +%Y%m%d)

# variable holding directories containing files to backup eg: BACKUPTHESE="/home /root /etc"
BACKUPTHESE="/var/www/html/adldap"

# Backup base place the back up base

B_BASE="/trynga"

cd $B_BASE

mkdir $B_BASE/vol
mkdir $B_BASE/vol/backup
mkdir $B_BASE/vol/backup/daily
mkdir $B_BASE/vol/backup/yesterday
mkdir $B_BASE/vol/backup/database

#directory containing today's backup
BKUPDIR="$B_BASE/vol/backup/daily"
#directory containing yesterday's backup
YDBKUPDIR="$B_BASE/vol/backup/yesterday"
#directory containing the day before yesterday's backup
DBBKUPDIR="$B_BASE/vol/backup/daybefore"
# DATABASE BACKUP DIR
DATABASEBACKUP="$B_BASE/vol/backup/database"
NCFTP="/usr/local/bin"

# device to monitor upon backup
DEVICE="/"

# recipient to send the backup report
#EMAILADD="backupadmin"
EMAILADD="rowell@mindragon.com"

# FTP SetUP
FTPS="192.168.1.118"
FTPU="rowell"
FTPP="mdi2005!"
NOW=$(date +"%d-%m-%Y")
BACKUP="$B_BASE/vol/backup/daily"
FTPD="serbackupfiles"

### MySQL Setup ###
MUSER="root"
MPASS="mdi2009!"
MHOST="localhost"
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"
GZIP="$(which gzip)"

# log files
LOG="/var/log/mail.txt"
ERRORLOG="/var/log/ERROR.txt"

# tag for backup file name
TAG="OpenSERbackup"

# function that quits and logs on error
exiterror()
{
# use this function by supplying $LINENO as first arg
echo "Fatal error caused by line ${1} of ${0}" >> $ERRORLOG
mail $EMAILADD -s "Backup Job ERROR" -v < $ERRORLOG
mail $EMAILADD -s "Backup Job Report" -v < $LOG
exit 1
}

# Step - 2 Start Email Message To Be Sent

# Remove mail message from previous backup
# I do this at the beginning of the script instead of the end
# in case the mail does not send for whatever reason or
# I need to debug it

rm $LOG > /dev/null 2>&1
echo "System Backup $BUTDSTAMP" >> $LOG

# The email sends a user friendly note showing the start and
# end time/dates. This is important so you can compare logs
# and see if a backup ran ok.

echo "Backup Began $(date)" >> $LOG

# Step - 3 Rotate Backups
# Simple three backupset rotation, keeps only last three
# Use directory /vol/backup as an example, make sure you change this path
# to fit your local settings. Have three directories in /vol/backup named daily,
# yesterday and daybefore.
# Make sure /vol/backup is on a different disk (preferably a different machine)
# than the files you are backing up

#looks specifically for backup files in case other files are kept in these directories
#for bkfile in $DBBKUPDIR/*; do
#echo $bkfile | grep $TAG >/dev/null &&
#(rm $bkfile || exiterror $LINENO )
#done
for bkfile in $YDBKUPDIR/*; do
echo $bkfile | grep $TAG >/dev/null &&
#(mv $bkfile $DBBKUPDIR/ || exiterror $LINENO )
(rm $bkfile || exiterror $LINENO )
done
for bkfile in $BKUPDIR/*; do
echo $bkfile | grep $TAG >/dev/null &&
(mv $bkfile $YDBKUPDIR/ || exiterror $LINENO )
done

# Step - 4 Archive Home Directories
# Creates a seperate tar file for each directory in the directories in $BACKUPTHESE


# change to dir to backup. Check in case cd failed for some reason.
cd $BACKUPTHIS || exiterror $LINENO
#backup all files listed in BACKUPTHESE
#for DIRTOBACKUP in $BACKUPTHESE; do
#for FOLDERNAME in $DIRTOBACKUP/*
#do
# Archives are created in the format someuser-$TAG-datetime.tar.gz
# basename is used here so as not to include absolute paths
# -p preserves permissions
#echo -e "-------------------\n>>>taring ${FOLDERNAME}\n\n" >> $LOG
#tar -czvpf ${BKUPDIR}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} >> $LOG || exiterror $LINENO
#done
#done

for FOLDERNAME in $BACKUPTHESE; do
echo -e "-------------------\n>>>taring ${FOLDERNAME}\n\n" >> $LOG
#tar -czvpf ${BKUPDIR}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} >> $LOG || exiterror $LINENO
tar -czvpf ${BKUPDIR}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} > /dev/null 2>&1 || exiterror $LINENO
done

# For MySQL backup database
echo -e "------------------\n>>>>back up the database\n\n" >> $LOG
### Start MySQL Backup ###
# Get all databases name

#mkdir $BACKUP/$NOW
mkdir $DATABASEBACKUP/$NOW

DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'show databases')"
echo "$DBS \n\n " >> $LOG
for db in $DBS
do
#FILE=$BACKUP/mysql-$db.$NOW-$(date +"%T").gz
FILE=$DATABASEBACKUP/$NOW/mysql-$db.${BUTDSTAMP}.gz
$MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE
done

# Step - 5 FTP backup start

echo -e "------------------\n>>>>Transfer backups to FTP server\n\n" >> $LOG

# not yet done Do an an FTP back up
# set the settings for FTP

### Dump backup using FTP ###
#Start FTP backup using ncftp
RETVAL=`$NCFTP/ncftp -u"$FTPU" -p"$FTPP" $FTPS <<EOF
mkdir $FTPD
mkdir $FTPD/$NOW
cd $FTPD/$NOW
lcd $BACKUP
mput *
lcd $DATABASEBACKUP/$NOW
mput *
quit
EOF`

echo -e "$RETVAL \n\n" >> $LOG

##### Let us use a a different version
#mkdir $FTPD
#mkdir $FTPD/$NOW
#ncftpput -m -z -u "$FTPU" -p "$FTPP" $FTPS


# Step - 6 Finish Email Report and Send

echo "Backup ended $(date)" >> $LOG

# df -h includes a human readable disk usage report of the media that /vol/backup
# is mounted on. Good to now if your backup disk is running out of space.
# Of course /dev/hdb1 is the device I use, modify it for your local settings

df -h $DEVICE >> $LOG
mail $EMAILADD -s "Backup Job Report" -v < $LOG

# Step - 7 All Done