Having been an avid perl user for quite some time now, I understand a lot more about why perl people hate RHEL, RedHat Enterprise Linux (as well as CentOS), and why some people hate perl.  RedHat has a bad, but well earned reputation for breaking perl.  They like to upgrade perl versions, and include some older perl modules, and other things that just break perl badly.  So what does a perl loving perlmonk do?

Well, you could whine about RHEL in #perl on the freenode servers.  But whining isn't productive or helpful, it just annoys people that need to get things done, and doesn't help anyone get the most out of their perl experience.

So what do you do?  Do what most power perl programmers do — Build your own perl.  That is probably the best kept secret in the perl developer community, unfortunately!

When you do that, you can safely ignore the perl your Linux distribution includes.  You can keep your Linux distribution updated, and you won't have to worry about breaking your production perl programs.  And, when there is a new version of perl, you can build it, and safely test your code with the new version!

Building perl

First, download the perl source.  You will end up with the file latest.tar.bz2. 

Extract with tar -xjf latest.tar.bz2 then change into the source directory.  Today, that is perl-5.10.1.

Now comes the questions:  Do you want 64 bit integer support?  Do you want threads, or do you want your perl to run faster?  There is some overhead in making variables thread-safe.  Some of these options are discussed at "When perl is not fast enough".

Here are the defaults that were recommended to me by Khisanth in #perl:

sh Configure -Dprefix=$HOMEperl5100 -Duseshrplib -Dusemultiplicity -Duse64bitint -Duseperlio -Duselargefiles -des

Ok, that's not what I use.  I don't need a shared perl library file (which run slower), because I won't be embedding perl anywhere.  I haven't found anything on usemultiplicity, so I don't use that either.  But, everything else is good.  This is what I have been using:

sh Configure -Dprefix=/opt/perl510 -Duse64bitint -Duseperlio -Duselargefiles -des

Note:  Using $HOME doesn't actually work, I think he meant not for the shell to replace $HOME with your home directory, but rather it to be replaced by the person typing.  Also, I am using the /opt directory, because I want my built perl to be used system-wide.  For local use, or testing, -Dprefix=/home/stevet/perl510 is what I use.

Next, comes the build process: make  Then, comes the test process: make test  If everything passes: make install

Congratulations, in your prefix directory, you now have a bin directory that contains: perl, perldoc, and cpan.  Try it out, /opt/perl510/bin/perl -v should display the version information:

This is perl, v5.10.1 (*) built for i686-linux-64int

Testing new versions of perl

To test new versions of perl, use the above directions, or run perl -V on your current production perl.  (This will show what build options you used.)  Grab the source to the new version of perl, and do everything the same except the -Dprefix.  Install the new perl into another directory.  Then, use the new perl with your code.  If it works, wonderful!  If not, continue to use the original perl binary until you get the new version worked out.

Most of the time, it is simply a matter of installing needed modules into the new version.  Make sure you run the new perl's cpan!

When cpan goes bad

This is why I am writing this.  Things went horribly wrong when I tried this.  Strange failures installing XML::Parser, and then again installing MIME::Entities.  Consult the guide and don't panic.

First, make sure that you have read the "Life with CPAN" guide.  Some of the things it talks about applies to having your own perl build.  But that only gets you so far.  Debugging cpan install failures is quite a challenge sometimes.  But, it does get easier with time and practice.  So let's get started with some CPAN debug tips, and then I'll finish up with my personal experiences with XML::Parser and MIME::Entities.

Debugging CPAN installs

Ok, you are at the cpan prompt, you type in: install DBD::mysql and suddenly pages of errors go flying by.  Now what?  Start with the very first line that gives an error.  But that was many screens ago, how do you find that?  Use the *nix tools available for just such things:  screen or script.  In screen, type ctrl-a and H.  That turns on screen logging, and you'll see it starts logging everything to a file called screenlog.0.  With script, you would type something like: script -c cpan and the output will be in a file called typescript.

Ok, so the first line that gives an error.  What type of error is it?  Does it look something like this:

In file included from dbdimp.c:20:
dbdimp.h:22:49: error: mysql.h: No such file or directory
dbdimp.h:23:45: error: mysqld_error.h: No such file or directory
dbdimp.h:24:49: error: errmsg.h: No such file or directory

This is from the c compiler (in this case gcc).  It is saying that it needs some header files.

Missing header files

This means that you don't have the development version of the program or library installed on your system.  Using your Linux distribution tools, install the development version. 

In the above case of mysql, it would be something like this for RHEL/CentOS: yum install mysql-devel.

Missing modules

When cpan goes to make test, does it error out with: Can't locate Test/Pod.pm in @INC (@INC contains: ...).  Ok, this is probably a minor bug in the module.  The module is supposed to list all of the modules that it has dependencies on, but maybe the author forgot one.  At the cpan prompt try: install Test::Pod, or whatever the module name was that it couldn't find.  If the missing module installs, retry installing the module that was giving you problems.  Odds are good that it will work fine now.

Failing tests, or what to try next

Probably the best part of perl, and of cpan, is the testing modules.  These allow the developer to design tests to exercise their code, and the libraries they depend upon to make sure that they function correctly.  But, what when those tests fail?

First stop is search.cpan.org.  Search for the module that you are having problems with.  Select the link to the module, and then on the right hand side click, View bug reports.  Look through the reports.  Are other people having the same error that you are?  Or, even better, are there patches available that fix the problem?

If you don't see your particular error listed, you might want to report it to the developer.  Or, even better, figure out what caused the error and report that!  Writing code that works well on many platforms is hard, and the developer might not have access to a machine like what you are using.

As a last resort you can also go to http://www.cpantesters.org and search for the troublesome module there.  While you won't find any answers there, you will see if anyone else is having problems installing the module, and what error messages they are getting.  Here are cpan testers' results of installing XML::Parser for example.

Google the error message

Just use google on error messages.  Don't use google when working with normal perl, because there is a lot of old perl code out there on the Internet.  There are still tutorials, which, when they were written 10 years ago were cutting edge.  But now, they show you the wrong way to write modern perl code.  If you want modern perl help use perlmonks.

Ok, so maybe you have found something, like what I found on my failed test installing XML::Parser.

XML::Parser

First, I got:

Expat.xs:12:19: error: expat.h: No such file or directory

Ok, so install the expat-devel, using yum and you're set, right?  No, this one gets much worse.  The module builds, but now it fails tests.  Searching around, I find out that debian is aware of this bug, thanks to failing perl tests.  What happened was a bug was found in the expat library that caused a security issue.  The library was quickly patched, however, the patch actually broke the expat library.  It wasn't detected by the expat developers, but it was detected by XML::Parser. 

Remember those module tests that are so great?  Yes, they are great at finding problems, but sometimes it is slow getting the fixes.  And RHEL, well, they have the broken security patch for expat.  They haven't gotten the latest correct patch into their system.  So, just like building your own perl, now you build your own expat.

Downloaded the expat 2.0.1 library from http://expat.sourceforge.net/ and installed it into /opt/expat.  Finally, a working, bug free, xml parsing library is installed.

Configuring XML::Parser to use the working expat library was a challenge, but by doing look XML::Parser in cpan, I was able to build using:

perl Makefile.PL EXPATLIBPATH=/opt/expat/lib EXPATINCPATH=/opt/expat/include

Then make (which compiles the module), make test (which now passed for me), and finally make install.

MIME::Entity

This on fails like this:

t/Smtpsend.t ......... accept failed: Connection timed out at t/Smtpsend.t line 46.
# Looks like your test exited with 110 before it could output anything.

If you are seeing this error, I would almost bet that you are using a dual-core or have multiple processors!  The bug is in the test script itself.  Use look MIME::Entity at the cpan prompt to get into the build directory.  Edit the t/Smtpsend.t with your favorite editor, and jump to line 41.  You should be right before the following lines:

# In the parent
my $s = $sock->accept();

Above these lines add: sleep 1

Ok, type: make test and it should pass just fine.  What I think happens is when the test script forks on a dual core system, the connection is accepted before the connection happens, so it fails.  But, with a 1 second sleep, the accept works correctly, and the test now passes.

Crypt::SSLeay

This fails on:

t/01-connect.t .. 1/8
#   Failed test 'Net::SSL->new'
#   at t/01-connect.t line 25.
# SSL negotiation failed:  at t/01-connect.t line 11
#  at t/01-connect.t line 11
# ; Interrupted system call at t/01-connect.t line 11
# ; Interrupted system call at t/01-connect.t line 11
# ; Interrupted system call at t/01-connect.t line 11
# Looks like you failed 1 test of 8.
t/01-connect.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/8 subtests
        (less 7 skipped subtests: 0 okay)

Again, the problem is with the test.  If you failed this test, it means you are installing this on a server that has https running on port 443.  Edit the t/01-connect.t script, and search for the two occurrences of 443 that look like:

        PeerPort => 443,

and

skip( "nothing listening on localhost:443", 7 )

Change them to some other port that isn't in use, like 4443.  Re-run make test.  If passing, now make install.

Wouldn't it be easier to just force the install?

No.  Forcing the install of perl modules that fail their tests is just asking for trouble.  Find out why it is failing.  If the test is bad, fix the test.  In the case of the expat bug, you would be opening your system up to exploits from malformed XML.  Really, don't force installs.

It is always nice when I get a chance to sit down and read a book I bought over year ago.  Finally!  I’ve been enjoying the book, and what I’ve enjoyed so far is the evolutionary process between attacks and web servers.

For example, Microsoft IIS was exploitable with long URLs.  Microsoft fixes that, but attackers learn that they can just keep running the attacks anyway, and eventually the server dies anyway when it runs out of disk space from logging all those long URLs.  Microsoft fixes that by reducing the length of information stored in the logs.  Attackers still continue to use long URLs, because their complete attempts won’t be logged.  You’ll know someone tried something with IIS, but you won’t know exactly what.  It is an interesting technology arms race.

The other thing that I’ve enjoyed so far about the book is the real stories about how systems have been audited, and how they find silly security flaws in the system.  Example:  Being able to view, edit, or become other accounts in the web application.

WebScarab is quite a useful tool to see what is going on between web browser and server.  And the ability to save all of the complete conversations within a browsing session is fantastic.  It is certainly going to be useful when I have to interface into websites that insist on using javascript for authentication and browsing.

While reading the SANS Internet Storm Center’s RSS feed, I found an interesting article on SQL Injections. http://isc.sans.org/diary.html?storyid=5416

The intriguing part was doing injections without the use of quote or semicolons.  Which allowed me to do some injections of my own on a production server!

It really is very important to treat any data coming from the Internet as ‘tainted’, and sanitize it.  If you think it can’t be ‘tainted’, download a copy of Opera.  Visit the web page with Opera.  Use View Source, and edit away!  Change all the default values you expect to unsafe data.  Click the Apply Changes button.  Use your altered form to inject with.

Notes from my experience:

1. Remove debugging messages that show the final SQL statement.  Showing the statement allows the attacker to see what they need to change in their input to cause the SQL to behave badly.  Note:  Showing "SQL error" is just as bad!  Better to say, "No records found".
2. Log SQL errors.  Any form input in production should never generate a SQL error.
3. Test, test, and retest.  Once you know what to look for, share what you have found with the rest of your development team.  Security is paramount in this day and age.
4. Code review.  Look for code that doesn’t sanitize input.
5. Add these checks into your routine testing process.
6. Try not to laugh when using PHP and security in the same sentence.  It can be done, really!  And secure PHP code can be written, really!

1. That can't happen.
2. That doesn't happen on my machine.
3. That shouldn't happen.
4. Why is that happening?
5. Oh, I see.
6. How did that ever work?

Logging to a file

Using search.cpan.org there are many options to logging to a file.  Finding just one that does exactly what you need can be a challenge.

Here, I will talk just about one:  Log::LogLite

LogLite was almost exactly what I was looking for, but I found it to be lacking in one area:  The date output formatting was most useless, and not customizable. "YYYY-DD-MM"!?  Why not YYYY-MM-DD like most computer programmers like, since it alpha sorts correctly?

A simple change near line 69 like so:

my $line = $self->{TEMPLATE};
  $line =~ s!<date>!$self->date_string()!igoe;
  # changed to $self->date_string() so module can be customized by deriving from.

And now I can derive modules from Log::LogLite and customize the date formatting.

package MyLog;
use Log::LogLite;
 
@ISA = ("Log::LogLite");
 
sub date_string {
 my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
 return sprintf( "%02d/%02d/%04d %02d:%02d:%02d",
  $mon + 1, $mday, $year + 1900, $hour, $min, $sec );
} # of date_string

But since I have to modify the original module in order to have the flexibility to customize it, I just renamed the module LogLite.pm to MyLog.pm, made the above two changes and copy it around to where I need it. 

So much for code reuse.

Creating the RSA Key

ssh-keygen -t rsa

This will create the id_rsa and id_rsa.pub files. The id_rsa.pub file located in ~/.ssh is your public key. Copy it to the system you want to automatically login to. See `man ssh-keygen` for more options. Why do I only show RSA keys? Because it's safer then the DSA keys according to the putty developers (see section 8.2.2) . I'm sure that all the bugs in DSA have been fixed by now, including the key generation problems, but why take unnecessary risks?

Public Key Adding Script

#!/bin/sh
# September 23, 2003 (initial)
# Updated August 23, 2006 (updated)
#
# Steve Thielemann
#
# A script file to add ssh key
#
# 1.) Verifies ~/.ssh directory exists, if not creates and
#     fixes permissions.
# 2.) Verifies ~/.ssh/authorized_keys exists, if not creates and
#     fixes permissions.
# 3.) Checks to see if ssh key is already present, if not adds to
#     ~/.ssh/authorized_keys.
#
# dependant on:  grep, touch, chmod, cut
#
if [ -z $1 ]
then
echo "I need the filename of a public key to add."
echo "(The .pub file created from ssh-keygen -t rsa)"
echo "Example:  $0 remote_rsa.pub"
exit
fi
if [ -f $1 ]
then
keyid=`cut -f3- '-d ' < $1`
else
echo "I can't find $1 !"
exit
fi
echo "Using ID $keyid"
# variables to make life easier
sshdir=~/.ssh
keyfile=~/.ssh/authorized_keys
if [ ! -d "$sshdir" ]
then
echo "Making $sshdir"
mkdir "$sshdir"
chmod 700 "$sshdir"
fi
if [ ! -f "$keyfile" ]
then
echo "Creating $keyfile"
touch "$keyfile"
chmod 644 "$keyfile"
fi
grep "$keyid" "$keyfile" > /dev/null
if [ $? = 1 ]
then
echo "Adding key to $keyfile"
cat $1 >> "$keyfile"
else
echo "Key $keyid already present."
fi
# the end

Or: download the auto SSH script .

Notes About the Script

The script doesn't do anything fancy, and once it has been run, adding additional keys is as simple as doing cat id_rsa.pub >> ./ssh/authorized_keys ! But, if the system hasn't been rsa public keyed before, this script will take care of setting all the file permissions correctly.

Configuring SSH to use the RSA Key

In your ~/.ssh/config file, add the following lines: Host nameyoucallsite Hostname 10.0.0.1 (or hostname) IdentityFile ~/.ssh/id_rsa User usernameonsite The hostname, if you set it with an IP address, it will always work (assuming your internet connection is working). If you use a hostname, it will only work if DNS is working. (Hint, if the server you are sshing into is the DNS server for that domain, make sure you have the IP address somewhere just in case!) User is the login name for that box. The IdentityFile is the file that you created, I usually create many keys, one for each project that I am involved in. Once configured, doing `ssh nameyoucallsite` is all that is required to login to the site.