My gut feel was that it was the disclaimer being inserted by an intermediary gateway ( one has to love corpmail).  Setting about proving this was the hard part.  The first issue at hand wa to actually extract the certificates so I could play with the verification.  Cert Viewer Plus for Thunderbird made this part a dream. Creating a modified version of the signed message was a little bit more problematic.

Trusting the command line, I started hunting around for details on OpenSSL support for SMIME, which it has.  OpenSSL needs a full CA path for being able to verify SMIME signed messages. One can obtain this from various places ( such as exporting form your browser) but in a case like this where a private hierarchy was being used, its enough to just make used of a somewhat smaller subset contianing only the certificates used in this chain.  These can be extracted using Cert Viewer Plus. Alternately some command line magic can be used to extract the PKCS7 formatted embedded certificates out in standard PEM format., using the following command:

openssl smime -pk7out -in mail.txt | \
openssl pkcs7 -print_certs > extract.crt

Now that we have a certificate chain we can attempt the verify. The extract.crt below can be either from the openssl method above or the Cert Viewer plus PEM dump.
openssl smime -CAfile extract.crt -verify -in mail.txt
Now we actually have a more usable error message. Although I really don’t know why I have such a deep distrust in GUI apps for actually telling me what is wrong.
Verification failure
88175:error:21071065:PKCS7 routines:PKCS7_signatureVerify:digest failure:/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/pkcs7/pk7_doit.c:808:
88175:error:21075069:PKCS7 routines:PKCS7_verify:signature failure:/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/pkcs7/pk7_smime.c:265:

As suspected the digest filed, which lead to a overall signature failure.  The next step was to see if removing the disclaimer worked.  Repeating on a slightly edited version of the the mail gave the following:

openssl smime -CAfile extract.crt -verify -in mail2.txt
...
mail contents deleted
...
Verification successful

So the original question posed was if the signature system was working correctly which it now was. The differences between the two mail files was checked using diff

diff -u mail.txt mail2.txt
--- mail.txt Mon Aug 25 18:06:33 2008
+++ mail2.txt Mon Aug 25 18:08:10 2008
@@ -61,10 +61,6 @@
South Africa

-Important Notice: This email is subject to important restrictions, qualifications
and disclaimers ("the Disclaimer") ..that all was one very long line that made
up the corporate disclaimer.....
...
------=_NextPart_000_0048_01C906C7.DB6FB700
Content-Type: application/x-pkcs7-signature;
name="smime.p7s"

From the above the only difference shown is that a mail gateway had added in a extra four lines of disclaimer and white space padding.  The question now evolves as to how to provide the now pretty much ubiquitous organizational disclaimer in outgoing mail in such a way that it doesn’t trash any cryptographic operations in which the mail is involved.  Ive gone back over mails from a  couple of other people in corporate South Africa that I know , and the problem seems to be widespread.

The solution may be that the disclaimer as such is encapsulated as a separate MIME component, which is what interestingly one university here does ( although it insists on prepending its mime encapsulated HTML disclaimer, which makes for really ugly mail reading!)

Apache’s mod_rewrite seems to be the obvious candidate for making this as transparent as possible. In essence what needed to happen was incoming requests for:

"GET /blog/?flav=rss&category=Security HTTP/1.0" and
"GET /blog/?flav=rss HTTP/1.0" respectively need to be converted into:


http://lair.moria.org/blog/archives/category/security/feed

and

http://lair.moria.org/blog/feed respectively

This was achieved matching on the QUERY_STRING variable within apache. The real trick came trying to get the new URLS to appear clean. This proved to be more difficult than I expected. My initial rewrite rules resulted int he following:

"GET /blog/?flav=rss HTTP/1.0" 301 249 "
"GET /blog/feed/?flav=rss HTTP/1.0" 200 54274 ""

The agent was directed to the right url but it still looks ugly. Note the use of a HTTP/301 status code indicating permanently moved rather than a 302 which mod_redirect usually provides. The solution to the appending of the query string turned out to be to force my own null string onto the redirect. The Apache Wiki was where I finally found the right answer. so the way to remove a QUERY_STRING is to append a blank string “?” to the redirect .

The final setup in my .htaccess for WordPress looks as follows:

RewriteCond %{QUERY_STRING} ^flav=rss$ [NC]
RewriteRule ^$ http://lair.moria.org/blog/feed? [R=301,L]
RewriteCond %{QUERY_STRING} ^flav=rss&(category)=Security$ [NC]
RewriteRule ^$ http://lair.moria.org/blog/archives/category/security/feed? [R=301,L]
RewriteCond %{QUERY_STRING} ^flav=atom$ [NC]
RewriteRule ^$ http://lair.moria.org/blog/feed/atom? [R=301,L]

Some issues specific to the Rhodes environment are noted at the end.

Get up to date:

aptitude update
aptitude upgrade
aptitude dist-upgrade

All fairly painless and out of the ordinary, bar the need to add the dapper-proposed repo to my /etc/apt/sources.list

Installing the requires base packages:

aptitude install update-manager-core

Before doing the upgrade I decided to set up the CDROM ISO as a local repository in order to save bandwidth whales etc.. (Having a system with real internet access, or a working apt-proxy may be a better solution). Once the Hardy DVD is available in a few weeks this may go a lot faster, as libraries such as QT and other components of main will be included in the larger image.

$ mount -t iso9660 -o loop ~bvi/ubuntu-8.04-server-i386.iso /cdrom
$ apt-cdrom add

This should show output similar tot he following as the CDROM is added tot he Repo list.

Using CD-ROM mount point /cdrom/
Unmounting CD-ROM
Waiting for disc...
Please insert a Disc in the drive and press enter
Mounting CD-ROM...
Identifying.. [b36baea778d37bbf48a3c8bd75b5cffb-2]
Scanning disc for index files..
Found 2 package indexes, 0 source indexes and 1 signatures
Found label 'Ubuntu-Server 8.04 _Hardy Heron_ - Release i386 (20080423.2)'
...


And should add a to the top of your /etc/apt/sources.list similar to the following:

deb cdrom:[Ubuntu-Server 8.04 _Hardy Heron_ - Release i386 (20080423.2)]/ hardy main restricted

Finally the update can be performed

do-release-update -p

The -p parameter is due to the fact that the LTS as defined at http://changelogs.ubuntu.com/meta-release-lts is still showing dapper as the LTS, and should be able to be omitted in the future. with the -p, the meta-release-lts.proposed file is used instead.

the bulk of the base operating system was happily upgraded form CDROM repo in a matter of minutes, and the remaining pile of mostly universe and multiverse packages took around an hour to download

3 hours and a reboot later and the server is happily running Hardy.

All in all its about the same time for doing a ‘buildworld dance’ with FreeBSD, along with a massive level of portupgrade.

Post Reboot

In order to validate the upgrade, we can make use of the Linux Standard Base support for Debian
utilities ( aka lsb_release)

We have gone from:

$ uname -a
Linux spy.ict.ru.ac.za 2.6.15-51-686 #1 SMP PREEMPT Tue Feb 12 16:59:15 UTC 2008 i686 GNU/Linux
$ lsb_release -a (output trimmed)
Distributor ID: Ubuntu
Description: Ubuntu 6.06.2 LTS
Release: 6.06
Codename: dapper

To:

$ uname -a
Linux spy.ict.ru.ac.za 2.6.24-16-server #1 SMP Thu Apr 10 13:58:00 UTC 2008 i686 GNU/Linux
$ lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 8.04
Release: 8.04
Codename: hardy

In reflection, far less pain than I expected certainly not enough to make me want to employ a depenguinator on this server yet.

Rhodes Specific notes:

• Preferably use ubuntu.rucus.ru.ac.za as your repo unless you would like your quota flattened. Hopefully it will be up.
• ftp://ftp.rucus.ru.ac.za/pub/linux/ubuntu/hardy has the ISO files
• You will need appropriate proxy settings in order for the do-release-upgrade to be able to access the changelogs.ubuntu.com site.

A couple of issues still need to be addressed:

• Rework the source to we can maintain a single source three for Windows and Unix targets- this is mostly slog rather than thinking work.
• There is a bug in that some images are coming out a little wrong, but I think this may have to do with line termination issues the good old \r\n vs. \n issue again.
• Write some decent docs!

With a bit of luck Nick should be returning to Grahamstown for a month or so thanks to some funding from the Center of Excellence in the department. Working on the Hilbert project will be one of his main priorities.

The basic process is about creating a magic memory File System 9MFS) based mini FreeBSD install one can then ddover the base MBR on the system This gets you jsut enough of a Real OS to carry on with the rest of your install.

What this means for me is the possibility of doing some nice rmeote upgrade, or more acuratlye nuke and paves on some remorely hosted equipment. The big cath of course is your remote systems should have sufficient bandwidth (, or at least a local copy of the ISO or CD handy and mounted. Exploring other hosting providers may also now be feasible, now that I can run my favorite server OS without having to try persuade remotes upport to just put a BSD CD in and let me pay for a KVM access window :-)