Static in the Ether

I had an interesting question posed ot me today by Dominic who asked me to verify whether his all new Digital certificate was correctly being used for signing mail. Thunderbird sadly complained that the signature was invalid, which was unexpected, and that the issuer was unknown ( expected since it comes form a private hierarchy.)  The question then lead to where did the problem lie?

My gut feel was that it was the disclaimer being inserted by an intermediary gateway ( one has to love corpmail).  Setting about proving this was the hard part.  The first issue at hand wa to actually extract the certificates so I could play with the verification.  Cert Viewer Plus for Thunderbird made this part a dream. Creating a modified version of the signed message was a little bit more problematic.

Trusting the command line, I started hunting around for details on openssl support for SMIME, which it has.  OPenSSL needs a full CA path for being able to verify SMIME signed messages. One can optain this from various places ( such as exporting form your browser) but in a case like this where a private hierarchy was being used, its enough to just make used of a somewhat smaller subset contianing only the certificates used in this chain.  These can be extracted using Cert Viewer Plus. Alternately some command line magic can be used to extract the PKCS7 formatted embedded certificates out in standard PEM format., using the following command:

openssl smime -pk7out -in mail.txt | \
        openssl pkcs7 -print_certs > extract.crt

Now that we have a certificate chain we can attempt the verify. The extract.crt below can be either from the openssl method above or the Cert Viewer plus PEM dump.

openssl smime -CAfile extract.crt -verify -in mail.txt

Now e actually have a more usable error message. Although I really don’t know why I have such a deep distrust in GUI apps for actually telling me what is wrong.

Verification failure
88175:error:21071065:PKCS7 routines:PKCS7_signatureVerify:digest failure:/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/pkcs7/pk7_doit.c:808:
88175:error:21075069:PKCS7 routines:PKCS7_verify:signature failure:/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/pkcs7/pk7_smime.c:265:

As suspected the digest filed, which lead to a overall signature failure.  The next step was to see if removing the disclaimer worked.  Repeating on a slightly edited version of the the mail gave the following:

openssl smime -CAfile extract.crt -verify -in mail2.txt
...
mail contents deleted
...
Verification successful

So the original question posed was if the signature system was working correctly which it now was. The differences between the two mail files was checked using diff

diff -u mail.txt  mail2.txt
--- mail.txt         Mon Aug 25 18:06:33 2008
+++ mail2.txt      Mon Aug 25 18:08:10 2008
@@ -61,10 +61,6 @@
    South Africa

-Important Notice: This email is subject to important restrictions, qualifications
 and disclaimers ("the Disclaimer") ..that all was one very long line that made
 up the corporate disclaimer.....
-
-
-
 ------=_NextPart_000_0048_01C906C7.DB6FB700
 Content-Type: application/x-pkcs7-signature;
        name="smime.p7s"

From the above the only difference shown is that a mailgateway had added in a extra four lines of disclaimer and whitespace padding.  The question now evolves as to how to provide the now pretty much ubiquitious organisational disclaimer in outgoing mail in such a way that it doesnt trash any cryptographic operations in which the mail is involved.  Ive gone back over mails from a  couple of other people in corporate South Africa that I know , and the problem seems to be widespread.

The solution may be that the disclaimer as such is encapsulated as a separeate mime component, which is what interestingly one university here does ( although it insists on prepending its mime encapsulated HTML disclaimer, which makes for relaly ugly mail reading!)

Something I usually ask my students to do us to draw up a frequency table of their use of references in their Theses. This is useful to see if one is over-citing particular sources, or disregarding sources that are more significant. Up until now most have gone the paper and pencil route.

Fred Otten came up with the following script using good old sed, awk and some plumbing, that draws up a nice list based on an input Lyx file.


#!/bin/sh
cat $1 | grep key\ \" |\
awk '{ print substr($2,2,length($2)-2)}' | \
sed -e s/,/\\n/g | \
awk 'BEGIN {i=0} \
{ if (temp[$1]) { temp[$1]=temp[$1]+1 } \
else { temp[$1]=1; tmp[i]=$1; i++; }; } \
END { for (j=0; j { print tmp[j] ” ” temp[tmp[j]] } }’ | sort


This gives a two column listing of the citation keys and their frequency count. This of course can be extended using further awk statements to transpose the columns, or sort by frequency, rather than citation key.

Jabref ships with a rather useful HTML export feature, which produces some rather nice output for publishing a somewhat interactive version of your .bib files online, or even as an electronic appendix to a thesis.

Mark Schenk has proviced some nice eexamples of other styles of export that one can use. Using these in conjunction with the Custom Export scripting built into Jabref, one should be able to achieve pretty much any kind of format or data manglin of references that you would require.

Roland Bless, has a rather useful resource of a set of BibTex information for all RFC documents for those working with RFC’s and needing to cite them using BibTex. Available for download is an automatically generated .bib database of all the current RFCs.

The 1.8 meg .bib file is probably a little large for general use but once can easily trim and copy entries required manually or using JabRef. Citations look like the following:

@MISC{rfc1466,
author = {E. Gerich},
title = {{Guidelines for Management of IP Address Space}},
howpublished = {RFC 1466 (Informational)},
month = may,
year = {1993},
note = {Obsoleted by RFC 2050},
number = {1466},
organisation = {Internet Engineering Task Force},
publisher = {IETF},
series = {Request for Comments},
timestamp = {2008.05.18},
url = {http://www.ietf.org/rfc/rfc1466.txt}
}

A resource certain to save typing or multiple c & p operations. The one possible change one may want to make is to include the RFC number in the document tile such as:


title = {{RFC 1466: Guidelines for Management of IP Address Space}}


Another changes may be to use the @TechReport type as opposed to @Misc. An other alternative (although out of date) is the repository at University of Utah Maths Department.

Related to this the W3C have a web page which allows for automated generation of bibTeX citation information for their publications.