Static in the Ether » Networking

Operating Systems seen on an African Network Telescope

Barry Irwin — Tue, 11 Aug 2009 10:01:44 +0000

I have been processing some of my network telescope data collected over the last four and a bit years. During this time I have classified a little over 3.2 million IP addresses by operating system making use of p0f

The results after the latest updates are:

OS Family		%
Windows		98.84258
Linux		0.811703
FreeBSD		0.170989
Proxyblocker		0.078751
NetBSD		0.030808
MacOS		0.02954
Other		0.035633

Windows is significant, although there has been a distinct scew towards this OS due to Conficker propagation, its still worth noting that prior to the last 7 months being imported data though December 2008 showed windows only 0.8% down on the values above, roughly evenly split between positions 2 & 3.. At the bottom end of the scale some interesting artifacts.

OS Family	Count

NMAP	151
OpenBSD	53
CacheFlow	46
Cisco	42
SunOS	27
Redline	27
Google	16
Eagle	11
HP	11
PocketPC	11
Checkpoint	10
ExtremeWare	5
BSD	3
Tru	1
NewtonOS	1
NetCache	1
SCO	1

Converting Internet Barometer Data

Barry Irwin — Tue, 30 Jun 2009 23:20:42 +0000

My first foray into the tag soup that is XSL and XSLT has been to turn the XML outputs from the InterNet Barometer System as discussed previously into plain text output which I can use more easily for comparing with some of my other data sources. While A cursory browse cannot find any Terms & conditions for the use of this data, I think I’m on safe ground given that all I’m doing is processing the same xml that is consumed by the flash objects and its not for any kind of commercial use. After hunting around for tools, and wasting a pile of bandwidth on “enterprise editions” I ended up constructing this based on some tutorials at w3c.org using good old vim. I was very tempted to just revert back to sed & awk, or even try my hand at python’s parsing, but decided that I may as well ‘do it right’. The result of a few hours work this evening while watching a filesystem rebuild is shown below:
, , , ,
This through the magic of xsltproc produces a nice plain text output:

xsltproc map2.xsl asia.xml

given the input from the Asia attack graph produces:

30-06-2009 05:00:17 GMT,RU,Russia,15387,green 30-06-2009 05:00:17 GMT,TR,Turkey,7137,green 30-06-2009 05:00:17 GMT,CN,China,2468,green 30-06-2009 05:00:17 GMT,MY,Malaysia,4158,green 30-06-2009 05:00:17 GMT,IN,India,2631,green 30-06-2009 05:00:17 GMT,TH,Thailand,1823,green

While not the most elegant code, its gets done what I need, and is easily extensible enough to be able to transform to other formats suitable for DB import. I’ll need to monitor data over the next couple of days to get an idea as to how the counters used are actually operating. Once that has been established I can star doing some meaningful comparisons.

A Mafia take on the OSI stack

Barry Irwin — Wed, 29 Oct 2008 08:29:39 +0000

Laura A. Robinson has a nice piece describing the traditional OSI stack in terms of the negotiation of a meeting between two mafia a Dons. While probably not for the technically minded ( I’m sure most people in the networking and security fields have their own little mnemonics to remember the PDNTSPA acronym), I think it may serve as a useful example when illustrating things to people who have at least seen the Godfather trilogy or the Sopranos.

C64 Web Server

Barry Irwin — Thu, 23 Oct 2008 21:11:45 +0000

I stumbled across this page where Shane Wood is is running a full Webserver off his 1982 Commodore 64. Maybe there really is the Use for my C64 that’s sitting in my garage.

I game across this via a link from the Contiki Project who has also jsut announced the availability of their uIPv6 Stack that has passed all the standard conformance tests. Ipv6 has now truely come to the world of constrianed memory devices. This is definatley something I want to play with post PhD

Security and Networks Research Group (SNRG) Site launch

Barry Irwin — Tue, 02 Sep 2008 09:26:39 +0000

After some preparation and navigation of technical SNAFUs the new website for the Security and Networks research Group (SNRG) that I run in the Rhodes CS Department is up and running.

While content is still a little thin on the ground, it does represent a major step forward in actually providing a point of collation of project information under our own control. A large task to be performed next term is to actually backfill with old project information as we can get it off CD.

Verifying Smime content with openSSL

Barry Irwin — Mon, 25 Aug 2008 16:51:03 +0000

I had an interesting question posed ot me today by Dominic who asked me to verify whether his all new Digital certificate was correctly being used for signing mail. Thunderbird sadly complained that the signature was invalid, which was unexpected, and that the issuer was unknown ( expected since it comes form a private hierarchy.) The question then lead to where did the problem lie?

My gut feel was that it was the disclaimer being inserted by an intermediary gateway ( one has to love corpmail). Setting about proving this was the hard part. The first issue at hand wa to actually extract the certificates so I could play with the verification. Cert Viewer Plus for Thunderbird made this part a dream. Creating a modified version of the signed message was a little bit more problematic.

Trusting the command line, I started hunting around for details on OpenSSL support for SMIME, which it has. OpenSSL needs a full CA path for being able to verify SMIME signed messages. One can obtain this from various places ( such as exporting form your browser) but in a case like this where a private hierarchy was being used, its enough to just make used of a somewhat smaller subset contianing only the certificates used in this chain. These can be extracted using Cert Viewer Plus. Alternately some command line magic can be used to extract the PKCS7 formatted embedded certificates out in standard PEM format., using the following command:

openssl smime -pk7out -in mail.txt | \ openssl pkcs7 -print_certs > extract.crt

Now that we have a certificate chain we can attempt the verify. The extract.crt below can be either from the openssl method above or the Cert Viewer plus PEM dump.
openssl smime -CAfile extract.crt -verify -in mail.txt
Now we actually have a more usable error message. Although I really don’t know why I have such a deep distrust in GUI apps for actually telling me what is wrong.
Verification failure 88175:error:21071065:PKCS7 routines:PKCS7_signatureVerify:digest failure:/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/pkcs7/pk7_doit.c:808: 88175:error:21075069:PKCS7 routines:PKCS7_verify:signature failure:/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/pkcs7/pk7_smime.c:265:
As suspected the digest filed, which lead to a overall signature failure. The next step was to see if removing the disclaimer worked. Repeating on a slightly edited version of the the mail gave the following:

openssl smime -CAfile extract.crt -verify -in mail2.txt ... mail contents deleted ... Verification successful

So the original question posed was if the signature system was working correctly which it now was. The differences between the two mail files was checked using diff

diff -u mail.txt mail2.txt --- mail.txt Mon Aug 25 18:06:33 2008 +++ mail2.txt Mon Aug 25 18:08:10 2008 @@ -61,10 +61,6 @@ South Africa-Important Notice: This email is subject to important restrictions, qualifications and disclaimers ("the Disclaimer") ..that all was one very long line that made up the corporate disclaimer..... ... ------=_NextPart_000_0048_01C906C7.DB6FB700 Content-Type: application/x-pkcs7-signature; name="smime.p7s"
From the above the only difference shown is that a mail gateway had added in a extra four lines of disclaimer and white space padding. The question now evolves as to how to provide the now pretty much ubiquitous organizational disclaimer in outgoing mail in such a way that it doesn’t trash any cryptographic operations in which the mail is involved. Ive gone back over mails from a couple of other people in corporate South Africa that I know , and the problem seems to be widespread.

The solution may be that the disclaimer as such is encapsulated as a separate MIME component, which is what interestingly one university here does ( although it insists on prepending its mime encapsulated HTML disclaimer, which makes for really ugly mail reading!)

Firefox 3.0, crashes and bandwidth overload

Barry Irwin — Tue, 17 Jun 2008 19:43:28 +0000

Some two and a half hours into the FF3.0 download campaign, and the toll is beginning to show.spreadfirefox.com seems to be refusing connections. While individual mirror sites seem up, it looks like the counts are going via some redirector script. getfirefox.com is however working fine.

The impact its having on mirrors seems to be quite intense. The following two images sow traffic stats from mirror.ac.za the mirror service run by TENET here in South Africa.

Update:

A Firefox 3.0 download counter is now available. 943806 currently averaging some 7000/minute. Some commentary on the outages, although they seem to have cleared.

New Hilbert Release

Barry Irwin — Wed, 04 Jun 2008 07:13:05 +0000

With Nick now in Grahamstown, development on the Hilbert Curve application has progressed well. Version 4.05 has been released around a month after the 2.05 edition previously mentioned, which is heading much closer towards completion. The unix build scripts still need to be integrated, but there has been much improvement. The most noticeable improvements are in the processing speed new around 90 seconds for a datafile of 53 million Addresses, and its ability to put out some very high res images (4096×4096) when working with higher order curves. At this resolution we are able to present a single pixel as representing a class C network or in effect 256 individual IP addresses. The updated release also allows for the application of image overlays when in interactive mode, which can make navigation significantly easier.

A sample of the kind of output is seen below (full resolution image is 990K) which shows destination IP addresses harvested from the Albany Schools Cache server during January through May 2008:

A plot of 53 million packets from the CAIDA telescope project – 27 Feb 2007 midnight to 6am:

With these higher resolution images available, analysis can be performed at a much finer grained level.

Internet Usage climbing

Barry Irwin — Sat, 31 May 2008 11:03:32 +0000

I spend part of my time providing network management and consulting services to a consortium of local schools – The Albany Schools Network (ASN). We have spent a significant amount of time over the last three years migrating form an ancient legacy system consisting of a decrepit SparcStation 5, and some ancient old Cisco Routers (AGS+ and CGS+ – both of which were eol’d in 1997) linked to nominally 33.6Kbit dedicated analog lines.

Thankfully we have come a long way and now boast a healthy modern network with peering at GINX. Early last year saw the introduction of the first of our 1Mbit DSL lines form Internet Solutions, the first real boost of bandwidth available to the schools, and something worthwhile doing since the actual links to the schools had been upgraded.

Looking over some stats I pulled up the following two reports from LightSquid while not the most scientific they convey the massive growth in utilization for the schools. While these represent the larges of the schools in terms of usage, the grown is pretty similar for all members of the consortium.

The first image is from may last year:

The second was generated this morning showing a somewhat heavier utilization. It is worth noting that during this period, the schools link was upgraded from 384Kbit to over a Megabit.

I find it extremely gratifying to see these schools making such full use of the resources at hand!

Gen3 Migration Complete

Barry Irwin — Sat, 31 May 2008 07:45:24 +0000

After a couple of false starts, Rhodes University has finally transitioned off the older Telkom Provides Gen2 ( although for the last few months we have been using the Telkom commercial Internet Service) to the New TENET Gen3 network being serviced by Neotel, and Internet Solutions. Probably the biggest difference is that TENET now gets a Layer 2 service with the net network rather than the Layer 3 which they previously had.

A almost unnoticeable transition, thanks to the hard work of the Rhodes and Neotel teams.

Before:

$ traceroute www.google.com
traceroute to www.l.google.com (64.233.183.147)
1  ict.gw.ru.ac.za (146.231.120.1)  0.533 ms  0.370 ms  0.364 ms
2  core-struben.gw.ru.ac.za (146.231.0.2)  0.363 ms  0.212 ms  0.363 ms
3  tenet.gw.ru.ac.za (192.42.99.1)  1.139 ms  1.143 ms  1.141 ms
4  * * *
5  196.43.9.54 (196.43.9.54)  182.751 ms  179.151 ms  189.128 ms
6  83.245.76.221 (83.245.76.221)  184.445 ms  201.326 ms  190.072 ms
7  209.85.252.42 (209.85.252.42)  175.554 ms  176.190 ms  176.808 ms
8  216.239.43.123 (216.239.43.123)  322.346 ms  190.382 ms  221.579 ms
9  72.14.233.79 (72.14.233.79)  191.322 ms
72.14.233.77 (72.14.233.77)  193.961 ms  193.978 ms
10  209.85.249.133 (209.85.249.133)  306.756 ms
216.239.43.30 (216.239.43.30)  194.754 ms
216.239.43.34 (216.239.43.34)  227.038 ms
11  nf-in-f147.google.com (64.233.183.147)  214.869 ms  229.846 ms  221.110 ms

Currently:

$ traceroutewww.l.google.com
traceroute to www.l.google.com (64.233.183.147)
 1  ict.gw.ru.ac.za (146.231.120.1)  0.538 ms  0.377 ms  0.376 ms
 2  core-struben.gw.ru.ac.za (146.231.0.2)  0.222 ms  0.382 ms  0.364 ms
 3  tenet.gw.ru.ac.za (192.42.99.1)  1.138 ms  0.988 ms  1.297 ms
 4  unknown.uni.net.za (155.232.145.83)  18.314 ms  16.744 ms  17.523 ms
 5  unknown.uni.net.za (155.232.145.226)  17.519 ms  17.838 ms  16.119 ms
 6  unknown.uni.net.za (196.32.209.25)  193.022 ms  207.077 ms  189.129 ms
 7  ldn-tch-i1-link.telia.net (213.248.79.193)  187.563 ms  188.824 ms  193.184 ms
 8  ldn-b1-link.telia.net (80.91.250.209)  189.126 ms  186.783 ms  190.687 ms
 9  ldn-bb2-link.telia.net (80.91.248.94)  192.086 ms  188.977 ms  188.036 ms
10  adm-bb2-pos6-0-0.telia.net (213.248.65.158)  210.237 ms  197.275 ms  200.360 ms
11  adm-b1-link.telia.net (80.91.252.21)  202.852 ms  197.879 ms  206.444 ms
12  google-ic-126116-adm-b1.c.telia.net (80.239.193.182)  203.944 ms  242.015 ms  205.974 ms
13  209.85.251.12 (209.85.251.12)  233.273 ms
    209.85.251.14 (209.85.251.14)  208.804 ms  206.761 ms
14  209.85.248.79 (209.85.248.79)  223.135 ms  210.032 ms  209.098 ms
15  * 72.14.233.79 (72.14.233.79)  230.516 ms  226.125 ms
16  209.85.249.129 (209.85.249.129)  243.568 ms
    216.239.43.34 (216.239.43.34)  230.785 ms
    209.85.249.129 (209.85.249.129)  210.033 ms
17  nf-in-f147.google.com (64.233.183.147)  221.588 ms  210.659 ms  211.602 ms

So a few more hops but much the same. Now to sit tight for the next 13 months until the SeaCom cable hopefully comes onstream!

Stable builds

Barry Irwin — Wed, 21 May 2008 21:37:02 +0000

I was cleaning out some old notes the other day and came across a pile of work relating to the development of some custom FreeBSD firewall solutions that Neil and I developed while at Itouch Labs, many years back in 2002.

The majority of the code is completely out dated, given these were based on the the then bleeding edge 4.3 branch. The only thing thats probably of value, is to sometime dust off the extensions we did to natd to provide an interactive console and the ability to reload rules without dumping its state table. Possibly post PhD.

I also found a link to a post made by Arri when he decommissioned them some four and a bit years later, with some impressive uptimes:

4.3-RELEASE-p28 FreeBSD 4.3-RELEASE-p28 #0
8:56AM up 1175 days, 14:25, 1 user, load averages: 0.01, 0.00, 0.00
4.3-RELEASE FreeBSD 4.3-RELEASE #3: Thu Aug 9 08:24:10 SAST 2001
8:55AM up 1353 days, 13:07, 1 user, load averages: 0.07, 0.03, 0.004.3-RELEASE FreeBSD 4.3-RELEASE #3: Thu Aug 9 08:24:10 SAST 2001
8:57AM up 1636 days, 12:16, 2 users, load averages: 0.01, 0.02, 0.00

BSD is great to work with! I see that the 2nd edition of Absolute FreeBSD is now finally available, and am eagerly awaiting for my copy to arrive.

RFC BibTeX resource

Barry Irwin — Sun, 18 May 2008 17:30:24 +0000

Roland Bless, has a rather useful resource of a set of BibTex information for all RFC documents for those working with RFC’s and needing to cite them using BibTex. Available for download is an automatically generated .bib database of all the current RFCs.

The 1.8 meg .bib file is probably a little large for general use but once can easily trim and copy entries required manually or using JabRef. Citations look like the following:

@MISC{rfc1466, author = {E. Gerich}, title = {{Guidelines for Management of IP Address Space}}, howpublished = {RFC 1466 (Informational)}, month = may, year = {1993}, note = {Obsoleted by RFC 2050}, number = {1466}, organisation = {Internet Engineering Task Force}, publisher = {IETF}, series = {Request for Comments}, timestamp = {2008.05.18}, url = {http://www.ietf.org/rfc/rfc1466.txt} }

A resource certain to save typing or multiple c & p operations. The one possible change one may want to make is to include the RFC number in the document tile such as:

title = {{RFC 1466: Guidelines for Management of IP Address Space}}

Another changes may be to use the @TechReport type as opposed to @Misc. An other alternative (although out of date) is the repository at University of Utah Maths Department.

Related to this the W3C have a web page which allows for automated generation of bibTeX citation information for their publications.