Static in the Ether » Security Tools http://lair.moria.org/blog Unix, Information Security & Systems Administration Wed, 11 Nov 2009 09:14:29 +0000 en hourly 1 http://wordpress.org/?v=3.0 Converting Internet Barometer Data http://lair.moria.org/blog/archives/214 http://lair.moria.org/blog/archives/214#comments Tue, 30 Jun 2009 23:20:42 +0000 Barry Irwin http://lair.moria.org/blog/?p=214 My first foray into the tag soup that is  XSL and XSLT has been to turn the XML outputs from the InterNet Barometer System as discussed previously into plain text output which I can use more easily for comparing with some of my other data sources. While A cursory browse cannot find any Terms & conditions for the use of this data, I think I’m on safe ground given that all I’m doing is processing the same xml that is consumed by the flash objects and its not for any kind of commercial use. After hunting around for tools, and wasting a pile of bandwidth on “enterprise editions” I ended up constructing this based on some tutorials at w3c.org  using good old vim. I was very tempted to just revert back to sed & awk, or even try my hand at python’s parsing, but decided that I may as well ‘do it right’. The result of a few hours work this evening while watching a filesystem rebuild is shown below:

<?xml version='1.0'?>
<xsl:stylesheet version="1.0"xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<!-- Barry Irwin bvi@moria.org XSL format for translating XML from interoute Barometer output -->
<xsl:output method="text"/>
<xsl:strip-space elements="*"/>
<xsl:template match="area">
<xsl:value-of select="../../allData/lastUpdated"/>
<xsl:text>,</xsl:text>
<xsl:value-of select="@mc_name"/>
<xsl:text>,</xsl:text>
<xsl:value-of select="@title"/>
<xsl:text>,</xsl:text>
<xsl:value-of select="@value"/>
<xsl:text>,</xsl:text>
<xsl:value-of select="@colour"/>
<xsl:text>
</xsl:text> </xsl:template>
<xsl:template match="allData">
</xsl:template>

This through the magic of xsltproc produces a nice plain text output:

xsltproc map2.xsl asia.xml

given the input from the Asia attack  graph produces:

30-06-2009 05:00:17 GMT,RU,Russia,15387,green
30-06-2009 05:00:17 GMT,TR,Turkey,7137,green
30-06-2009 05:00:17 GMT,CN,China,2468,green
30-06-2009 05:00:17 GMT,MY,Malaysia,4158,green
30-06-2009 05:00:17 GMT,IN,India,2631,green
30-06-2009 05:00:17 GMT,TH,Thailand,1823,green

While not the most elegant code, its gets done what I need, and is easily extensible enough to be able to  transform to other formats suitable for DB import. I’ll need to monitor data over the next couple of days to get an idea as to how the counters used are actually operating. Once that has been established I can star doing some meaningful comparisons.

]]> http://lair.moria.org/blog/archives/214/feed 0 Internet Attack Barometer http://lair.moria.org/blog/archives/205 http://lair.moria.org/blog/archives/205#comments Tue, 30 Jun 2009 08:16:51 +0000 Barry Irwin http://lair.moria.org/blog/?p=205 Interoute has launched a new online Internet Barometer detailing attacks as observed from their 22 monitoring stations across the European portion of the Internet.

The site provides rich graph and chart interfaces, which are nicely interactive.  There are definatley some ideas I want to incorporate form this into my own Network Telescope management console.  It is however worth bearing in mind that his is a Eurocentric view and is only based on their observed traffic. As such the “attacking countries” view seems to be a bit skewed.

Interoute World view 2009-06-30

Interoute World view 2009-06-30

After digging around with squid and wireshark, its evident that a lot of the data is actually served up as XML files, and as such can potentially be postprocessed. The Adobe AIR Barometer Widget they provide also makes use of these. One issue I had getting this installed is you need Air 1.5.1, and the 1.0.8 version I had wouldn’t auto upgrade correctly.  A little disappointing in that I was expecting a map view, it provides the basics of a total count and cycles through various country stats.

Interroute Barometer Widget

Interoute Barometer Widget

Where the real value  comes form is having another independent source of reporting ( even at the highly granular level) that can be used to correlate observations with my own data sets, and those available form places like dShield and ISC. Maybe I should dust off my old Infocon alert plugin for Firefox and integrate some of this data.

]]> http://lair.moria.org/blog/archives/205/feed 1 Security Tools update http://lair.moria.org/blog/archives/179 http://lair.moria.org/blog/archives/179#comments Fri, 06 Feb 2009 21:48:53 +0000 Barry Irwin http://lair.moria.org/blog/?p=179 Over the last week or so a number of new tools have been release either for the first time or as updated versions:

  • tcpreplay is now at version 3.4.0 with a number of significant bugfixes. This staple of packet analysis allows for the replay of captured pcap file back over network interfaces.  Its a great way of having a repeatable test framework, or for explosing yout NIDS system to collected bad traffic.
  • picviz 0.5 has been released. I blogged about this before and the project seems to be comming on nicely. Formy own purposes its not much use with my network telescope data, but does produce some pretty pictures for some other work Ive been doing of late. The new version comes with a bumber of new log parsers. A slide deck discussing its use as presented at USENIX 2008 is also available.
  • pcapr is the new tool out and describes itself as “web 2.0 meets packets“, and “pcapr does to packets what flickr does to pictures”. If it performs as promised it could make life a lot easier maintinaing libraries of packet captures. The fact its a hosted service does have some distinct disadvantages. Currently there seems to be quote a lot of little snippets. An RSS feed of new content is also available.  Another similar repository s that of openpacket.org
  • libtrace while not a new tool as such, is somethign I’ve started workign with recentlyafter comming across it in Dean Pemberton’s MSc Thesis2007 on Internet Background Radiation Arrival Density and Network Telescope Sampling Strategies. The api looks pretty clean and it comes with a couple of nice demo tools which are actually useful.  the URI syntax it uses for accessing files is a little strange but managable.
]]> http://lair.moria.org/blog/archives/179/feed 1 Applied Security Visualization released http://lair.moria.org/blog/archives/106 http://lair.moria.org/blog/archives/106#comments Thu, 21 Aug 2008 06:54:44 +0000 Barry Irwin http://lair.moria.org/blog/?p=106 I probably should have posted this a while back but, its still worth noting that Raffael Marty’s Applied Security VisualizationApplied Security Vizualisation has been released, and includes a copy of the DAVIX CD as distributed at Defcon 16 (davix-1.0.1-defcon16.iso.gz – also obtainable from the homepage, includes a couple of packet traces as used in the Defcon workshop) , which includes a copy of InetVis as one of their four chosen visual analysis tools on the live CD.

For the impatient some instructions are available for getting started. Now to work out what else to order form amazon so that the 40USD specail shipment fee hurts a little less ( thanks to our totally criminal and incompetent post office in South Africa)

Failing that its time to wait till the local places get round to stocking it.

]]> http://lair.moria.org/blog/archives/106/feed 0 Defcon16 Toolsets http://lair.moria.org/blog/archives/94 http://lair.moria.org/blog/archives/94#comments Wed, 20 Aug 2008 21:29:14 +0000 Barry Irwin http://lair.moria.org/blog/?p=94 With the 16th incarnation of Defcon having come and gone last week, a number of people have put together a nice list of the various tools released. The ZDnet’s Rob Fuller has done all the hard work of tracking down the various tools and their websites in his article -  entitled “ DEFCON 16: List of tools and stuff released
“seems to be the most definitive.  Another (updated) list is on Rob’s personal site in which he includes some other items like Packet-O-Matic, PE-Scambler andVMware Pen-Testing Framework, alng with a link to the ISO.

Of all the tools release its DAVIX, that makes me happiest, other than it being a relaly slick Compilation of VizSec tools, it also features InetVis, which is a part of the postgraduate research by one of my students (Jean-Pierre van Riel), which I previously posted about.

iKat is the other tool that tickles my fancy.

There are some interesting setf odf Defcon Photos floating around such as these by sits, who has also made available a zip of the of the Defcon 16 CD contents.An 732MB ISO  version is also available that at least has checksums.

I see some fun times ahead!

]]> http://lair.moria.org/blog/archives/94/feed 3 DAVIX live CD looking for Beta Testers http://lair.moria.org/blog/archives/55 http://lair.moria.org/blog/archives/55#comments Sun, 15 Jun 2008 19:34:14 +0000 Barry Irwin http://lair.moria.org/blog/?p=55 DAVIX is the upcoming live CD for data analysis and visualization, which will be released at Blackhat/DEFCON in Las Vegas this summer, with another talk at VizSec 2008. From the VizSec.org announcement:

Applied Security VizualisationJan Monsch and Raffael Marty and have prepared the second beta version of DAVIX. And are now seeking for beta testers that have the time to test DAVIX and answer the questionnaire that comes along with the beta version. All completely filled out questionnaires received by me until Monday 23 June 2008 18:00 UTC will enter a raffle for one autographed copy of Raffy’s upcoming book “Applied Security Visualization”.

If you want to participate in the beta test please contact: jan.monsch ät iplosion.com

What makes me quite happy is that they have included InetVis as one of their four chosen visual analysis tools on the live CD.

I’ve been paying with this since this morning and so far so good.

]]> http://lair.moria.org/blog/archives/55/feed 1