Home

As as I mentioned before I've been recently converting an old project to Java.

This old project was an MDI application, and when creating the UI for the conversion, I found the default JDesktopPane to be rather crappy. Google revealed others thought the same, one of the results that turned up was: http://www.javaworld.com/javaworld/jw-05-2001/jw-0525-mdi.html

So, I created DFDesktopPane based on this code, with some extra changes:

• Frames can't end up with a negative x/y
  
• Respond to resize events of the JViewport parent
  
• Iconified icons move themselves to remain inside the desktop at all times.
  
• Handles maximised frames correctly (desktop doesn't scroll, option to hide/remove titlebar)
  

My modified JDesktopPane can be found as here part of my dflibs google code project.

Other useful things can be found here, take a look and leave any feedback either here or on the project issue tracker

As Noted by Chris recently on IRC, Google Mail lacks a feature in its ability to automatically label/filter messages - you can't do it based on emails from people in a contact group, short of adding a filter with all their email address on it.

At the time it was mentioned this didn't affect me, however later when I got round to adding loads of labels/filters in gmail (yay for, nicely coloured inbox!) to nicely separate things for me I also ran into this problem, so came up with the following python script that does it for me.

It checks messages, sees if the sender is in the contacts, then checks each group to see if there is a label with that group name that is not already set, then checks to see if the contact is in the group, and finally sets the label if everything matches up.

I ran it initially to tag my entire inbox (set "checkAllIndex" to "True" change "ga.getMessagesByFolder(folderName)" to "ga.getMessagesByFolder(folderName, True)") and now have it running on a 15 minute cron (not using loopMode) to tag new messages for me.

Hopefully this will be useful to someone else, I'm not sure how well it works in general, it worked fine for me with ~700 messages at first, however after a few runs (due to regrouping some contacts) I was greeted by an "Account Lockdown: Unusual Activity Detected" message when trying to do anything - This went away after about 20 minutes, but don't say you wern't warned if it happens to you.

#!/usr/bin/env python
"""
 This script will login to gmail, and add labels to messages for contact groups.

 By default the script will only check items from the past 2 days where email
 was recieved.

 Loop mode can be enabled to save logging in repeatedly from cron.
 Loop mode may fail after some time if google kills the session, or gmail
 becomes unavailable or so. (Untested in these situations). On the other hand
 it may also just keep running indefinetly as if no problem occured, loop mode
 is relatively untested and was added as an after thought.

 When running in loop mode, it is best to have a crontab entry also that checks
 and restarts the script if it dies.

 Copyright 2008 Shane 'Dataforce' Mc Cormack

 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
 in the Software without restriction, including without limitation the rights
 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 copies of the Software, and to permit persons to whom the Software is
 furnished to do so, subject to the following conditions:

 The above copyright notice and this permission notice shall be included in
 all copies or substantial portions of the Software.

 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
"""

# Uncomment the lines below if python can't find libgmail on its own, and edit
# the sys,path.insert to point to where libgmail.py is.

# import sys 
# sys.path.insert(0, 'libgmail')
import libgmail
import time

###############################################################################
# Configuration
###############################################################################

# Email Address
email = "YOUR EMAIL HERE"
# Password
password = "YOUR PASS HERE"
# Check all on index, rather than just the first 2 dates found
checkAllIndex = False
# Use Loop (if true the script will keep looping, and sleep between checking
# for new mail to modify)
useLoop = False
# Time in seconds to sleep when looping (300 = 5 mins)
loopTime = 300
# Label Prefix - if group-based labeles are prefixed, set the prefix here.
# (eg "Groups/")
labelPrefix = ""
# What folder to check? ('inbox' or 'all' are probbaly the most common settings)
folderName = 'inbox'

###############################################################################
# Helper classes/methods
###############################################################################

class ContactGroup:
	def __init__(self, id, name, contacts):
		self.id = id
		self.name = name
		self.contacts = contacts
	
	def containsContact(self, contact):
		for knownContact in self.contacts:
			if knownContact[0] == contact.id:
				return True
		return False
	
		def __str__(self):
			return self.name

# Get Contacts and Groups
# Modified from libgmail 0.1.10 to include groups aswell
def getContacts(account):
	"""
	Returns a GmailContactList object
	that has all the contacts in it as
	GmailContacts
	"""
	contactList = []
	groupList = []
	# pnl = a is necessary to get *all* contacts
	myUrl = libgmail._buildURL(view='cl',search='contacts', pnl='a')
	myData = account._parsePage(myUrl)
	# This comes back with a dictionary
	# with entry 'cl'
	addresses = myData['cl']
	
	# Now loop through the addresses and get the contacts
	for entry in addresses:
		if len(entry) >= 6 and entry[0]=='ce':
			newGmailContact = libgmail.GmailContact(entry[1], entry[2], entry[4], entry[5])
			contactList.append(newGmailContact)
	
	contacts = libgmail.GmailContactList(contactList)
	
	# And now, the groups
	for entry in addresses:
		if entry[0]=='cle':
			newGroup = ContactGroup(entry[1], entry[2], entry[5])
			groupList.append(newGroup)
	
	return contacts, groupList

###############################################################################
# Setup
###############################################################################

print "Running.."
print "Use Loop:", useLoop
if useLoop:
	print "  Loop Time:", loopTime
print "Check all on index:", checkAllIndex
print "Label Prefix:", labelPrefix
print "Checking Folder:", folderName
print "libgmail Version:", libgmail.Version
print ""

# Login to gmail
print "Logging in as", email
ga = libgmail.GmailAccount(email, password)
ga.login()

# Loop at least once.
loop = True;

while loop:
	loop = useLoop
	
	print "Getting label names.."
	# Get Labels
	labels = ga.getLabelNames(refresh=True)
	# Get Messages
	print "Getting messages.."
	inbox = ga.getMessagesByFolder(folderName)
	# Get Contacts
	print "Getting contacts and groups"
	contacts, groups = getContacts(ga)
	
	# Check each thread in the inbox
	lastDate = '';
	secondDate = False;
	for thread in inbox:
		# Only check dates we are supposed to.
		if not checkAllIndex:
			# Get the date
			threadDate = thread.__getattr__('date');
			# Make sure a date is set
			if lastDate == '':
				lastDate = threadDate
				
			# If this date is different to the last one do something.
			if lastDate != threadDate:
				# If we are already on the second date, then we stop now
				if secondDate:
					break;
				# Otherwise, if the new data is a non-time date, we can change to the
				# second date.
				elif "am" not in threadDate and "pm" not in threadDate:
					lastDate = threadDate
					secondDate = True
		
		print "Thread:", thread.id, len(thread), thread.subject, thread.getLabels(), thread.__getattr__('date'), thread._authors, thread.__getattr__('unread')
		try: 
			# Current Labels
			threadCurrentLabels = thread.getLabels();	
			# We will add labels here first to prevent dupes
			threadLabels = set([])
			# Check each message in the thread.
			for msg in thread:
				print "  Message:", msg.id, msg.sender
				# Check if sender is a known  contact
				contact = contacts.getContactByEmail(msg.sender)
				if contact != False:
					# Check each group for this contact
					for group in groups:
						# If we have a label with this group name
						labelName = labelPrefix+group.name
						if (labelName in labels) and (labelName not in threadCurrentLabels):
							# And the group contains the contact we want
							if group.containsContact(contact):
								# Add it to the list
								print "    Sender Label:", labelName
								threadLabels.add(labelName)
		except Exception, detail:
			print "  Error parsing messages:", type(detail), detail
			
		# Now add the labels
		for label in threadLabels:
			print "  Adding Label:", label
			thread.addLabel(label)
		# If thread was unread, make it unread again.
		if thread.__getattr__('unread'):
			print "  Remarking as unread"
			ga._doThreadAction("ur", thread)
	
	if loop:
		print ""
		print "Sleeping"
		time.sleep(loopTime)
	else:
		print "Done"

On a related note, I've also recently started to use the "Better Gmail 2" addon for firefox (Official page seems down atm, but more info here) mostly for the grouping of labels feature.

Edit: Script will now preserve unread status of threads.

I was recently looking at converting an old application from VB6 to Java that used MD5 in its output files as hashes for validation.

The first thing I did was to make a java class that read in the file and checked the hashes, I tried it on a few files and it worked fine, then I found a file that it failed on.

Now, this app wrote all the files using the exact same function, so it seemed odd that 1 of them wouldn't parse and the rest would.
When I looked at the file closer, I found that this one contained some symbols in the output that the others didn't - I eventually figured out that the symbol that was causing the problem was the pound sign (£).

Without going into too much detail, this presented a major problem, the string in question was used as part of the password validation for the app (the output files are encrypted using the password as a key), and the java code was getting different results than the old VB6 code, and was unable to decode the file as a result.

So, this sparked my curiosity a bit, the VB6 code I was using wasn't a built in, it was code I'd gotten elsewhere and used, so I assumed it was faulty code (not that this helped me much, as I needed to get the exact same output, but ignoring that).

I edited the initial form of my application to return the MD5 String for "£" on its own, and got: "d527ca074d412d9d0ffc844872c4603c"
I did the same for my java code and got: "6465dad1d31752be3f3283e8f70feef7"

So now all I needed to do was to see which was right, so I made a quick PHP script, and did the same and got: "d99731d14c7750048538404febb0e357" ... Yet another different hash!?

Ok, I thought, md5sum will help me figure out which one is right. one `echo '£' | md5sum -` and I had "67160ce935d7cb5339047b12ad4611cb". Yes, that is correct, a 4th different hash.

So here I was with 4 different hashes and no idea which one was correct.

So after a bit of googling, I discovered that the MD5 RFC (1321) had the source code for a test application in it.
So I extracted the code from the Appendix of http://www.ietf.org/rfc/rfc1321.txt and tried to compile it with `gcc md5.c mddriver.c -o mddriver` only to discover that it failed to compile with lots of errors, fortunately this was an easy fix, near the top of mddriver.c, change "#define MD MD5" to "#define MD 5" and then it compiles without problem.

So, I ran "./mddriver -s£" and got the output "MD5 ("£") = d99731d14c7750048538404febb0e357" which agreed with what the PHP md5() function gave.
(Its worth noting that `echo '£' | ./mddriver` agreed with md5sum, which made me remember that `echo` appends a "\n", which was why I got a different output, running `echo -n '£' | md5sum` gives the correct result, and would have saved me googling and finding the test suite!)

I tested a few other things and got the following results:

        mddriver: d99731d14c7750048538404febb0e357
             PHP: d99731d14c7750048538404febb0e357
           mySQL: d99731d14c7750048538404febb0e357
          python: d99731d14c7750048538404febb0e357
      postgreSQL: d99731d14c7750048538404febb0e357
          md5sum: d99731d14c7750048538404febb0e357

      JavaScript: d527ca074d412d9d0ffc844872c4603c
     VisualBasic: d527ca074d412d9d0ffc844872c4603c
         Eggdrop: d527ca074d412d9d0ffc844872c4603c
   Java (custom): d527ca074d412d9d0ffc844872c4603c

 Java (built in): 6465dad1d31752be3f3283e8f70feef7

• JavaScript implementation from http://pajhome.org.uk/crypt/md5/
  
• VisualBasic implementation from http://www.frez.co.uk/freecode.htm#md5;
  
• Custom Java implementation from http://www.freevbcode.com/ShowCode.Asp?ID=741
  

There is also a list of MD5 implementations at http://userpages.umbc.edu/~mabzug1/cs/md5/md5.html

----
The differences are primarily due to character encoding in the different languages. (In the case of my app, there was also a flaw in the implementation for strings where (length % 64) is > than 55 as well)

Example:

[07:14:55] [shane@Xion:~]$ php -r 'echo md5(utf8_encode("£"))."\n";'
2ccf59396b3c0958eec4ba721e2d083f
[07:15:01] [shane@Xion:~]$ php -r 'echo md5("£")."\n";'
d99731d14c7750048538404febb0e357

Java: System.out.println((int)'£'); => "163"
PHP: echo ord('£'); => "194"
PHP: echo ord(utf8_encode('£')); => "195"

I fixed the rss feed for the site today, its been broken for a while now (it was using a different user name to connect to the SQL DB than the rest of the site, and I changed access permissions on a load of things a while ago).

Despite all the "This was amazing" "fantastic" reviews I seem to find for this everywhere, I found myself dissapointed after watching it.

The first 2 acts were funny and very rewatchable, there was gold bars that became gold liquid, the "Bad Horse" letters and phone calls, Captain Hammer and his "Hammer", they made me laugh and alongside the humor was a kick ass sound track (my personal favourite being "Its a brand new day").

Act 3 on the other hand was a complete change in direction, the songs weren't as good, it wasn't really all that funny (Infact I think the onyl bit I laughed at was him stopping his song to correct the spelling of his name), it suddently became all serious. All in all I found it a rather dissapointing, and somewhat obvious, end and a let down to an otherwise awesome show.

Despite this I'm still going to buy the DVD (and hopefully the OST if one comes out), as I approve of the idea of a web-streamed show (I think there was a push to get a firefly season 2 done in this way at one point) and would like to see more of them.

For a while now (pretty much since I've been using linux) I've been using yakuake, and I've been looking for something similar for OS X (at the moment I tend to ssh from my desktop to my OS X machine to do anything console like).

I found visor which wraps terminal.app, but overall this is a poor replacement primarily for the lack of tab support (I use tabs alot in yakuake, at the moment on my desktop I have ~15 open)

I tried to get yakuake working on OS X a while ago and failed (it wouldn't compile) so gave up, recently however I decided to try again and have made better progress.

This is what I did, its probably not the best way of doing it (for example, the initial 3GB download could probably be reduced).

First off, download the "everything" package from http://mac.kde.org/?id=download (its a torrent)

Open a terminal, and cd to the directory you downloaded kde-mac to and run:

chmod a+x *.pkg/Contents/Resources/postflight

Then install the kde.mpkg package contained (this installs everything nicely)

Now, back in the terminal:

sudo /opt/kde4-deps/bin/update-kde-mac.sh
/opt/kde4/bin/kbuildsycoca4
cd ~
svn co svn://anonsvn.kde.org/home/kde/trunk/extragear/utils/yakuake/
PATH=/opt/qt4/bin/:${PATH} cmake -DCMAKE_INSTALL_PREFIX=/opt/kde4/
make
sudo make install
sudo /opt/kde4-deps/bin/update-kde-mac.sh
/opt/kde4/bin/kbuildsycoca4

This installs yakuake to /opt/kde4/bin/yakuake.app next to the other kde apps.

If you run it however, the first time it runs it will give you the "choose a key" popup, but after that its not yet possible to do anything, pressing the key shows it attempting to appear but not quite getting there

Almost

Back to visor I guess

As I mentioned in my last post, One of the useful advantages of the network boot setup is that I can use it to quickly install virtual machines.

Now a few things:

• My Desktop is a lot more powerful than my server, so I run the virtual machines on it.
  
• I use virtualbox rather than vmware.
  
• All the network boot stuff is on my server not my desktop (obviously)
  

So in order to allow this, virtualbox needed to be setup to bridge to my existing adapter, this was quite straight forward, pretty much exactly as the manual said.

	sudo apt-get install bridge-utils

Edit /etc/network/interfaces, and add

auto br0
iface br0 inet dhcp
	bridge_ports eth0

Now the next suggestion was to setup a tkap0 device and tell virtualbox to use that, or to use a dynamic configuration.
The dynamic configuration sounded better as it meant I didn't need to remember to add a new tap device for each vm.

The suggested dynamic configuration suggests using kdesu/gksudo and a script in the home dir of the user that will setup and cleaup the tap device (this means inputting your password every tiem you start/stop the VM and requiring a separate script for each user that wants to have a vm with bridging) this seemed rather annoying so I came up with an alternative.

/usr/bin/setuptap

#!/bin/bash

# Make sure we are root
if [ $(whoami) != root ]; then
        exit 1;
fi;

# Create an new TAP interface for the user and remember its name.
interface=`VBoxTunctl -b -u ${SUDO_USER}`
# If for some reason the interface could not be created, return 1 to
# tell this to VirtualBox.
if [ -z "$interface" ]; then
        exit 1
fi
# Write the name of the interface to the standard output.
echo ${interface}

# Bring up the interface.
/sbin/ifconfig ${interface} up
# And add it to the bridge.
/usr/sbin/brctl addif br0 ${interface}

/usr/bin/cleanuptap

#!/bin/bash

# Make sure we are root
if [ $(whoami) != root ]; then
        exit 1;
fi;

# Remove the interface from the bridge.  The second script parameter is
# the interface name.
/usr/sbin/brctl delif br0 $2
# And use VBoxTunctl to remove the interface.
VBoxTunctl -d $2

Now these scripts run with sudo as any user will setup the tap device for that user (thats what ${SUDO_USER} is for)

This still requires a password for starting/stopping the VMs tho, so we use

sudo visudo

or if you prefer nano

sudo EDITOR=nano visudo

and add

# Allow virtualbox users to setup/cleanup tap devices
%vboxusers        ALL=NOPASSWD:/usr/bin/setuptap,/usr/bin/cleanuptap

now:

• configure virtualbox to attach the network device to a "host interface"
  
• leave the Interface name blank (setuptap creates the next available one)
  
• Setup Application: "sudo /usr/bin/setuptap"
  
• Terminate Application: "sudo /usr/bin/cleanuptap"
  

And virtualbox will be able to create/destroy the tap device as needed.

However. there is still one problem, DHCP will not work for these VMs without a little help, so we need to:

sudo apt-get install dhcp3-relay

and answer the questions asked. (DHCP Server IP, and INterface to listen on (br0))

Virtualbox unfortunatly seems to need a little push to actually network boot, so I also use an etherboot iso to actually boot from the network along with the "PCnet-FAST III" adapter type.

and thats all there is to it, you can now network boot and dhcp from virtual machines not hosted on the server.

Another post in under 3 months?

So as you may or may not know from time to time I have the joy of fixing computers for various people. Alot of these fixes result in a reinstall of windows and away.

This is a rather easy enough job, I have a KVM switch that I attach to the machine, pop a windows CD in (I used to have an unattended CD but don't anymore), answer a few questions and then occasionally switch the KVM over to see if the install died or so.

Now this is all well and good except for 2 problems:

1. It means I need to keep (or remember to bring) windows CDs at home (where I do most of my jobs)
   
2. I recently had a machine to fix that had a non-working CD Drive
   

Now, the first one isn't so much of a problem, but the second one was.

So for some reason known only to him, my dad a while ago decided to invest in an External CD Writer rather than an internal one, so I do have a USB cd drive.

First port of call was to attach the CD Drive, pop in the CD, reboot the machine, tell it to boot from usb... oh, it doesn't recognise the drive. bugger.

So I googled a bit, There was lots of suggestions mostly to use a floppy disk with the USB drivers to bootstrap the install (no thanks, I doubt I 6 (yes, SIX!) working floppies required to bootstrap the windows installer).

Then I remembered ages ago when I was making my unattended CD, I discovered an app called (shockingly) "unattended" (link) so I updated the copy of unattended I had on my server and went to investigate how to use it

The main suggested methods:

1. Burn a CD
   
2. Create a boot floopy
   

Neither of these were appealing (Floppies suck, I probably don't have a spare floppy anywhere that works) and the reason I was even looking at this was because the machine had no CD Drive.

However there was an alternative, network booting. Quickly check the back of the laptop, bingo! a network port!

So, I quickly (I say quickly, but my server was still Redhat 9 at the time, so rather slowly and painfully) I installed the tftp server (apt-get install tftp-hda on ubuntu), configured xinet.d (see below) and my dhcp server (see below).

xinet.d/tftp:

# default: off
# description: The tftp server serves files using the trivial file transfer \
#       protocol.  The tftp protocol is often used to boot diskless \
#       workstations, download configuration files to network-aware printers, \
#       and to start the installation process for some operating systems.
service tftp
{
        socket_type             = dgram
        port                    = 69
        protocol                = udp
        wait                    = yes
        user                    = root
        server                  = /usr/sbin/in.tftpd
        server_args             = -s /tftpboot
        disable                 = no
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

dhcpd.conf:

# Not sure if this is needed, I added it anyway
allow bootp;
# My Servers IP
next-server 192.168.0.5;
# PXE Boot
filename "pxelinux.0";

pxelinux.0 and its config directory can be found in bootdisk/tftpboot in the unattended distribution.

I also configured my internal DNS server as required by unattended to provide the ntinstall host.

This allowed me to boot up the machine using the network and install windows as normal (There are a few issues with this, namely that the windows xp installer sucks and requires a fat32 partition for swap space, so you can't use unattended to upgrade an existing ntfs install, it has to format the drive as fat32, install, convert it to ntfs, and defrag it)

This made me quite pleased, I copied my windows disks into the install/os directory, and my office disk into the appropriate directory (see the unattended site for all related configuration etc) and left it be.

A few days later I after I restarded one of my machines, it managed to network boot itself into the unattended menu rather than the hard disk, I quickly googled to find out how to make it boot its main hard drive, it gets IP 192.168.0.10, so I created /tftpboot/pxelinux.cfg/C0A8000A with the contents:

default local
label local
localboot 0

This then prompted me to look at the pxelinux config a bit more, Wouldn't it be awesome to be able to install ubuntu OR windows using network boot? Yes, it would. I also threw in network boot support for DBAN aswell.
my /tftpboot/pxelinux.cfg/default now looks something like this:

DEFAULT menu.c32
PROMPT 0

MENU TITLE Network Boot Options

LABEL disk
        MENU LABEL ^Local Disk Boot
        MENU DEFAULT
        LOCALBOOT 0

LABEL unattended
        MENU LABEL ^Unattended Windows Install
        KERNEL /unattended/bzImage
        APPEND initrd=unattended/initrd

LABEL autonuke
        MENU LABEL DBAN ^Autonuke
        KERNEL /dban/kernel.bzi
        APPEND initrd=dban/initrd.gz root=/dev/ram0 init=/rc nuke="dwipe --autonuke" silent

LABEL dban
        MENU LABEL ^DBAN normal
        KERNEL /dban/kernel.bzi
        APPEND initrd=dban/initrd.gz root=/dev/ram0 init=/rc nuke="dwipe" silent

MENU SEPARATOR

LABEL -32
        MENU LABEL Ubuntu i386:
        MENU DISABLE

LABEL 32install
        MENU LABEL Ubuntu i386 Install
        MENU INDENT 1
        KERNEL ubuntu-installer/i386/linux
        APPEND vga=normal initrd=ubuntu-installer/i386/initrd.gz --

LABEL 32cli
        MENU LABEL Ubuntu i386 CLI
        MENU INDENT 1
        KERNEL ubuntu-installer/i386/linux
        APPEND tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false vga=normal initrd=ubuntu-installer/i386/initrd.gz --

LABEL 32expert
        MENU LABEL Ubuntu i386 Expert
        MENU INDENT 1
        KERNEL ubuntu-installer/i386/linux
        APPEND priority=low vga=normal initrd=ubuntu-installer/i386/initrd.gz --

LABEL 32cli-expert
        MENU LABEL Ubuntu i386 Expert CLI
        MENU INDENT 1
        KERNEL ubuntu-installer/i386/linux
        APPEND tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false priority=low vga=normal initrd=ubuntu-installer/i386/initrd.gz --

LABEL 32rescue
        MENU LABEL Ubuntu i386 Rescue
        MENU INDENT 1
        KERNEL ubuntu-installer/i386/linux
        APPEND vga=normal initrd=ubuntu-installer/i386/initrd.gz rescue/enable=true --

MENU SEPARATOR

LABEL -64
        MENU LABEL Ubuntu x68_64:
        MENU DISABLE

LABEL 64install
        MENU LABEL Ubuntu x86_64 Install
        MENU INDENT 1
        KERNEL ubuntu-installer/amd64/linux
        APPEND vga=normal initrd=ubuntu-installer/amd64/initrd.gz --

LABEL 64cli
        MENU LABEL Ubuntu x86_64 CLI
        MENU INDENT 1
        KERNEL ubuntu-installer/amd64/linux
        APPEND tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false vga=normal initrd=ubuntu-installer/amd64/initrd.gz --

LABEL 64expert
        MENU LABEL Ubuntu x86_64 Expert
        MENU INDENT 1
        KERNEL ubuntu-installer/amd64/linux
        APPEND priority=low vga=normal initrd=ubuntu-installer/amd64/initrd.gz --

LABEL 64cli-expert
        MENU LABEL Ubuntu x86_64 Expert CLI
        MENU INDENT 1
        KERNEL ubuntu-installer/amd64/linux
        APPEND tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false priority=low vga=normal initrd=ubuntu-installer/amd64/initrd.gz --

LABEL 64rescue
        MENU LABEL Ubuntu x86_64 Rescue
        MENU INDENT 1
        KERNEL ubuntu-installer/amd64/linux
        APPEND vga=normal initrd=ubuntu-installer/amd64/initrd.gz rescue/enable=true --

I can now boot the local hdd (default, incase I don't want any of the network boot options), securely wipe drives, install windows (via unattended), or use any of the features from the ubuntu disks (both 64bit and 32bit).

I would like to add some other options at a later date such as BSD/Solaris Installers or knoppix network boot as the main use for this are for fixing PCs for people (hense windows and knoppix) and a side benefit of making installing OSs in VMs easier (VM network boots to the boot menu for me to install from)

If anyone wants to know more about the setup or has any questions, just use the comment form.

So, it was brought to my attention that I havn't actually updated this in a while, so here you go, an update! (However dissapointing you might find it!)

Ok, so the most important update of recent is that I finally (after 4 years of being out of date) upgraded the OS on my server from Redhat 9 (EOL April 2004!) to the ubuntu server 8.04 ("Hardy Heron").

This also as a result of to a change in iptables between the ancient 2.4.30 Kernel and the current 2.6.24 prompted me to rewrite my IPTables firewall scripts (As the old one broke stuff, like my multiple IPs - which in turn managed to break logging into Authgate in a fun way!).
For anyone who is interested, they can be downloaded here.
Once downloaded, extract them to a directory (I use /root and thus they are currently configured as such) and edit the "nat" file to change the main settings.
You can then run it by using:

/bin/bash /root/nat

(Change to suit where you extracted to)

I have also included shape.php, which allows you to setup traffic shaping for People >:E
It needs configuring separately to the main script, and is called by nat-shape.sh (which is called by the main script).
Once you have configured shape.php you will need to edit nat-shape.sh to make it actually call it (remove the first echo, and remove the # from the start of the other 2 lines)

The next thing I've done recently is to tidy up the DMG generation script for DMDirc.
Now it manages to create compressed images on linux aswell as on OS X, and doesn't require stupidly ugly code or a patched version of Apples DiskDev tools!

And finally I'm now back home for the next 4 months rather than at uni, yay for no more stupid download cap! (Seriously, 20GB for a whole month kills me!)

Recently I finally finished adding support for AuthGate to be an OpenID Provider.

This prompted Chris to develop Poidsy, which I have also implemented into AuthGate. This allows people to login to AuthGate (and thus also here, and other AuthGate powered sites) using OpenID instead of having to register with AuthGate itself.

I've made a few small changes to comment posting on here.
Firstly, if you are logged in, you will not have to enter the captcha that non-logged users have to enter.
Secondly, the original poster of the article (usually me ) will have a * next to their name in the comments.
Thirdly. Posts posted by logged in users with the appropriate user level (ie people who have posting access, aka me) have their username in blue to make them stand out.
Finally comments by non logged in users have (Guest) next to them

These changes also affect all previous comments posted

On a daily basis, I get around 800 emails to my email accounts, of which most of it is spam.

Now as good as thunderbird is at detecting spam, even it fails at a lot of the spam I recieve, leaving me with arround 100-200 spam per day that gets into my inbox.

I've dealt with and accepted this for over a year now, before a discussion on IRC made me decide to do something about it. (When I say discussion, I mean Chris pasted one line showing how good the UTD-Hosting mail server was at preventing junk getting to him)

So, I recently (Today and yesterday) started prodding my postfix config to help with the problem.

Firstly I added some RBL checking, this was easy enough, 3 lines to my config in the smtpd_recipient_restrictions bit:

    reject_rbl_client list.dsbl.org
    reject_rbl_client zen.spamhaus.org
    reject_rbl_client dnsbl.sorbs.net

I also added:

    reject_non_fqdn_recipient
    reject_unknown_recipient_domain

I also added the following lines:

smtpd_helo_required = yes

smtpd_delay_reject = yes

smtpd_helo_restrictions =
    permit_mynetworks
    check_helo_access hash:/etc/postfix/helo_access
    reject_non_fqdn_hostname
    reject_invalid_hostname
    permit

smtpd_sender_restrictions =
    permit_mynetworks
    reject_non_fqdn_sender
    reject_unknown_sender_domain
    permit

/etc/postfix/helo_access looks like this (Its surprising how many mails this catches, 114/7500 - altho they would probably be caught later on):

soren.co.uk            REJECT You are not me.
207.150.170.50         REJECT You are not me.

Next step was SPF checking, this involved adding to smtpd_recipient_restrictions:

    check_policy_service unix:private/policy

and to master.cf

policy  unix  -       n       n       -       -       spawn
        user=nobody argv=/usr/bin/perl /usr/lib/postfix/policyd-spf-perl

(One can apt-get install postfix-policyd-spf-perl or download it from http://www.openspf.org/Software)

Currently I use catch-all on all my domains (yes this is stupid I know) and as a result, I get alot of spam to 1) Addresses that don't exist and never have 2) Addresses that used to exist for others but now don't.
To combat this, I added this line to smtpd_recipient_restrictions:

    check_recipient_access hash:/etc/postfix/recipient_access

/etc/postfix/recipient_access looks something like this:

foo@example.com REJECT This account is no longer valid.
bar@example.com REJECT This account is no longer valid.
baz@example.net REJECT This account is no longer valid.

The result of all this can be seen by running the mailstats script Chris was kind enough to share with me:

root@soren:/etc/postfix# ./mailstats.php

Incoming --(7500)--> Valid HELO --(6707)--> Valid Sender --(6705)--> Passed by dsbl --(6136)--> Passed by spamhaus --(811)--> Passed by sorbs --(568)--> Passed by relay check --(565)--> Passed by SPF --(542)--> Forwarded to shinobu --(390)--> To a valid domain --(339)--> To a valid user --(306)--> Dropped Spam --(306)--> Delivered.
Total Rejections: 7194 (Unknown Reason: 0 | Pretended to be me: 114)

The "Forwarded to shinobu" entry is a server for which I am the backup MX for, this accounts for 152 mails (about 2%)
The delivered count of 306 mails is about 4%, meaning that 94% of all the junk mail I recieve is now dropped by postfix and not delivered to my mailbox!

These simple additions have made a huge difference! I have a 10day holiday coming up, and now rather than coming home to 8000 mails, I'll only come home to 320!

As a further line of defense, prior to being sent to my mailbox, those 4% of mails get filtered thorugh spamassassin (which I have configured to only run for certain domains, with different scores for different domains/users as needed) which does a good job of catching the spam that thunderbird misses, configuring a mail filter on thunderbird to filter these mails (Which get subject tagged with {Spam?}) into my junk folder (aswell as configuring thunderbird to trust what the spamassassin headers say) means very little, if any, spam now reaches my inbox!

Brilliant!

This post is about 10 days late, but meh.

I passed my first year of uni

Roll on year 2!

DMDirc 0.4 has now been released

I (Along with Chris 'MD87' Smith and Greg 'Greboid' Holmes have recently started working on DMDirc (again, only this time its in Java to help towards the original aim of the project to be a decent, truly cross platform IRC Client)

Its coming along quite well

You can track the progress, (and download the current release which is 0.1) at the Google Code Project or the Project Website

Lemme know what you think.

I recently obtained a "la fonera" from fon.com

Its a neat little device that is small and portable. With firmware based on OpenWRT (That means it runs linux under the hood!) it was bound to be hackable and customiseable.

I've put together a page detailing different tricks you can do with your fonera. You can find it here.

I don't want to go into the ethics of what this page describes. Yes you shouldn't abuse it like that, its a good idea - its just not practical. WIFI Devices have a short range, and thus you have to be very close to the device to be able to use it. At least this way its more useful to _you_.

With the release of windows vista, comes the start-of-the-end for Windows XP. with its EOL (End-Of-Life) date now set at January 30th 2008 (thats less than a year away), people (by people I mean windows users) who are unable to upgrade to vista (due to Lack of computing power or so) or don't want it (its crap, proprietary, riddled with DRM and probably bugs - Microsoft are already producing SP1!) need to start looking for alternatives, unless they want to stay using an unsupported (this means no more bug/security fixes) Operating System.

Imo, The best alternative is some derivative of Linux. (Although there is others such as MacOS x86 although its not supported on non-mac hardware, FreeBSD but I don't think its desktop oriented, and others such as beOS or so)

As of Saturday 27/1/07 I have started using KUbuntu Linux as the main OS on my desktop, as a trial to see how well I can get by without my "trusty" windows installation.

I chose KUbuntu due to its use of KDE, and the fact it was ubuntu, which is one of the more well-established desktop-friendly distros of Linux available at this time. It also has one of the best communities and followings, as well as a solid base (Debian).
(A distro (Distrobution) is the term used to refer to a specific version/flavor of Linux. Other distros include Fedora Core, Redhat, Slackware, Debian and Gentoo)

The installation went smoothly, it resized my NTFS partition to allow for an ext3 partition, and installed away happily, I was even able to irc/browse the web whilst it did it due to the live cd based installer (Which is just pure genius!)

After the install, I rebooted (twice into windows first to allow the NTFS Journal to be reset for the new size, and then to confirm it was "clean", then into Linux) and was greeted with a nice graphical login screen, that had detected the correct resolution for my monitor and everything.

The next task was to update the system, and install the nvidia drivers so that i could use dual monitors. This was a relatively painless process with kubuntu. I edited the "sources.list" file (alt+f2 then "kdesu kate /etc/apt/sources.list") and uncommented the disabled repositories and opened a konsole window (alt+f2 then type "konsole") and entered the following commands "sudo apt-get update" "sudo apt-get upgrade" "sudo apt-get install nvidia-glx". Alas after rebooting, X didn't want to work and I was greeted with a text-only console. Fortunatly the fix was easy, adding "deb http://albertomilone.com/drivers/edgy/latest/32bit/ binary/" to the sources.list and running the above commands again and rebooting fixed the problem.

The next task was to enable ntfs write. Following http://ubuntuforums.org/showthread.php?t=217009 I was able to simply add some repositories to my sources.list, "sudo apt-get install ntfs-config" and then run ntfs-config and follow the prompts. Restarted and it was done.

I now had a fully usable desktop, running both monitors at their native resolution, and i was even able to easily setup the task bar to show only tasks on the monitor it is on.

Since installing I've installed quite a few packages (apt-get makes it really easy) and done alot of customisation. Suffice to say i'm pretty happy with my install, and don't think I'll be returning to windows any time soon.

One package I would recommending installing would be "beryl" (Google for "Beryl Ubuntu" for installation guide, and check youtube for videos of what it can do as well as the standard window decorating).

The great thing about Ubuntu is the community behind it at ubuntuforums.org. pretty much any problem you have, has been reported there by someone else, and subsequently solved - this makes problem solving a snap! (If a problem isn't there, searching Google for "Ubuntu <problem>" usually solves most things.) Another benefit is the fact it derives from debian which is a well established distribution in the Linux community, and subsequently has a lot of packages available for non-source applications should you not want to be compiling from source all the time to add things.

A tip to KUbuntu users wishing to follow some of the guides on the forums that were designed for the standard gnome-based version of Ubuntu, open a konsole and run the following 2 commands: "sudo ln -s /usr/bin/kate /usr/bin/gedit" and "sudo apt-get install gksudo synaptic". These 2 commands install "gksudo" which some of the guides use to gain root, synaptic which is the most referenced package manager, and makes the editor "kate" work when the guides say gedit (as gedit just looks ugly on KDE. An alternative would be to run "sudo apt-get install gksudo synaptic gedit" which will give you the real gedit in all its ugliness.)

Overall, I'm now a happy linux-desktop user, and I've yet to reboot back to windows!

You can get a copy of KUbuntu here: http://www.kubuntu.org/download.php

Edit: www.ubuntuguide.org provides a page full of instructions on how to install commonly-wanted stuff.

Due to a recent need in a project of mine for mySQL access from delphi/freepascal I have adapated the version of mysql.pas from http://www.fichtner.net/delphi/mysql.delphi.phtml to load both libmysql.dll (on svn) and libmysqlclient.so (Usually located in /usr/lib/mysql/).

I also created a wrapper class for it (TSQL in SQL.pas)

Downloads can be found here: http://home.dataforce.org.uk/viewcvs/misc/MySQL/

Any queries/questions should be left in the comments.

(This has been tested, and compiled on Freepascal on linux (1.9.8) and windows (2.0.0) and in Delphi (6/7/Turbo) on windows.)

Whats this? 3 news posts in 1 day? I must be ill!

Anyways, on to the point. I have recently (Yesterday) decided to start experimenting with AJAX.
As I test things, I will put them up on the page at http://ajax.home.dataforce.org.uk.

Currently I have 1 test script that turns an int into a time as you type it, however the AuthGate login page now uses AJAX to login to prevent the need to send the users password in plain text.

It is now possible to choose a style when viewing this site.

Simply choose "Page Style" form the menu, and choose one of the styles there.

The style is stored in a cookie, and if you are logged in, it is also stored in your user profile so that the cookie is restored when you login.

Feel free to contact me if you wish to submit any styles.