wally’s blog

Is it ironic that this Youtube video is refused (in the UK) due to “copyright”?

(It’s the IT Crowd piracy parody)

Disclaimer: I’m certainly no expert in the area of data recovery, and I will not take any responsibility for damage of you, your or anyone else’s hardware/data caused by following these ideas. This entry is nothing more than a diary of what I did for my and other people’s future interest or reference.

Recently I received a Windows XP machine which “wouldn’t start up”. The BIOS would simply output “A error reading disk drive” after the POST. Note this isn’t the same error as “Insert system disk” etc.

I proceeded to check the obvious, trying every boot order and boot option in the BIOS, all without avail. I then booted Knoppix and attempted to mount the data partition (what was C: in Windows), without success.

Important note: I made two stupid mistakes here. Firstly, I tried to initially mount it in Konqueror, and secondly I tried to mount it read/write. If you’re suspicious of a device’s health, mount it readonly and mount it via the terminal. It’ll possibly save your data and save you some wasted time.

I then tried to mount it from a terminal, but I received a variety of errors regarding DMA read failures. I proceeded to disable DMA (K Menu/KNOPPIX/Utilities/Harddisk/CD/DVD DMA Acceleration) and tried again. Still read errors.

At this point I plugged in an external 500GB hard drive (USB) and started an ntfsclone with the command

ntfsclone -o /mnt/sda1/hda1.img -s --rescue /dev/hda1

Whether due to the computer’s USB speed speed or (more likely) the HD’s state, it took about 9 hours to copy 10GB of data off. It unearthed several hundred unreadable sectors in the process.

It’s important to note at this point (if you’re not familiar with the NTFS tools) that the “-s” switch for ntfsclone causes it only copy parts of the filesystem which are actually in use. That means it does not do the same as:

dd if=/dev/hda1 of=/mnt/sda1/hda1.img conv=noerror,sync bs=8k

I’ll probably go down the same route as before, using my Autopsy script to get off what good sectors I can. But the above commands and flags might just make some else’s life a little easier.

It’s been on my mind for some time now … possibly due to my work (where I am interacting with “technical” people who even themselves struggle) - there must surely be a “near perfect” home router design, something which anyone who has used the Internet on their home computer can install and configure.

I currently believe that the “perfect” design must be a simple design, and have a simple interface.  It must rely on doing everything it can, itself.  It must not ask any questions that don’t need answering.  It must only include features that the home user would want - no VPN, no VLANs, no PPoE, and so on.

So in my mind the device looks something like a WRT54GL.  It has 1 “Internet” port, and 4 “other” ports.  It has wireless antenna.  And it has a power LED.

When you plug the device in, it offers DHCP on every interface - that’s WiFi, LAN and WAN.  The user opens “the Internet” and trys to browse, perhaps the documentation suggests going to “router.lan”.  Dependent on which interface they’ve connected through, a webpage will be displayed welcoming them to their “network”.  If they’ve plugged themselves into the WAN port - the page will ask them to connect using a different pot, because this is for the “Internet”.  For those using WiFI - a congratulations your Wireless network is working!

Once they’ve “OK’ed” the WiFi welcome or changed to a LAN port, a Wizard will walk them through setting a wireless network name and setting a passphrase.  Then it’ll try and connect to the Internet, using DHCP on the WAN port (the DHCP server is now disabled on that interface).

The router hosts it’s own DNS server - directing “router.lan” to itself.  Until the Wizard is finished, all DNS requests will point to itself, enforcing the user to finish before getting online.  The Wizard will ask for a “computer name”, allowing the user to setup DNS records for each device on their network.  Every computer’s MAC is saved for DHCP - ensuring each device gets the same IP every time.

The network would be in the 10.0.0.x range, possibly using a /16 mask to allow more than 253 devices to ever be connected (remember we assign an IP permenantly to each device).  This plus lifetime IP assignments should make it easier for non-techies to operate their network … “ten dot zero dot zero dot five” I believe is  a lot easier than 192.168.0.5.  And everything keeps the same IP, so little Jimmy’s netbook is always going to be 10.0.0.8.  With the “computer names” (DNS entries - possibly automagical via NetBIOS for the most part) make connecting between devices even easier still, i.e. wii.lan and netbook.lan etc.  We also minimize the likelyhood of conflicting with our modem device.

Finally there are “additional” options - not included in the wizard - allowing “Advert blocking” and “Website logging” etc.

The router should attempt DHCPing the WAN port - and only in the situation that DHCP fails should it provide the option to enter some details manually.  Home users don’t (and shouldn’t) have to care about their LAN DHCP pool or DMZs.  The router should offer some advice, allowing you to select what “kind” of device various clients are - for example “10.0.0.14 (XBOX 360) connected 12m 34s” - allowing the router to offer suggested port forwards (or just doing it with basic consent?), and potentially performing a portmap scan for “servers” to help the user understand what things they might want to allow (like RDP or HTTP).

Finally some access restrictions might be good, giving the option to stop access late at night, and of course some enforced QoS.

So the router is going to make a lot of assumptions - but all in the name of simplicty.

Kamikazi developers anyone?

There are plenty of HOWTOs on so called NIC teaming or NIC bonding for Debian, but of the ones I’ve seen, they’ve all missed some rather important bits.

Below is the method I personally found to work reliably:

apt-get install ifenslave
echo "bonding" >> /etc/modules
modprobe bonding
ifdown eth0

edit /etc/network/interfaces, changing “eth0″ (or your configured NIC) to “bond0″, and replacing “allow-hotplug” with “auto”.  The following sed line give you this automagically:

sed 's/eth0/bond0/;s/allow-hotplug bond0/auto bond0/' </etc/network/interfaces

Tell ifup and ifdown to add/remove devices to bond0 lines:

echo " up /sbin/ifenslave      bond0 eth0 eth1" >> /etc/network/interfaces
echo " down /sbin/ifenslave -d bond0 eth0 eth1" >> /etc/network/interfaces
ifup bond0;

And you’re done!

You can add all the options for the bonding module to the line in /etc/modules, like:

echo "bonding mode=1 miimon=250 primary=eth0" >> /etc/modules;

Details of all options available at linuxfoundation.org

On that note: I be feeling lucky!

If you’re an Essex University student (i.e. you have a valid Essex login) and you’re tired of having to enter campus for access to the Apple Higher Education store, you’ll probably be pleased to know you can do it from home.

If you’re running Linux/Unix (including OSX), you’ve already got everything you need.  Windows users will need to grab Putty and an X server.

Users of real operating systems will just have to open a terminal and enter:

ssh username@unix4.essex.ac.uk -X
Password: [enter password]
$ firefox

Windows users will still be downloading and installing the needed tools at this point, when they’re eventually ready, they’ll have to run Putty, enter “unix4.essex.ac.uk as the host, enter their username and enable X forwarding. Hopefully, if your X server is running, typing “firefox” at the prompt (after your password) will bring a firefox window to your desktop.

Now this firefox window will be slow, because it’s running on the University’s server. So be patient and use it only for what you need. As it’s running at the University, it’ll have no problem accessing the Apple Higher Education store.

I’m currently getting stuck into “The pleasures of God” by John P. I have to say, having read several of his books, this one has moved me most thus far. I’m sure that when I come to reread previous books again, I’ll be equally shifted spiritually and theologically; but as to where I am right now, God is speaking loud and clear through this one.

Something I’ve come to admire so much of Piper is his almost blinkered, whole centered, obsession with God’s glory, and desire for having a Biblical practicing of it.

I’m becoming more convicted each day of the truth that if we as Christians could find more enjoyment in God, if we could find out satisfaction purely in Him, that is absolutely to say in scrapping TVs, game consoles, unnecessary holidays, loans, gadgets and the rest, we would become more human. We would become more satisfied, more excited, more joyful, more complete, more as we were designed to be.

A superb sermon yesterday evening looking at an introduction to prayer, considering that God promises that if we delight ourselves in Him, He will give us the desires of our heart. That’s powerful, deep stuff. God will always without hesitation satisfy fully, most completely, without restraint, our desire for Him. So if we solely seek satisfaction in God, our good and holy, perfect God, we will be made totally satisfied.

As my fallen nature encourages me to struggle through this life by my own futile strength, seeking satisfaction in money, relationships, jobs, and often even knowledge or experience, I become more and more dissatisfied, and I’m starting, just starting, to understand why that is. I know it at an intellectual level, but not at a real, applied one.

So my prayer today and forever until He returns or calls me home is that me and all my brothers and sisters would find more delight in the Lord each day. And for those who are no yet saved, that God would graciously and mercifully reveal His glory to them, so they may too find real satisfaction for an eternity.

~

Something else, slightly off topic which had never occurred to me, is that humanity is immortal.

Now that needs to be expanded because upon first reading it may sound like we are God. We are not God. All I said was, we are immortal.

We are immortal, because God designed us so. We are both physical and spiritual beings, one and at the same time. As Christians we know that when we die, we will go to be with our Lord forever. But those who are not elect don’t just cease to exist. On the contrary: they exist for eternity too. In turmoil.

So again, I’m reminded to be careful in what I say. Our fallen human bodies are not immortal, but our persons are. That is a powerful statement for evangelism.

Recover data even when an NTFS (or other) won’t mount due to partial hard drive failure.

This was born when someone brought me a near dead hard drive (a serious number of read errors, so bad that nothing could mount or fix the filesystem), asking if I could recover any data.

Now obviously (as almost any geek would know), the answer of course is a very likely yes. There are many ways of recovering data. One such way (which I performed) is using Foremost to find files based on their headers and structures. While this technique works really quite well, it does miss a lot of files, fragment others up, leave bits out and generally not retrieve any metadata (such as filenames etc).

This makes Matt mad. No filenames == days of renaming files.

So I booted up Helix, created a quick image of the drive to a 500GB external drive, and tried running Autopsy (the GUI of Sleuthkit). This is where things got interesting.

I say interesting, because Sleuthkit couldn’t read the filesystem. But it could retrieve the inodes, and the metadata along with them. And it could accordingly retrieve the data content of (some) files.

Observing this, I realized there was a high probability that I could somehow use Sleuthkit’s command line tools to retrieve the files which were not on bad clusters and recover the filenames from the inode. As it turns out, this wasn’t such a bad idea!

There are 3 tools which proved useful:

• ils
• ffind
• icat

ils “lists inode information” from the image, ffind “finds the name of the file or directory using the given inode” and icat “outputs the content of the file based on it’s inode number”. Using these three tools and a bit of bash, we can grab a list of inodes, get the filename from the metadata, create the directory structure beneath it, extract the file content, move on to the next.

So for this task I knocked up the following (really ugly, potentially unsafe) script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

#!/bin/sh
for inode in $(cat /tmp/inodes) ; do
 
/KNOPPIX/usr/local/sleuthkit-2.09/bin/ffind /dev/hda1 $inode
 
if [ $? -eq 0 ]
then
	echo "INODE: $inode"
	INODEDIR=`/KNOPPIX/usr/local/sleuthkit-2.09/bin/ffind /dev/hda1 $inode`
 
	REALDIR=/mnt/out`dirname "$INODEDIR"`
	FILENAME="/mnt/out$INODEDIR"
	mkdir -p "$REALDIR"
 
	echo "FILENAME: $FILENAME"
	/KNOPPIX/usr/local/sleuthkit-2.09/bin/icat /dev/hda1 $inode > "$FILENAME"
 
	if [ `du "$FILENAME" | awk '{print $1}'` == 1 ]
	then
		rm "$FILENAME"
		mkdir -p "$FILENAME"
	fi
	echo ""
fi
done

Really, I do warn you, take serious care running this!

It needs a lot of work, but enough is there for it to function. It reads a file of inode numbers (one per line) and uses ffind to get the filename. We extract the path, attempt to create it, output the file content and (this is important), take a wild guess at if the inode was a directory. Please note this is wildly inaccurate and needs serious rethinking! Currently we look at the file size, and assume directories alone use 1 byte.

We can populate a file with inode numbers like so:

ils -a /dev/hda1 | awk -F '|' '{print $1}' > /tmp/inodes

(Users of Helix will need to use the full pathname to ils as in the above script).

At some point (no garuntees when) I’ll tidy up the script and make it more bullet proof. In the meantime, I hope this saves some data!

Remember: No matter how much data you have, it’s always better to have 2 hard drives of half the size, mirrored than it is to have one large expensive drive. They will die unexpectedly! When you next buy a bigger hard drive, consider this: 1×500GB drive will loose you 500GB of data. 2×250GB will 99.9% probability loose you nothing. So if you’re on a tight budget, buy twice smaller. If you’ve a lot of money, buy twice big.

Oh, and always make regular backups. Cheap USB drives are good for this!

To celebrate my 2:1 degree (praise the Lord), my MacBook decided to aquire a none to small crack on the wrist rest. Naturally Apple have spent the weekend getting new parts and fixing it for free, but one does have to wonder if they’re falling victim to cheap materials?

If you’re responsible for one or more Linux servers, (they provide packages for Linux distributions only, but it may be easily ported to other Unixes) you may be interested in the lesser known tool MondoRescue.

It’s essentially nothing more (but it is a lot!) than a front end to several other GPL’ed tools, including BZip, growisofs and busybox.  To cut a long story short, it builds a set of CDs, DVDs, USB sticks with a tiny bootable Linux distribution based on your server’s kernel, which can restore parts or all of your filesystem from tarballs on the media.  It can also backup to tape drives, NFS mounts or a local filesystem.

It has (imo) a nice, clean ncurses interface; and it’s quick to use.  On an AMD K6 clocked at 350MHz, with 60MB free memory, it took just under 6 hours to compress 8.5GB of data down to a single DVD (4.7GB).  Doing the backup with an “average” compression took far less time (around an hour), but would have eaten up several DVDs.

You’re presented with a wide variety of options when you boot up the produced rescue media.  You can “nuke” the system and start from scratch, or just restore the parts you want.  It can handle new disk geometry, and because the entire rescue system is burnt onto the media, (assuming you used CDs, DVDs or USB storage), it’ll work even if the target machine is clean.