Random Musings

The future of networking

2006-09-07T23:26:00.000-07:00

Back in January, I read a blog post about Van Jacobson's talk at Linux Conf.au 2006. (LWN's coverage and the slides) This is some amazing stuff. I can't really do these justice - go read. The slides show some amazing numbers in terms of potential speedups through rearchitecting.

What prompted this entry, however, has nothing to do with that. Instead, it has to do with another Van Jacobson talk at Google that deserves attention - namely A New Way to Look at Networking. Where the other paper and talk were about performance, this talk is more about thinking about what's next - what will the ubiquitous network of 20 years from now look like, and how to make it work. (I'll spoil a slide near the end, as a teaser - PGP meets BitTorrent and hitches a ride on an airplane. (Not Pacific Air 121, however.))

Ye Olde Scrum

2006-09-07T21:44:00.000-07:00

At work this week there was a very good talk about "Scrum Et Al", given by Ken Schwaber.

Executive summary: Go watch this, if you're at all interested in Agile teams and iterative development, it's definitely worth your time.

What struck me most while watching this, was how much I wished I had gotten around to reading something about it years ago. (I still haven't, but this video is enough to get me interested now.) At my last job, we were doing something informal, but not too far off of this. Quarterly releases, along with a relentless pace of new features, improving interfaces, improving security, improving developer productivity, etc. Seeing this talk reaffirms my belief that the only good way to build software products is iteratively. Being able to say, "I only need to worry about X right now." and sitting down and doing X, getting it done, clean, bugfixed, and then being able to say, "Ok, what's next?", all the while having a product that can be released at basically any time, maybe multiple times, and a simply relentless pace of improvements? This is great! Unfortunately, I don't think very many business majors have figured this out yet.

Git presentations

2006-08-19T08:48:00.000-07:00

At Penguicon in April, I gave a basic Git tutorial, and this week at Ubucon I gave a small variation of that talk, where I tried to cover why distributed SCMs are so much better for free software work.

These presentations are both up at http://h4x0r5.com/~ryan/presentations/

Also, Junio Hamano (Git Maintainer), gave a great talk at OLS: http://members.cox.net/junkio/200607-ols.pdf

Hardware update (belated)

2006-07-03T15:17:00.000-07:00

So, eventually, I did figure out what was wrong with my hardware.

It appears to have been nothing more than a horribly overheating processor, and thus, fixed by a simple application of some thermal paste. (65C = bad, generally)

On that note, I now have to decide what kind of upgrading I want to do, and whether or not I build a MythTV box (probably), but at least I can sit on it for a bit until prices drop as rumored in July.

Passwords, argh

2006-06-02T06:04:00.000-07:00

With all the things I need to do to get organized around a new job, one of the things I've ended up doing a lot of is creating new accounts. With new passwords, of course.

What really irks me is that there is *no* consistency in the password requirements, so, even if I wanted to reuse a password for a few things, it would be darn near impossible.

{0,2}capital letters, {0,2}numbers {0,2}lowercase letters, {4-6}minimum characters, {4-8}maximum characters.
The situation has gotten, well, frankly ridiculous. Maybe Passport (or something similar) wasn't as horrible an idea as it first appeared.

One dead (sorta) computer

2006-05-24T21:59:00.000-07:00

Now that I'm moved, and I'm starting to get settled in, finally, I setup some computers tonight.

Well, my gaming/desktop computer worked fine (actually, haven't booted Linux on it, yet, but Windows is fine.)

My old webhost/server box is up fine, as well.

Unfortunately, my primary NFS/file server won't boot.

I get a fun variety of errors:

Sometimes, I get a recursive fault and the kernel locks (2.6.16, though, I'm not sure it matters)

Sometimes, I get a "failed to mount /dev/md4, please specify root fs with root=" (etc/paraphrased)

Sometimes, I get a crc error during "Uncompressing Linux"

Sometimes, I get an "Invalid format" during "Uncompressing Linux"

Sometimes, I get partway into boot and a beautiful screen full or random colors and letters flashes, and the machine reboots.

Given how inconsistent the failures are, my gut says "motherboard, ram or CPU", and not harddrive. (I did really expect at least one harddrive to fail, though.)

The case for this machine did get a nice dent added to it during the move (on the motherboard side, not the other side).

I think I need to make a DSL boot USB key, and see if that fixes it, just to confirm, otherwise, debugging this is going to be a pain. Damn fileserver shouldn't be the broken machine. :(

News from a new state

2006-05-14T14:09:00.000-07:00

So, as a few of the readers of this blog know, I'm moving to California (Bay Area/Silicon Valley)
Well, technically, I suppose, I've moved. My stuff hasn't quite arrived yet, but within the next day or two it should all arrive. It will be nice to get my normal computers back and be able to work on a machine that isn't a laptop, while at home. I suppose I'll have to finally redefine home to be my new (expensive) apartment, as well.

On to impressions of the area:

Fry's Electronics (Sunnyvale) - what a cluttered mess. They have an impressive range of stuff, but the place just feels cluttered and disorganized.

IKEA (I know, not really a Cali thing, but we don't have one, yet, in Michigan) - Damn, that's an impressive place to shop.

San Francisco itself: Umm, bring a map next time 'cuz wow, was I lost.

Admittedly, I haven't really gotten a lot of impressions together yet. I've spent a lot of time in this first week of my new job at work, and I honestly expect that to resume after I get my stuff settled in at my new apartment, and some older commitments taken care of. So, I think I'm liking things here, though I'm still not really entirely comfortable with it all.

Anyway, I need to kill my bank account by paying all my bills (I can't wait until the reimbursement checks hit). More later when I get some free time and something more meaningful to blog about. (When my stuff shows up, I think I'm going to be setting up a Sunray, for example)

Penguicon and Git

2006-04-20T13:29:00.000-07:00

Penguicon 4.0 is here (tomorrow), and I'll be giving a talk on Git
at 11pm. There should be a bunch of interesting things going on there, and free beer.

Spanning tree and NFS

2006-04-06T08:45:00.000-07:00

For the record, the fact that Debian's startup tends to fail in the presence of smart switches that try to do spanning-tree detection gets really annoying when you have production machines that depend on NFS servers being available.

Technically, this would be solvable by building a proper hosts file on each server, but that's what DNS is for.

So, I'm going to try a script like this on each server during the startup sequence, to try to delay the startup until the default gateway comes up:

#!/bin/bash -e

found_addr=0
loop_count=0

gw=$(/bin/netstat -rn | awk '/^0.0.0.0/{print $2}')
echo "Gateway is $gw"
echo "Waiting for $gw to appear in the arp table."

while [ $found_addr -eq 0 ]; do
echo "Triggering arp lookup with a single ping"
/bin/ping -c 1 -q $gw >/dev/null || true
info=$(/usr/sbin/arp -n $gw | grep "^$gw")
addr=$(echo "$info" | awk '{print $1}')
hwtype=$(echo "$info" | awk '{print $2}')
echo "hwtype = $hwtype, addr = $addr"
if [ "$hwtype" = "ether" ]; then
found_addr=1
fi

loop_count=$(expr $loop_count + 1)
if [ $loop_count -gt 120 ];
then
echo "120 passes through the loop and nothing happening."
echo "Giving up and moving on."
exit 0
fi

if [ $found_addr -eq 0 ]; then
sleep 1
fi
done

Mind mapping - Vym

2006-03-16T16:19:00.000-08:00

This week, at the Michigan User Group meeting, I started playing with a new tool, Vym, that allows you to lay out thoughts, reorganize at will, and get things into somewhat of a structure. So, here's the mind map I made:

(or here is a full size version). Also in that directory I have the actual .vym file I made. This is, in my opinion, and incredibly neat tool for so many things, so I have to rave about it.

Open Source Accessibility

2006-03-16T14:38:00.000-08:00

Jonathan McDowell comments on the accessibility of people in the Open Source community - and I agree, it is an important aspect, but he mentions the fact that he hasn't asked for an autograph in a long time. It occurred to me that in the OSS community, you don't really need autographs, all it takes is a positive comment on code you've submitted, because that's an infinitely more meaningful gesture than a meaningless autograph.

Perl cross-platform compatibility

2006-02-26T10:05:00.000-08:00

Anyone that knows me knows that my generally preferred language is Perl. (I'll wait for the jeering to die down.)
Generally, this isn't a bad thing, as Perl is remarkably cross-platform compatible. Unfortunately, Windows is an utter disaster in terms of compatibility with Unix based systems.

Working on a few Git (Introduction) related tools, such as a replacement for "cvs annotate"/"svn blame", we've learned that the current best practices for doing safe pipe opens aren't very compatible.

For example, to call a program like "ls", you might do something like this:

open(PIPE, "-|", "ls");

while(<PIPE>) { do_something($_); }

close(PIPE);

Unfortunately, that fails on Perl 5.6, and on ActiveState (Any version, I believe.)

So on Perl 5.6, the solution is to instead do the two steps by hand:

my $pid = open my $kid, "-|";

defined $pid or die "Cannot fork: $!";

unless ($pid) {

exec "ls";

die "Cannot exec ls: $!";

}

while(< $kid>) { do_something($_) }

close($kid);

But this fails on ActiveState, because ActiveState doesn't handle the forked pipe open well (ok, at all). ActiveState doesn't appear to have a good solution for this, at all, so at this point, we're forced to fall back on something like backquotes (``) or the easier to read qx(). (qx() is just an alternate form of backquotes, that gives you some control of how interpolation happens.

The problem with using qx() is that it is a backquote form, so it return a list of lines, not a filehandle. Since that's a very different way to do things, and potentially a big performance hit on systems that don't need the qx() work arounds, I forced myself to find a way to just hide the complexity:

sub open_pipe {
if ($^O eq '##INSERT_ACTIVESTATE_STRING_HERE##') {
return open_pipe_activestate(@_);
} else {
return open_pipe_normal(@_);
}
}

sub open_pipe_activestate {
tie *fh, "Git::ActiveStatePipe", @_;
return *fh;
}

sub open_pipe_normal {
my (@execlist) = @_;

my $pid = open my $kid, "-|";
defined $pid or die "Cannot fork: $!";

unless ($pid) {
exec @execlist;
die "Cannot exec @execlist: $!";
}

return $kid;
}

package Git::ActiveStatePipe;
use strict;

sub TIEHANDLE {
my ($class, @params) = @_;
my $cmdline = join " ", @params;
my @data = qx{$cmdline};
bless { i => 0, data => \@data }, $class;
}

sub READLINE {
my $self = shift;
if ($self->{i} >= scalar @{$self->{data}}) {
return undef;
}
return $self->{'data'}->[ $self->{i}++ ];
}

sub CLOSE {
my $self = shift;
delete $self->{data};
delete $self->{i};
}

sub EOF {
my $self = shift;
return ($self->{i} >= scalar @{$self->{data}});
}

It should, hopefully, be fairly obvious what is going on there.

For a normal system, we use the Perl 5.6 compatible method, and return a filehandle that works normally.

For an ActiveState system, we tie a glob to a special object that provides some very basic emulation of a filehandle, and internally, calls qx() and indexes across an array returning the data.

So, that's how "git annotate" is going to be cross-platform compatible, at least, as of today, that is my plan.

Thanks to Randal Schwartz (merlyn) for providing part of the inspiration for this, and the rest of the people on the Git list that helped hash out various approaches to this that didn't quite seem as clean or as nice as this one, in the long run.

I think this method keeps all the security advantages of the argument list forms, while actually still managing to work on crippled systems, so I'm fairly pleased with it.

Yaird support for module options

2006-02-16T16:33:00.000-08:00

I recently was given a Dell Latitude 610 to use for work. So of course I'm running Debian on it. Tonight, I decided to play a DVD, and realized I had never tried to use the DVD player.

Turns out that with SATA drives, you need to give the module option atapi_enabled=1 to make it work. To get this option into the initrd, I had to create a file named /etc/modprobe.d/local as follows:

options libata atapi_enabled=1

and then recreate the initrd with:

mkinitrd.yaird -o /boot/initrd.img-2.6.15-rc5-686 2.6.15-rc5-686

update-grub

and now I'm watching Kill Bill 2.

Switching to a UTF-8 based system

2006-02-05T16:33:00.000-08:00

On Friday, I started switching my work machines over to a default locale of en_US.utf8, so diff would work correctly on UTF-8 text files containing Korean.

At home, I read my work email (typically) via mutt, which I leave running in a screen session. It turns out that this change means I need to switch basically everything at home over, as well. Well, I'll leave that for a rainy day, but for the time being, I've started up a few shells (urxvt) with LANG=en_US.utf8 instead, so things will work in the meantime.

I should have done this a long time ago, to be honest, but I guess I'm still a little bit worried about how widespread support for this is.

The Linux Kernel, Debian packages, and fakeroot.

2006-01-17T07:46:00.000-08:00

In a vain attempt to let more people stumble across this, the "deb-pkg" target in the Linux Kernel makefile now needs to be run with fakeroot -u instead of fakeroot, due to a change in the way scripts/setlocalversion tracks down the current git version.

I plan on taking another look at the builddeb script to see about incorporating this into it, internally, but I also need to verify if the build process for a rpm needs to be run as root or if it can run as any user.

Free newgroup access...

2005-10-14T02:42:00.000-07:00

Free download limit: 2.00GB
Current usage: 1.68GB
As of: 10/14/2005
Monthly projection: 871.59GB

I appear to have, umm, timed my usage today quite well. :)

Serenity

2005-10-11T15:43:00.000-07:00

There's a good review of Serenity on Reason.com. (Spoilerific.)

I don't follow many of the references present in that review, but I think I get the idea, and I'll summarize it: Serenity is, in every sense of the word, Sci-Fi. Much of what we call Sci-Fi in the modern genre misses many of the older, classic Sci-Fi meanings. I think Serenity is a well executed reminder that Sci-Fi is more than just "guns in space".

LaTeX, grad school

2005-09-25T14:00:00.000-07:00

In the interests in doubling up some learning this weekend, I spent 5 or 6 hours using LaTeX to do some homework for my graduate-level Databases Class. (Aside: I'm attending Grad School at Oakland University now. I kinda missed going to school. On the other hand, I kinda miss free-time now. but whatever.) So, I really am intrigued by the power LaTeX has. The learning curve on it is a bit daunting at first, but once you get the hang of typing out equations in it, it's really not so bad.
I don't think I could imagine using anything else to do equation-laden things in the future.... So, for anyone reading this and pondering how to handle equation-laden stuff - I really don't think there are any realistic options for you, outside of learning LaTeX. If anyone is interested, post a comment and I'll put up the raw .tex file I used for this. (Well, after I turn the homework in. Gotta have some ethics.)

I spoke to soon

2005-09-05T02:58:00.000-07:00

The Intel wireless drivers are rather old, and seem to be less stable than the current development drivers.

I don't really understand why the 1.0.0 driver went in, rather than the 1.0.6 driver, given that the 1.0.6 driver has been out for several months.

In any case, I put a vote in for a more current driver, on the kernel list. Now I wait.

Finally, Intel wireless drivers in the kernel

2005-09-03T14:32:00.000-07:00

I've been hand-building my IPW2200 drivers for a while, but as of today (maybe yesterday), the drivers are part of the main kernel tree.

This makes me very happy. My laptop is now fully functional from the mainline kernel. (Toshiba Satellite A55-[something])

GenCon 2005

2005-08-20T06:47:00.000-07:00

I'm at GenCon again this year, working for SaberTooth Games. (A division of Games Workshop now.)

This is pretty nice, relaxing, running small events and helping teach people how to play the game - and getting to know the people we'll be working with next year if things grow bigger. It's really the first low-stress GenCon I've had in 4 or 5 years. Something about not trying to run tournaments for 200+ people for 3 days straight is rather nice.

Oh, and I think the new Warhammer 40k CCG (Dark Millenium) is actually a pretty easy to learn game. Simple rules, nice level of tactics to expand into... so, I'm kinda enthused by it, too.

So, anyway, that's my thoughts from GenCon.

Stupid web tests

2005-08-13T02:21:00.000-07:00

Getting noticed, corrollaries

2005-07-28T19:15:00.000-07:00

I've picked up some web browsing via "planet" sites recently. Most notably, Planet Arslinux, but also Planet Debian and Planet Kernel. On Planet Arslinux today, I saw Jay Wren's post about products being noticed.. This got me to thinking - and especially, his points about iTunes and Google made me think about why I'm a user of those two things, and also, other things.

As I ponder this, I realize that the products I'm passionate about, all share a common trait. Mostly, I hate their competition because it sucks. I think this is just another way of saying the "10 times" rule Jay postulates.

With Google, I switched because it worked, and found me the information I needed. This was a rather remarkable feature at the time. Mostly because search engines just sucked. With iTunes, I started using it because I bought an iPod. On the other hand, it's actually a really good mp3 player. And CD ripper. ID3 tag editor..... etc. It does a good job at all these things. None of the other players I've used provided all these features in one spot and got more than one of them right. In fact, most failed to get even one right. So, it's not that iTunes is great. It's just that the rest suck. (My mp3 player of choice before I stumbled on iTunes was a hand-written Perl script that I called "weightedplay", that automatically noticed new mp3s and played more recently added stuff slightly more often than older things. It was really quite simple, and other than it's lack of GUI, it was better than anything else I'd found for just playing songs.)

So those are the things that come to mind immediately, and why I switched away from their competitors. I'll bring up another one, that's occurred to me as I write this. A long, long time ago, I switched from using RedHat as my Linux distro of choice, to using Debian. In this case, it wasn't because Debian was great. I had done some customizations to my RedHat installation in strange places (/etc/inetd.conf, IIRC), and doing an upgrade wiped out those changes without warning. In retrospect, after years of being away, I realize that my customizations *might* have been saved in .rpmsave files. But I was angry, so I found another distro that I was promised would tell me before it nuked my customizations. In the meantime, I've realized that apt actually works, and that I don't really mind my packages being a little bit old if they work, so I'm happy with Debian. But I think this proves that it's not how good the new choice is, but how much you dislike the old choice that sets these patterns in our minds.

This is the thing MicroSoft doesn't get on search. For a very long time they thought that it was OK to fool their users by sneaking in paid-for results in their search listings. That's bad, and hurts the accuracy of their results, in the user's minds. This is why they will be struggling to recover influence in search for a very long time. (Oddly, I've noticed that Google's advertisements are usually well targetted at my searches, which is pretty nice. I've even occassionally clicked on one. The beauty of relevance.)

So, I guess that's my take on the thought. It's not how good you are, but how bad you make your competitor look.

Ok, fine...

2005-07-28T13:04:00.000-07:00

Meme of the week - what OS are you?

Which OS are You?

Humourously, the only computers I have at home that aren't running Debian at the moment are my WRT54G and my Tivo.

That makes 7 others that are running Debian. I need a local mirror.

Git - A short update

2005-07-22T22:05:00.000-07:00

I've been spending my free time hacking away on "Git", the source code management tool created by Linus Torvalds. It's rather absorbing now that I'm really starting to grok how the command line tools all work, and I'm adding my own into the mix to fill the gaps. I guess the trick to everything is to just start using it so you figure out what is missing, either in the tool, or in your own knowledge.

I wish I felt up to trying to convert work to this tool - maybe I'll do that next year when our BK licenses come up for renewal again. I suppose I should get around to figuring out how to pull all the data out of BK and reimport all of it into Git then.

Ugh, to say the least.