Thursday, September 7, 2006

The future of networking

Back in January, I read a blog post about Van Jacobson's talk at Linux Conf.au 2006. (LWN's coverage and the slides) This is some amazing stuff. I can't really do these justice - go read. The slides show some amazing numbers in terms of potential speedups through rearchitecting.


What prompted this entry, however, has nothing to do with that. Instead, it has to do with another Van Jacobson talk at Google that deserves attention - namely A New Way to Look at Networking. Where the other paper and talk were about performance, this talk is more about thinking about what's next - what will the ubiquitous network of 20 years from now look like, and how to make it work. (I'll spoil a slide near the end, as a teaser - PGP meets BitTorrent and hitches a ride on an airplane. (Not Pacific Air 121, however.))


Ye Olde Scrum

At work this week there was a very good talk about "Scrum Et Al", given by Ken Schwaber.

Executive summary: Go watch this, if you're at all interested in Agile teams and iterative development, it's definitely worth your time.

What struck me most while watching this, was how much I wished I had gotten around to reading something about it years ago. (I still haven't, but this video is enough to get me interested now.) At my last job, we were doing something informal, but not too far off of this. Quarterly releases, along with a relentless pace of new features, improving interfaces, improving security, improving developer productivity, etc. Seeing this talk reaffirms my belief that the only good way to build software products is iteratively. Being able to say, "I only need to worry about X right now." and sitting down and doing X, getting it done, clean, bugfixed, and then being able to say, "Ok, what's next?", all the while having a product that can be released at basically any time, maybe multiple times, and a simply relentless pace of improvements? This is great! Unfortunately, I don't think very many business majors have figured this out yet.

Saturday, August 19, 2006

Git presentations

At Penguicon in April, I gave a basic Git tutorial, and this week at Ubucon I gave a small variation of that talk, where I tried to cover why distributed SCMs are so much better for free software work.

These presentations are both up at http://h4x0r5.com/~ryan/presentations/

Also, Junio Hamano (Git Maintainer), gave a great talk at OLS:  http://members.cox.net/junkio/200607-ols.pdf

Monday, July 3, 2006

Hardware update (belated)

So, eventually, I did figure out what was wrong with my hardware.

It appears to have been nothing more than a horribly overheating processor, and thus, fixed by a simple application of some thermal paste. (65C = bad, generally)

On that note, I now have to decide what kind of upgrading I want to do, and whether or not I build a MythTV box (probably), but at least I can sit on it for a bit until prices drop as rumored in July.

Friday, June 2, 2006

Passwords, argh

With all the things I need to do to get organized around a new job, one of the things I've ended up doing a lot of is creating new accounts.  With new passwords, of course.

What really irks me is that there is *no* consistency in the password requirements, so, even if I wanted to reuse a password for a few things, it would be darn near impossible.

{0,2}capital letters, {0,2}numbers {0,2}lowercase letters, {4-6}minimum characters, {4-8}maximum characters.
The situation has gotten, well, frankly ridiculous.  Maybe Passport (or something similar) wasn't as horrible an idea as it first appeared.

Wednesday, May 24, 2006

One dead (sorta) computer

Now that I'm moved, and I'm starting to get settled in, finally, I setup some computers tonight.

  • Well, my gaming/desktop computer worked fine (actually, haven't booted Linux on it, yet, but Windows is fine.)

  • My old webhost/server box is up fine, as well.

  • Unfortunately, my primary NFS/file server won't boot.


I get a fun variety of errors:

  • Sometimes, I get a recursive fault and the kernel locks (2.6.16, though, I'm not sure it matters)

  • Sometimes, I get a "failed to mount /dev/md4, please specify root fs with root=" (etc/paraphrased)

  • Sometimes, I get a crc error during "Uncompressing Linux"

  • Sometimes, I get an "Invalid format" during "Uncompressing Linux"

  • Sometimes, I get partway into boot and a beautiful screen full or random colors and letters flashes, and the machine reboots.


Given how inconsistent the failures are, my gut says "motherboard, ram or CPU", and not harddrive.  (I did really expect at least one harddrive to fail, though.)

The case for this machine did get a nice dent added to it during the move (on the motherboard side, not the other side).

I think I need to make a DSL boot USB key, and see if that fixes it, just to confirm, otherwise, debugging this is going to be a pain.  Damn fileserver shouldn't be the broken machine. :(

Sunday, May 14, 2006

News from a new state

So, as a few of the readers of this blog know, I'm moving to California (Bay Area/Silicon Valley)
Well, technically, I suppose, I've moved.  My stuff hasn't quite arrived yet, but within the next day or two it should all arrive.  It will be nice to get my normal computers back and be able to work on a machine that isn't a laptop, while at home. I suppose I'll have to finally redefine home to be my new (expensive) apartment, as well.

On to impressions of the area:

  1. Fry's Electronics (Sunnyvale) - what a cluttered mess.  They have an impressive range of stuff, but the place just feels cluttered and disorganized.

  2. IKEA (I know, not really a Cali thing, but we don't have one, yet, in Michigan) - Damn, that's an impressive place to shop.

  3. San Francisco itself: Umm, bring a map next time 'cuz wow, was I lost.


Admittedly, I haven't really gotten a lot of impressions together yet.  I've spent a lot of time in this first week of my new job at work, and I honestly expect that to resume after I get my stuff settled in at my new apartment, and some older commitments taken care of.  So, I think I'm liking things here, though I'm still not really entirely comfortable with it all.

Anyway, I need to kill my bank account by paying all my bills (I can't wait until the reimbursement checks hit).  More later when I get some free time and something more meaningful to blog about.  (When my stuff shows up, I think I'm going to be setting up a Sunray, for example)

Thursday, April 20, 2006

Penguicon and Git

Penguicon 4.0 is here (tomorrow), and I'll be giving a talk on Git
at 11pm. There should be a bunch of interesting things going on there, and free beer.

Thursday, April 6, 2006

Spanning tree and NFS

For the record, the fact that Debian's startup tends to fail in the presence of smart switches that try to do spanning-tree detection gets really annoying when you have production machines that depend on NFS servers being available.

Technically, this would be solvable by building a proper hosts file on each server, but that's what DNS is for.

So, I'm going to try a script like this on each server during the startup sequence, to try to delay the startup until the default gateway comes up:

Thursday, March 16, 2006

Mind mapping - Vym

This week, at the Michigan User Group meeting, I started playing with a new tool, Vym, that allows you to lay out thoughts, reorganize at will, and get things into somewhat of a structure. So, here's the mind map I made:
Mug Security mind map

(or here is a full size version).  Also in that directory I have the actual .vym file I made.  This is, in my opinion, and incredibly neat tool for so many things, so I have to rave about it.

Open Source Accessibility

Jonathan McDowell comments on the accessibility of people in the Open Source community - and I agree, it is an important aspect, but he mentions the fact that he hasn't asked for an autograph in a long time.  It occurred to me that in the OSS community, you don't really need autographs, all it takes is a positive comment on code you've submitted, because that's an infinitely more meaningful gesture than a meaningless autograph.

Sunday, February 26, 2006

Perl cross-platform compatibility

Anyone that knows me knows that my generally preferred language is Perl. (I'll wait for the jeering to die down.)
Generally, this isn't a bad thing, as Perl is remarkably cross-platform compatible. Unfortunately, Windows is an utter disaster in terms of compatibility with Unix based systems.

Working on a few Git (Introduction) related tools, such as a replacement for "cvs annotate"/"svn blame", we've learned that the current best practices for doing safe pipe opens aren't very compatible.

For example, to call a program like "ls", you might do something like this:
open(PIPE, "-|", "ls");

while(<PIPE>) { do_something($_); }

close(PIPE);

Unfortunately, that fails on Perl 5.6, and on ActiveState (Any version, I believe.)

So on Perl 5.6, the solution is to instead do the two steps by hand:
my $pid = open my $kid, "-|";

defined $pid or die "Cannot fork: $!";

unless ($pid) {

exec "ls";

die "Cannot exec ls: $!";

}

while(< $kid>) { do_something($_) }

close($kid);

But this fails on ActiveState, because ActiveState doesn't handle the forked pipe open well (ok, at all). ActiveState doesn't appear to have a good solution for this, at all, so at this point, we're forced to fall back on something like backquotes (``) or the easier to read qx(). (qx() is just an alternate form of backquotes, that gives you some control of how interpolation happens.

The problem with using qx() is that it is a backquote form, so it return a list of lines, not a filehandle. Since that's a very different way to do things, and potentially a big performance hit on systems that don't need the qx() work arounds, I forced myself to find a way to just hide the complexity:
sub open_pipe {
if ($^O eq '##INSERT_ACTIVESTATE_STRING_HERE##') {
return open_pipe_activestate(@_);
} else {
return open_pipe_normal(@_);
}
}

sub open_pipe_activestate {
tie *fh, "Git::ActiveStatePipe", @_;
return *fh;
}

sub open_pipe_normal {
my (@execlist) = @_;

my $pid = open my $kid, "-|";
defined $pid or die "Cannot fork: $!";

unless ($pid) {
exec @execlist;
die "Cannot exec @execlist: $!";
}

return $kid;
}

package Git::ActiveStatePipe;
use strict;

sub TIEHANDLE {
my ($class, @params) = @_;
my $cmdline = join " ", @params;
my @data = qx{$cmdline};
bless { i => 0, data => \@data }, $class;
}

sub READLINE {
my $self = shift;
if ($self->{i} >= scalar @{$self->{data}}) {
return undef;
}
return $self->{'data'}->[ $self->{i}++ ];
}

sub CLOSE {
my $self = shift;
delete $self->{data};
delete $self->{i};
}

sub EOF {
my $self = shift;
return ($self->{i} >= scalar @{$self->{data}});
}

It should, hopefully, be fairly obvious what is going on there.

For a normal system, we use the Perl 5.6 compatible method, and return a filehandle that works normally.

For an ActiveState system, we tie a glob to a special object that provides some very basic emulation of a filehandle, and internally, calls qx() and indexes across an array returning the data.

So, that's how "git annotate" is going to be cross-platform compatible, at least, as of today, that is my plan.

Thanks to Randal Schwartz (merlyn) for providing part of the inspiration for this, and the rest of the people on the Git list that helped hash out various approaches to this that didn't quite seem as clean or as nice as this one, in the long run.

I think this method keeps all the security advantages of the argument list forms, while actually still managing to work on crippled systems, so I'm fairly pleased with it.

Thursday, February 16, 2006

Yaird support for module options

I recently was given a Dell Latitude 610 to use for work. So of course I'm running Debian on it. Tonight, I decided to play a DVD, and realized I had never tried to use the DVD player.

Turns out that with SATA drives, you need to give the module option atapi_enabled=1 to make it work. To get this option into the initrd, I had to create a file named /etc/modprobe.d/local as follows:
options libata atapi_enabled=1

and then recreate the initrd with:
mkinitrd.yaird -o /boot/initrd.img-2.6.15-rc5-686 2.6.15-rc5-686

update-grub

and now I'm watching Kill Bill 2.

Sunday, February 5, 2006

Switching to a UTF-8 based system

On Friday, I started switching my work machines over to a default locale of en_US.utf8, so diff would work correctly on UTF-8 text files containing Korean.

At home, I read my work email (typically) via mutt, which I leave running in a screen session.  It turns out that this change means I need to switch basically everything at home over, as well.  Well, I'll leave that for a rainy day, but for the time being, I've started up a few shells (urxvt) with LANG=en_US.utf8 instead, so things will work in the meantime.

I should have done this a long time ago, to be honest, but I guess I'm still a little bit worried about how widespread support for this is.

Tuesday, January 17, 2006

The Linux Kernel, Debian packages, and fakeroot.

In a vain attempt to let more people stumble across this, the "deb-pkg" target in the Linux Kernel makefile now needs to be run with fakeroot -u instead of fakeroot, due to a change in the way scripts/setlocalversion tracks down the current git version.

I plan on taking another look at the builddeb script to see about incorporating this into it, internally, but I also need to verify if the build process for a rpm needs to be run as root or if it can run as any user.