I just ran across Jordan Sissel's sysadvent project, where he posted one article about systems administration each day from the 1st of December to the 25th.

Jordan has been a great communicator when we've exchanged mail (he's the author of xdotool, which i maintain for debian). Unsurprisingly, his posts in sysadvent are also excellent. He has a broad knowledge of what tools are available, clever insights in how to connect them together, an engaging and clear writing style, and sharp sense of what really should matter to a systems administrator. Definitely worth reading!

I've been playing around with an eeePC 900, which is very well-supported by debian. Kudos to the eeepc team!

I had one problem with it, after upgrading eeepc-acpi-scripts from version 1.0.4 to 1.0.9: with 1.0.4, i was able to use an ACPI hotkey to disable and re-enable the wireless. With 1.0.9, the device did not come back up for me after a toggle. The problem was resolved for me with:

echo pciehp >> /etc/modules

which i figured out from reading a brief post on the very-informative debian-eeepc-devel mailing list. The rest of this post explores why that was the answer for me.

This entry has been truncated read the full entry.

Ever thought that there should be an automated way to handle ssh keys? Do you know the administrators of your servers, and wish that SSH could verify new host keys from them automatically, based on your personal connections to the web-of-trust? Do you wish you could revoke and/or rotate your old SSH authentication keys without having to log into every single machine you have an account on?

Do you administer servers, and wish you could re-key them without sowing massive confusion among your users (or worse, encouraging bad security habits among them)? Do you wish you could grant access to your users by name, instead of by opaque string? Do you wish you could rapidly revoke access to a user (or compromised key) across a group of machines by disabling authentication for that user?

A group of us have been working on a public key infrastructure for SSH. Monkeysphere makes use of the existing OpenPGP web-of-trust to fetch and cryptographically validate (and revoke!) keys. This works in both direction: authorized_keys and known_hosts are handled. Monkeysphere gives users and admins tools to deal with SSH keys by thinking about the people and machines to whom the keys belong, instead of requiring humans to do tedious (and error-prone) manual key verification.

We have debian packages available which should install against lenny (for i386, amd64, powerpc, and arm architectures at the moment), a mailing list, and open ears for good questions, suggestions and criticism.

If you have a chance to give it a try (as a user or as an admin), it would be great to get feedback.

On the testing and unstable systems that i run, i occasionally get a disturbing "there are N newly obsolete packages" message. For example

[0 root@squeak ~]# aptitude update
Hit http://cmrg.fifthhorseman.net unstable Release.gpg
 [...]
Get:42 http://ftp.us.debian.org unstable/main [2077kB]                          
Fetched 15.2MB in 57s (262kB/s)                                                 
Reading package lists... Done
Updating debtags database...... Done

Current status: 3 updates [+3], 1136 new [-21212].
There are 1804 newly obsolete packages.
[255 root@squeak ~]# 

Usually, i can fix this with another aptitude update. Does this mean i've hit a mirror at an inopportune time? Or do i have apt mis-configured somehow?

Other people have documented this before, and casual conversation with friends lets me know it's not just me.

If this is due to some inconsistency at the mirrors (particularly for suites with rolling updates?), is there some way to engineer the mirror transfers so that this doesn't happen? For example (i have no idea how this stuff is done, so i'm just making this up):

1. rsync the pool/ without --delete to all mirrors,
2. rsync the Packages and Release, and DiffIndex files to all mirrors,
3. rsync the pool/ with --delete

Or should apt itself deal with these circumstances differently? It seems like signed apt ought to be able to detect when something like this is amiss.

I'm reluctant to consider the situation acceptable, because i'd like to be able to trust apt in general. In particular, i can imagine apt eventually becoming more assertive about suggesting removal of newly obsolete and unsupported packages. If it is wrong about what's obsolete and unsupported, this could be dangerous.

A high school student i work with is trying to gather data on the prevalence of mirror-reading (the ability to easily read text that is reversed left-to-right).

He's posted a simple webapp to gather data points for his experiment. It should take only a couple of minutes to try it out and give him some data. If he gets enough data to write something up, i'll try to encourage him to publish his results as well.

The CIO and the Office For Technology of New York State (USA) just released a new report on electronic records strategy, titled A Strategy for Openness: Enhancing E-Records Access in New York State.

I submitted my own comments during the call for public comments earlier this year -- apparently there were only 114 comments submitted by the public (you can see the call for comments and an overview/analysis of the responses). I wish there'd been more, since it looks like the 59 individual commenters were overwhelmingly supportive of a requirement for the Gov't to use ODF. The rest of the comments were from governments, NGOs, and businesses, and they don't seem as unanimous as the individual commenters.

Looking at the metadata for the executive summary shows just how far we have to go:

[0 dkg@squeak doc]$ pdfinfo PartIerecordsStudy.pdf  | egrep '^Title|Producer|Creator'
Title:          Microsoft Word - PartIerecordsStudy.doc
Creator:        PScript5.dll Version 5.2.2
Producer:       Acrobat Distiller 8.1.0 (Windows)
[0 dkg@squeak doc]$ 

And in a message sent to commenters announcing the report, they seem to be aware that they're not quite doing their best at interoperability:

(Please note this is the first time any of NYS government's agencies has ever published a document in ODF format. We ran into problems this morning not caused by the format, but rather with some controls on our systems which were not prepared to encounter such documents. So the website links to ODF versions are not working at the moment, but will be repaired soon).

What web server are they running that is having trouble "encountering" ODF? No surprise:

[0 dkg@squeak doc]$ wget -S -O/dev/null 'http://www.oft.state.ny.us/policy/esra/erecords-study.htm' 2>&1 | grep Server
  Server: Microsoft-IIS/6.0
[0 dkg@squeak doc]$ 

Sigh. My initial reaction to the report (i've only read the executive summary, not the supporting documentation) is disappointment. While they claim that "openness" is an important feature, they fall far short of taking a strong stand for free and open formats. The main thrust of the executive summary seems to be (paraphrasing here, i welcome corrections):

• it's a bad idea to mandate a single particular technology because technologies will change faster than law.
• we need an Electronic Records Committee (ERC) to provide regular, executive guidance on electronic record storage and maintenance
• "openness" is good, but needs to be weighed against (ill-specified, at least in the executive summary) "other features"
• we're not going to make any concrete recommendations about what to do next

While these are unsurprising conclusions, they don't take the strong stand i'd like them to take. They seem aware of the "chicken and egg" problem of wide adoption and vendor support, and aware of the anti-democratic effects of proprietary formats. But they don't seem to be willing to act as a catalyst to push free formats explicitly. If they did, they could break the chicken/egg vendor lock-in cycle and facilitate better open government practices while simultaneously encouraging a healthy information ecosystem.

At this point, i suppose my biggest hope from the process is that the proposed ERC forms pilot groups with long-term goals modeled after Munich, but i'm not holding my breath. If there are any advocates or participants in the Munich process who want to share what's been working for you (and what hasn't), i'd be interested in hearing about it.

An interesting thread on openssh-unix-dev points out a way that a compromised remote ssh account or host could potentially trick users into divulging the passphrase for their local secret key. Given that we're all dealing with the potential for compromised ssh hosts lately, i think this is an important consideration. (this has apparently been raised on openssh-unix-dev as far back as 8 years ago).

Unfortunately, a couple messages with mangled headers seem to have broken the thread into two chunks in the archive, severing the start of the thread from the part where it starts getting really good: In particular, Damien Miller's suggestion is to forbid in-terminal key-based authentication entirely, and rely instead on ssh-agent. My impression of this strategy is that it's analogous to forcing an out-of-band verification process -- since the in-channel communication is happening in the tty, and the remote host will have some level of control over the tty once authentication succeeds, it's important that any locally-divulged secrets (e.g. the passphrase for the local secret key) are not transmitted over the tty in question.

Of course, disabling in-terminal key-based authentication creates something of a usability problem for users who aren't using X11 (and who don't have some other method for an out-of-band ssh-askpass). And it also points out the additional problems with X11 forwarding (e.g. a compromised host allowed to forward X connections could trigger a mimicry of a standard ssh-askpass password prompt). This is something that debian might want to consider, as we've diverged from upstream on the default settings for ForwardX11Trusted.

I'm uncomfortable with how Damien's suggestion raises the bar to key-based authentication in general, since users will now have to understand both keys and agents in order to effectively authenticate this way. But maybe that's what's needed, since we desperately need to phase out password-based authentication (and all the security pitfalls associated with it) in this brave new networked world.

At any rate, i've adopted the suggestion for now:

[0 dkg@squeak ~]$ tail -n1 .ssh/config
IdentityFile none
[0 dkg@squeak ~]$ 

I'm curious to know if other people have adopted this strategy, or have other mitigating techniques.

I'm excited by version 7 of debhelper, in particular the opportunity for debian/rules minimization. I suspect this will lower the barrier for rapid, reasonable packaging of simple software tools in a way that should be easy to audit and maintain.

The most-minimized debian/rules possible with it is just (from dh(1)):

#!/usr/bin/make -f
%:
	dh $@

And all of the project-specific interesting bits go into nice, clean, well-named files under debian/.

However, lintian (at least as of 1.23.48) complains loudly about the minimized debian/rules:

E: xdotool source: debian-rules-missing-required-target binary
E: xdotool source: debian-rules-missing-required-target binary-arch
E: xdotool source: debian-rules-missing-required-target binary-indep
E: xdotool source: debian-rules-missing-required-target build
E: xdotool source: debian-rules-missing-required-target clean

I'm not sure the right way to proceed: should i try to manually add overrides for each package that uses a minimized debian/rules? Should lintian recognize debhelper-specific minimized rules files and accept them? Should debhelper assuage lintian somehow? Or is this rules file minimization actually not as good an idea as I think it is?

I notice that mr, which is Joey Hess's first example package using the minimized rules also reports the same errors.

So i've been struggling with an Ubuntu 8.04 networked workstation. It uses libpam-mount to mount the user's homedir (actually, the mountpoint is one level up from the homedir) automatically at login over CIFS.

One of the problems i ran into with this arrangement happened because i was following the pam_mount configuration instructions too literally. In particular, README.Debian.gz says:

For every application used for logging in, there is a file of the form /etc/pam.d/xyz, add the following line at the end of the file:
@include common-pammount

In particular, ubuntu's /etc/pam.d/gdm defaults to:

#%PAM-1.0                                                                                                                                                               
auth    requisite       pam_nologin.so
auth    required        pam_env.so readenv=1
auth    required        pam_env.so readenv=1 envfile=/etc/default/locale
@include common-auth
auth    optional        pam_gnome_keyring.so
@include common-account
session required        pam_limits.so
@include common-session
session optional        pam_gnome_keyring.so auto_start
@include common-password

When i added the @include common-pammount directive to the bottom of this file when using pam_mount, new GNOME sessions failed badly: the gnome-panel didn't appear (which means that the user couldn't log out conveniently), and two error messages popped up at each login with nasty details like:

No database available to save your configuration: Unable to store a value at [...], as the configuration server has no writable databases.

The problem seems to be that libpam-gnome-keyring actually kicks off gconfd-2 during its PAM session invocation. If that comes before libpam-mount's PAM session invocation, then the home directory isn't mounted for the keyring, and gconfd-2 decides that it is unable to save any settings. Since gconfd then persists for the rest of the session, further GNOME session components try to talk to it and it refuses, even though the gconf db is now available (via the mounted homedir).

Since the order of the lines in a /etc/pam.d/* are semantically relevant, i'm usually very reluctant to tamper with the defaults. However, i think the correct /etc/pam.d/gdm for this scenario (or any pam-mount scenario using GNOME where the homedir might not be present at all before the session) is actually:

#%PAM-1.0                                                                                                                                                               
auth    requisite       pam_nologin.so
auth    required        pam_env.so readenv=1
auth    required        pam_env.so readenv=1 envfile=/etc/default/locale
@include common-auth
@include common-account
session required        pam_limits.so
@include common-session
@include common-password
@include common-pammount
auth    optional        pam_gnome_keyring.so
session optional        pam_gnome_keyring.so auto_start

With this configuration in place, i can successfully log in with a test user, anyway (and move on to the next problem, which appears to be SQLite over CIFS, ugh).

These sorts of problems are tough to nail down:

• Is this overall problem due to a bug in the documentation for libpam-mount?
• In gdm for its default weirdly-ordered PAM config?
• In libpam-gnome-keyring for launching gconf-d-2 in the first place?
• In libgconf2-4 for gconfd-2 not being able to notice when the directories it wants become available?

Or maybe it's just a configuration detail that i should have known about and expected to deal with in the first place. Ah, well.

I finally got to try out goodbye-microsoft.com this past weekend. It uses similar principles as UNetbootin, which Utumno recently wrote about. I had a donated machine in a computer lab i volunteer at with a flakey CD-ROM which i couldn't get to netboot. Since it had Windows already on it, i booted to Windows, hooked into the network, downloaded a debian installer for Windows, ran it, and was on my way.

The project is clean, simply done, and provides a good interface in the leadup to rebooting into a debian installer. I highly recommend it for folks who need to switch a 'doze machine to the One True OS.

My one concern with this approach is if the debian install fails midway through, (and you haven't done any gymnastics to preserve a dual-boot scenario) you could be left with an unbootable machine. But the debian installer is pretty much rock solid these days, especially with common commodity hardware, so you'd probably have to trip over the power cord to get caught out like that.