Everything Breaks at Once

It seems that when one thing breaks, everything breaks. Ceiling starts leaking, TV loses convergence, power supply on computer goes out, server's UPS battery dies, the lithium battery pack for a portable DVD player starts to swell and explode, iPod ends up in the toilet, phone LCD cracks. It's partly because I'm over-tired and accidentally breaking crap, but it also appears that entropy has hit my living room en force.

I re-caulked the upstairs shower, replaced the TV, dug a new power supply out of the closet and got a new iPod. However - I've been busy with other stuff, so I haven't swapped the PSU in the workstation yet and I haven't completely re-installed all the home theater stuff, including MythTV box. And since I've already reduced every other electrical doodad in my house to its bare components, I might as well strip apart the Myth box and update it as well.

I have a pcHDTV card, now sitting dormant in the aforementioned workstation with a fried PSU. I'm thinking of ripping it out of the workstation and placing it into the MythTV Mini-ITX box. There's only room for one PCI card... so I'll have to remove the old Hauppauge cable TV tuner. Yet the Hauppauge WinTV-PVR-150 is also the IrDA receiver for my remote control - so replacing the tuner card would mean losing the DVR's infrared receiver.

I started looking around for alternatives and found a whole slew of Linux-ready solutions. An easy solution that I didn't think of at first was using a generic HID Windows Media Center remote, since it just pretends to be a USB keyboard. Re-map the keycodes and you're pretty much ready to go... assuming you're willing to re-map a universal remote and the Myth keyboard bindings to accommodate the case-sensitivity.

One could also use the receiver from a SnapStream Firefly Mini - they appear to be well supported by MythTV and would be a nice out-of-the-box alternative, given that everything is ready to go. Hopefully the reports of limited range wouldn't be a problem given my spartan living room setup.

Of course, like every self-respecting home theater geek, I already have a programmable universal remote that controls my whole setup. I don't need the additional remote that the previous two solutions bring, so it seems unnecessary. The first thing I would do is throw away the packaged remote, re-program my Harmony 670 and just use the new IR receiver. So it probably makes sense to just buy the receiver if at all possible.

There is a fairly flexible Linux-happy USB infrared receiver/transmitter that is supported by recent versions of lirc and MythTV, but it might be more hardware than I need.

IguanaWorks makes some powerful receivers and transmitters as well, in both serial and USB flavors. Considering serial access is considered the "legit" way for IrDA access, it seems like the cleanest way to go. They are evidently fairly powerful and receptive, but I would still need to get an extension USB cable to rope around to the front of the MythTV box.

Another cottage shop sells a RS232 LIRC receiver built into a DB9 backshell. It's much cheaper and much lower profile and designed to work with lirc natively. The most simple solution to the problem it seems.

I'll probably wander around the local electronics shops looking for a Firefly Mini or MCE remote. If I can't find one this week I might put in an order for a serial port receiver. Never figured I'd use a 9-pin serial port again, that's for sure.

Massive Server Farms Can Equal Massive Failure

A quick trip over to the Amazon Web Service Status Page reveals that massive server resources don't exactly equate to massive uptime numbers. The S3 storage cloud has been down pretty much all day due to "an issue with the communication between several Amazon SQS components." This has affected both the EU and the US, causing some big headaches.

I'm not all that disturbed. I'm just happy that someone makes even bigger impacting gaffes than I do.

Where Are the Java Physics APIs?

If I'm going to try and port Deskblocks to Java then I'll need to find a native Java physics API. If my only choices are native libraries then it makes more sense just to stick to Qt 4 - I'd rather keep this entirely in the "compile once run anywhere domain." Managing native libraries with wrappers functions is just a pain in the butt.

I was surprised to see slim pickins for physics API. For 2D physics I just found Phys2D and for 3D physics I found JBullet, a port of the Bullet physics API. Both seem to be great projects, and both seem to be currently active. Indeed, I'll probably give Phys2D a try and see if I can use it. I guess I just expected an ODE port by now.

KDE 4 Really Fast, Except When It's Really Slow

I've enjoyed openSUSE 11.0 not necessarily because it works flawlessly, but because it's the best working install of KDE 4. KDE 4 has a lot of great potential, but it's not fully realized until you hit KDE 4.1. Since 4.1 is the actual version of KDE 4 meant for widespread usage, I've been downloading the unstable builds from SuSE's build service.

Things have been working fine except for one extremely nagging thing - the initial draw of a konsole window takes 5-10 seconds. It's extremely obnoxious, especially when you need a terminal so often.

I found out this was due to NVIDIA's drivers on newer boards - as described in the KDE techbase and illustrated in the Ubuntu forums and How-To's. Luckily it was an easy fix - running nvidia-settings -a InitialPixmapPlacement=2 -a GlyphCache=1 as root set the appropriate settings in the NVIDIA driver to allow windows to resize at the speed they should. But I don't necessarily blame NVIDIA - the blame deserves to be cast further.

The Linux Hater's Blog brings up a great point about Xorg not being about to allocate offscreen buffers - something that I didn't realize. Xorg lacks a memory manager, so all the stuff you need for full OpenGL support just can't be done. It simply can't be done with Xorg. All the points made in the rant are absolutely right - the memory management infrastructure for pbuffer and framebuffer objects have to be there, otherwise you're hosed.

The core issue that comes from this deficiency is that X11 in and of itself is inherently unable to support OpenGL. Lack of offscreen buffers means that all the great stuff you should be able to do directly in hardware can only be accomplished with a software renderer. Of course, this defeats NVIDIA's entire business model of making the GPU the most important part of your workstation. So they had to massively replace parts of X11; the NVIDIA Linux drivers must, by sheer necessity, replace huge chunks of the XOrg implementation.

After reading the Linux Hater's post a lot of other stuff made sense - why NVIDIA's drivers are so invasive, why you magically don't need to install Xgl to run Compiz Fusion if you are using the proprietary nvidia driver (because it already replaced Xorg for you, thanks), and why KDE's desktop effects had window resizing slowdowns.

NVIDIA didn't break things - they fixed things. They're just trying to live in our broken world.

Java DeskBlocks

Right before I started my latest hacking-for-cash endeavor I was working on DeskBlocks, a physics sandbox rendered directly on the desktop. I was using Qt 4 for development, mostly so I could use ODE and refresh my C++ coding. One problem in development was that things would work perfectly fine in X11 - rounded edges, nice circles bouncing around, render speed was great - then things would work horribly in Windows. Or vice-versa. I could never get things to behave properly cross-platform.

With Java's big update, it's windowing toolkit now allows for translucent and shaped windows on platforms that support it. That means the rendering issues I had with Qt 4 may be solved with Java 6u10.

It makes me start thinking if I should move the project to Java instead. It appears there are Java bindings for ODE... so it just might work.

Real-Life High Dynamic Range Lighting

I've been a sucker for HDR in gaming for a while now. So when I saw mention on Hack a Day about turning your point-and-shoot camera into a full-featured model that allows you to do stop-motion and high dynamic range photography my curiosity was piqued.

The Stuck In Customs HDR Tutorial gives a good reason why HDR photography can be so appealing: our eye adjusts constantly as it is observing its environment, quickly dilating and contracting the pupil to modify the range of light and color hitting the retina. HDR photography does the same thing, re-sampling the image to take in a varying amount of exposure and light at different depths of field.

Hack a Day turned me on to using the CHDK firmware add-on with a Canon SD870 IS. The CHDK add-on software allowed me to do exposure bracketing in continuous shooting mode.

Luckily HDR photography is all the rage lately, so I even had a Grumpy Editor's guide to HDR with Linux. It was great - it introduced me to PFScalibration and Hugin's image alignment, which are both nicely wrapped together in the fantastically easy to use Qtpfsgui toolkit.

Of course I had to turn the flash off and lower the resolution (to allow the continuous mode to write to the SD card faster), but in the end I had a perfect stack of images at varying exposures to import into Qtpfsgui. A tree turns into something more provocative pretty quickly:



I was amazed at the quality of open-source options for photography - Qtpfsgui was great for HDR, and Rawstudio was even more fantastic in dealing with my RAW digital negatives. The SD870 IS doesn't have native RAW file support, but thanks to CHDK and DNG4PS-2 I was able to quickly pull DNG files off of my SD card and start editing them in Rawstudio.

Unfortunately I don't have much if any time to try out new things, but this was a pretty pleasant diversion for the evening.

OpenSUSE 11.0 - Desktop Linux Actually Done Right

I've been pretty unhappy with SuSE Linux as of late. I thought 10.1 was well done, but subsequent releases were of poor build quality and in places just sloppy. There were times it began to show promise, only to fall just short.

I've been living inside of openSUSE 11.0 for the past several days, and tried out RC1 as well. All in all it's a very impressive distribution, and I'm surprised they were able to put this level of polish on KDE 3.5, KDE 4 and Gnome at the same time.

Package management, the Achilles' heel of SuSE up to this point, is scores better. YaST2 actually loads its package management tool quickly, and metadata indexing doesn't take decades like it used to. Compiz is nicely integrated, and KDE 4 is actually quite well done. I'm installing the unstable 4.1 packages right now, and we'll see how that looks as well.

There are some speed issues, but then again I'm using KDE 4 with desktop effects cranked up. However, the fixes to package management alone make SuSE a fantastic winner for the desktop Linux space. Honestly, once KDE 4.1 comes out I wouldn't be surprised at all to see more corporate desktops and developer machines turning to SuSE Linux Enterprise Desktop.

Counting on Trolls Under Your Bridge

It seems no matter if you're in a huge corporate project or a smaller cadre of independent developers, the rules of managing people and code remain the same. And open-source projects tend to manage people much better than their corporate brethren.

In projects with multiple people involved you have to continually worry about people leaving, contributing crappy code or going on a complete tangent. Usually in corporate life people just want the thing to work, and if it randomly happens to hit production and not have complete show-stopping errors, great. But the code could be complete cruft and no one would care.

With an open-source project, however, you must have complete transparency. Brian Fitzpatrick and Ben Collins-Sussman recently gave a presentation on managing people in OSS projects effectively. Code reviews, definite goals and communicating "just enough" were all key. Hrm... how many commercial software developers do the same thing?

It's not without its difficulties, however. Especially with the kernel, contributions can be a big mixed bag. Sifting the good contributions from the pointless ones can be time-consuming and tedious. So projects are split into more minute portions, and newbies are either mentored or given a sandbox to play in.

It seems like there is a role model out there for taking a wide variety of people with an even wider variety of backgrounds and time commitments and having them all contribute to a well-developed end product. It's a shame it isn't emulated more often.

Google Sites = The Reason for a Google Account

The whole reason I wanted to start developing on Google App Engine was because I wanted to start building a repository of code samples, How-To's, documentation, projects, blog posts, all that stuff. Something akin to Confluence. Google already did it for me.

Google just recently launched Google Sites, which basically becomes a content management system for whatever you like. It's exactly what I needed - and I'm planning on moving my docs, code snippets and projects over soon.

Wow... Actual Support! They Know We Exist!

Wanted to buy an album I had just discovered... but didn't want to haul myself to the neighborhood Best Buy. So I wanted to see if the band had an online purchase method. They didn't... but they sold through Amazon. I knew Amazon sold DRM-less MP3's, so I decided to check it out. For whole album downloads you are required to use their lil' download app, so imagine my aghast expression when I saw:

Sweet. Works only on 32-bit Linux (can work on 64 but has problems with library dependencies), but otherwise purchasing was swell. No issues.

Thanks Amazon for realizing Linux users buy music, too!

Spent

A while back I received this e-mail, completely out of the blue:

Wise men say, you only have to resign yourself to what you cannot improve

I'm not sure why I got the e-mail. Or who sent it. No real idea. Could well be spam. But it kinda stuck with me for some odd reason.

I've been working the 80 hour weeks lately, as promised. That means that I've had to give up working on my open-source projects. I feel real pangs of guilt when people ask for bugfixes or when the next release will come out... especially when users as kind as tomasio even volunteer their own time for icon assets. But I've cut sleep down to a few wee hours and just have nothin' left.

I'm hoping I can get everything going at work, get things on a stable foundation, then give myself free time once again. At least, that's the lie I tell myself.

Two Great Tastes - maemo & Qt

Fresh off the wire, it has been announced that Nokia will introduce Qt, my favorite C++ toolkit, to the maemo platform, my favorite portable hardware platform. Two great tastes that go great together.

If this kind of platform expansion and cooperation with Qt developers (such as KDE authors) is what will come of the Nokia acquisition of Trolltech, it may not be as bad as I predicted. Here's to hoping that Nokia sees Qt as a toolkit that will serve mobile, embedded and desktop platforms. Especially with the recent fame of low-cost low-footprint laptops, Qt and maemo have to be getting some additional attention.

If the WiMAX-enabled Nokia 810 becomes more powerful and popular, the addition of Qt and simplified cross-compiling could provide a huge increase of third-party applications hosted on an already open mobile device.

Java 6.9

Someone just directed my attention to the Java 6 update 10 intro on Sun's site. What the living...?

First off, this isn't a minor update. This is taking a backhoe to the foundations of Java, hitting a water line, but digging a basement anyway. Why this wasn't released in 7 I don't know... I guess it's because the update is largely centered around deployment of Java as a platform and not adding any functionality to the underlying API. But damn, it's an overhaul.

First, Java is now chopped neatly into libraries, so you only download what you need. That means Java installations can be one-third of what they were in the previous release. Java can now be downloaded and installed more efficiently as well, thanks to some much easier-to-use JavaScript and HTML-fu.

Konqueror has already done this for a while now, but applets will now execute within a full JVM instead of a half-baked nsplugin. This allows for more robust applets and, from my experience with Konqueror, plugins that are more crash-resistant.

Finally the fairly... blech... look of Java has been completely overhauled with Nibus, long at last. Previously I've had to use javootoo's Look and Feel libraries to make things look remotely presentable. Now Nimbus should be able to fill that gap nicely by adding more modern window decorations and UI components.

These were all desperately needed improvements to have Java make inroads into the desktop space. Let's hope it isn't too late.

Independent Horticulture

Another great invention by the creators of Penny Arcade: Greenhouse.

Steam has done a great job making independent and smaller titles much apparent to the populous, and since titles don't have to compete for shelf space a genre for every palate can be made readily available. And while CodeWeavers has done their best to allow Steam & Source titles to run on Linux & OS X, it can't be said that Steam is a cross-platform solution.

Not so with Greenhouse. It offers native support for OS X, Windows and Linux in tandem. And their inaugural title will be cross-platform. And if they continue to support independent and episodic titles, this could be a bigger competitor to Steam than GameTap.

Introversion's Procedural Art

The huge amount of effort required for content creation was a hot topic a couple of years ago, as many people saw the enormous cadre of artists and animators making AAA titles and realized no garage developers could hope to reach that type of scale. The fear at the time was that this would mean the end of indie development.

Of course after Peggle, Portal and Crayon Physics hit the mainstream it suddenly became apparent bigger doesn't equal better. Or more sales.

I've always loved the approach Introversion has taken with development. They're truly dedicated garage developers, spending more time trying to perfect a fractal tree than they really should. But I can respect spending an inordinate amount of time trying to wrap ones head around a concept like procedurally generated landscapes.

When I heard that Gamasutra was hosting an event with Chris Delay speaking on the topic of procedurally generated content, I definitely wanted to jump on the opportunity. While they had a fairly unrehearsed HP shrill asking the questions, Chris had some great points.

Chris emphasized that the main reason his titles have procedurally generated textures and meshes was because artistic content is just not a space he feels Introversion can compete within, since other companies have mastered that area. He saw it as neither a positive or negative thing, it's just the case for Introversion. Should artists be afraid? Chris doesn't think so. Procedural content cannot replace people, since it ultimately can't produce those unique items that make an environment distinct. While you can generate the landform that the world consists of (mountains, hills, streets, clouds, etc) it cannot add fine-grained details to the world.

You automagically gain several efficiencies with procedural content:

You don't have to re-draw or re-generate a scene if you need to modify level of detail

You end up with a large amount of content and detail that artists can't get (you can delve as deep as you want into a fractal)

If you do procedural animation, you can have adaptive animations that exist as a consequence to a number of actions

While the idea of using fractal algorithms for landscape generation or building trees, I hadn't thought much about procedural animation. Of course Spore uses it for their character builder, but introducing this as a new way of rigging meshes would again immensely help developers. Not needing an entire team of dedicated animators or texture artists would make things much more palatable.

There are tradeoffs of course, and Chris repeatedly mentioned that procedurally generated content requires a different way of thinking about memory management. Rather than loading assets off disk, you load them in memory at runtime - so you don't worry about texture compression, but you do now have to worry about LOD given to your algorithm and how much memory the resulting data structure will reside within. You can't let your procedure go willy-nilly and create too many verts.

Introversion's latest undertaking, Subversion, sounds interesting. Right now Chris describes it as more of a thought experiment, so who knows if we'll actually see it. But what he's pursuing is procedurally generating cities from 10 kilometer view all the way down to pens and desks inside a building. Not only does this employ a landscape generator for hills and mountains, but also will procedurally generate streets and buildings based on markets and traffic demand. Each procedural algorithm feeds its brothers, affecting its ultimate output. For example, more traffic makes more roads which can make bigger buildings.

One difficulty Chris found with this approach was that it was often ard to find out bad results - sometimes you would have cities being built on entirely one side of an area, with another being completely blank. Or sometimes a fire escape would open into nothingness on 30th floor. It's all a matter of finding a way to re-seed or compensate when these failures occur. Or maybe it just makes the whole concept quaint.

Intel Not Killing VPU After All

Looks like Intel isn't killing the VPU after all, but instead birthing it. Larrabee, their GPU/HPC processor, is supposedly an add-in proc slated for 2009/2010. Although I'm going to put myself out on a limb and say it will probably become part-and-parcel of their mainline CPU and, instead of being a discrete co-processor, will quickly be absorbed as additional cores of their consumer processor line. But I digress.

Additional information about Larrabee continues to trickle out, but it definitely seems to introduce vector processing instruction sets to be used by general computing, not just as a GPU.

Even if this comes out as a daughterboard or discrete chipset, it should be a compelling reason to pick up a good assembly programming book and start hacking again. How long will it take (non-Intel) compilers to optimize for the vector instruction sets?

Nifty Nokia

I'm really enjoying the n770. I'm definitely putting an n810 on my wishlist for the end of this year.

First thing I did was re-flash the device with OS 2007 Hacker Edition, an OS intended for the n800 but crammed into n770 hardware. It works rather well, only occasional reboots, but then again I'm working with a heavily used and refurbished unit. Who knows if it's the OS or the device. Google Talk, contacts, Bluetooth, 802.11b/g, a stripped-down mozilla engine and MPEG4 playback all works well.

I turned my lil' Nokia into a pocket translator with the Google Talk translator bot - the streamlined chat interface of OS 2007 turned the Nokia into a very handy (and quick) translation service.

Also tried to crack a test WRT54G router I have laying around using Aircrack, but I couldn't inject wifi packets using the OS 2007 wireless drivers so had to resort to the slower WEP cracking that needs a fair amount of seed traffic. It was still neat to browse all surrounding AP's on a full-screen xterm. With the n770's fantastic resolution, even the small typeface was definitely readable.

Also been mowing through a number of third-party apps. There is a fantastic developer community around the device - their Sourceforge-like approach to the Maemo Garage and the extensbility of the platform has served the developer and user community well.

It took me a while to find out what type of video the n770 will natively accept. There are several good resources out there, such as Andrew Flegg's Perl script that easily transcodes video into a n770-digestible format. The wide screen and nice resolution make mobile video much more palatable. The only caveat was that newer releases of MPlayer tag video with a newer but much less understood "FMP4" codec tag which OS2007HE doesn't understand. I had to tweak the script to pass the value "DX50" to the ffourcc option in order for the built-in media player to recognize the MPEG4 codec used. I also had to make sure encoding only happens at 15 frames per second, otherwise audio quickly gets out of sync.

When I get a free second I'm going to try getting some OpenVPN binaries to work as well. Would be very nifty to have an SSH stack and VPN access wherever I go.

Got Flash 9 somewhat working, although sound doesn't appear to work. Not a deal breaker tho, considering I'm working on a refurbished device running an unsupported OS meant for an entirely different hardware platform.

All in all, I'm a pretty happy gopher. Not sure what that means, but I am.

When $300 Is More Popular than Free

For the past two years digital delivery has supplanted shelf space, but those attached to selling physical inventory have poo-poo'ed the viability of such consumerism. But good ole' Trent may be proving that the merch sells itself once and for all.

The Reg puts it well when it says "Nine Inch Nails cracks net distribution" - their latest album has gone up for sale in several interesting ways on their site: get the first volume (nine tracks) for free. If you like it, you can buy all the volumes lossless (36 tracks) including a 40 page PDF booklet for a measly $5. For only ten stinkin' bucks you can get the whole thing as a two disc CD set with a printed booklet. For $75 you can get the audiophile version, digital versions, Red Book CD versions, hardcover slip case and more. Or you can pay $300 and get a super-mega-uber-limited-edition-collectors pack.

Or at least you could before all 2,500 sold out.

At a time when people keep claiming that pirated music is killing the industry and no one will pay for music anymore, it seems awful incongruous that 2,500 units at $300 a pop sold out in almost a day.

Same thing happened back in the day when I bought a copy of Uplink. I could buy it cheaply on its own or shell out some extra bucks and get the signed "limited edition." Of course I now have a proudly signed copy of Uplink on my shelf.

It's not hard to upsell customers, even (or especially) with digital distribution. Give them schwag and they will come.

A Rite of Passage

I purchased a well used Nokia n770 Web tablet from a friend last month and, as tradition dictates, I must christen the device by authoring an entire blog post using only said device.
It really is a sweet little device... and since it runs a Debian-derived distro I can do pretty much anything I want with it. From checking e-mail to WEP cracking it runs the gambit.
The screen is positively beautiful. Video on this thing makes me giddy. Plus I have more connectivity options than I can shake a stick at.
I can totally understand why the n800 has the following it does now.

Procedurally Generated Pinkslips

Penny Arcade's recent podcast featured a rant - no... more of a reckoning... versus Spore. I find Spore's idea of dynamically generated content interesting, mainly because of my bias towards one-man development teams and procedurally generated content. But Mike and Jerry don't want to see artists and writers out of a job... and the concept that zombie algorithms can build music or images is looked upon with disdain. To them games are an artistic outlet for modelers, musicians and authors. But to developers they can seem like a growing necessity that a garage studio simply can't bankroll.

Eskil Steenberg's Love is described by Rock, Paper, Shotgun as "...lavish impressionistic artwork brought to life... in motion it was suggestive of a smokey, dynamically lit version of Okami." Dynamic terrain deformation and procedurally generated assets allow Eskil to wrap some amazing gameplay into what looks like a surreal and compelling atmosphere.

Not only does this mean that players get to glimpse into chaos, they get to play with it. And anyone who names such an ambitious effort after "For The Love Of Game Development" inspires hope in a lot of indie developers.