Your Barn Door is Closed

I added the security camera functionality to the Raspberry Pi I've been working on, using an inexpensive HP Webcam HD-3110 and the Motion subsystem. Both worked out of the box using the Raspberry Pi and the default (but upgraded to latest) NOOBS installation. Neither had an issue, and I was up and running with a security camera quickly. Another nice thing about Motion is that it exposes several external commands as a means to provide API exposure... so I can integrate a web application or send an e-mail at the start of an event. It would be great to send an e-mail whenever motion is detected, and send the video or photos related to the event.

Once the security camera was working and stable, I added back in the GPIO garage door opener remote that I hacked together. Working together, I now have a security system for the garage that can open and close the garage door from any vantage point. I deployed it up on a shelf, next to the windshield wiper fluid, where the webcam would have a nice perch.

I whipped together a quick Python script that would briefly engage the GPIO pin on the Raspberry Pi, so now I can sit at my laptop and watch the garage door go up and down. Ultimately I will place this feature behind a web application for ease of use. The Python support for Raspberry Pi seems to be a first-class citizen, so I might use Bottle to create a minimalist webapp for exposing the webcam feed and garage door controls.

I still need to build an enclosure... and perhaps upgrade the camera to one without an infrared filter and an array of IR LEDs. The web interface will be the next thing on the docket however.

Your Barn Door is Open

One compulsive behavior I can't seem to shake is the fear that I have left the garage door open. I count the squares on the front, I measure the light and shadows, I obsess. To tell the truth, even if I had a magic device that would send me a message every time the garage door opened or shut, I wouldn't trust it. I have to see it.

To feed my neurosis, I began working on a combo surveillance camera / garage door controller with an Internet interface. The first thing I attempted to get working was the method of opening the garage door; up to this point it seems like most efforts have been using a relay to close the switch of the hard-wired "big button" for the garage door opener. I wanted to have the freedom to place the device anywhere I wish, not unlike the garage door keypad that comes with most openers now. A spare Chamberlain universal remote was just sitting in my garage, so I decided to shuck it and rip out the logic board within.

The board actually had a fairly nice layout and a hidden third button. I only needed one button... and a way to trigger it without a physical press. Using a complicated tool called a "screwdriver-like thingee" I pried the physical button off, leaving nothing but the leads behind.

Instead of sharp metal prongs, I got the ole' trusty soldering iron out and melted the solder enough to pull the leads out. I replaced two of the leads with pins for easy breadboarding.

I have a weird affinity towards MOSFETs. I'm not entirely sure why. I have a feeling that my affinity to using a MOSFET instead of a relay is similar to most people's opinions on using an Arduino instead of a Raspberry Pi. With such snobbery in mind, I prototyped out a circuit that would use an N-channel MOSFET to close the circuit instead of a switch.

Bear in mind I paid no attention in 6.002x, so my circuit has less to do with elegance and common sense as it did with "let's slap together a MOSFET and pull down resistor." Still, sending the 2.2V from a GPIO pin into the MOSFET's gate pin allows the circuit to close on the garage door opener, sending the door up and down.

Next up is adding a surveillance to the Raspberry Pi using a USB camera I found lying around. The plan is to try out the Motion subsystem, then adapt things as needed. One particular enhancement I'm keeping my eye on is the Raspberry Pi Infrared Camera, which I might pair with a set of IR LED's and a fisheye lens. Doing so may require some patching of Motion, but should otherwise be straightforward.

My Electric Slide

I love Hack a Day. I've never actually constructed a featured project, but the site has always been a fantastic read to see what's top on the minds of the hacker front. One particular Hack a Day review caught my eye - "a light following bristlebot as a way to teach science." What I found particularly interesting was how well the build instructions were documented on the Science Buddies site. The background was descriptive without being verbose, the parts were easy to come by from both Fry's & SparkFun, and the component count was nice 'n' low. I wasn't sure if the kids would enjoy it, but I knew that at least I would.

Now bear in mind I attempted to take MIT's 6.002x class before my brain took an enormous sidetrack and instead ended up with a comparative analysis with Stanford's Coursera. While I did find out what a high-voltage pickle looked like, my understanding of Thevenin voltage is limited to indecipherable algebra illustrating the "Thin Mint voltage" of a circuit. I generally don't know what I'm doing, and exploding diodes are not a rare occurrence. It took me considerable research before coming to understand what the three pins of a MOSFET are responsible for. To this day I cannot quite grasp why a pull-down resistor is required for normal operation (even when there is no "bad wiring"). I need simple, otherwise something is going to end up charred and melted.

The project went much better than expected. I performed some initial prototyping to make sure it was easy to teach the kids, and things went surprisingly well out of the gate. We ended up sticking with the project far longer than I expected, and even added our own enhancements to boot. We're now on build #4 and going strong. Directional control is actually quite good, and there was enough room for minor tweaks and improvements on the breadboard.

Conveniently enough, when I placed my initial SparkFun order I also slipped in a Raspberry Pi - Model B as well. In the spirit of teaching electronics while remaining completely selfish, I justified the purchase by teaching Python programming using Minecraft while experimenting with GPIO programming on the sly. I was able to have moderate success teaching Python; I constructed a small lesson plan using Martin O'Hanlon's API tutorial as a reference. That turned into a good weekend software development lesson, however I didn't get as much traction with that project as I did with the bristlebots. Luckily for my ulterior motives, I now had a general purpose postcard computer for more breadboard projects. Several tiny wires later, I was able to get a Python script to blink an LED on a breadboard. Between vibrating toothbrush heads and a $40 miniscule flashing light, I started to brainstorm bigger things.

Now I'm getting back into 6.002x - Circuits & Electronics and starting over again. I'm much more engaged given this new context - and will likely take the class MUCH slower than the real-time course would otherwise allow. Should be fun.

The Standing Sloth

I returned to work after Thanksgiving last year, sat down at my desk and felt... absolutely gross. There was stuffing in my veins, broccoli cheese casserole in my gut and I could feel my flesh slowly molding around my office chair. The previous summer the Internets were all the rage about the famous infographic detailing how "Sitting is Killing You," and sites such as Lifehacker were consistently running stories of people constructing their own standing desks. I decided enough was enough and decided to try the standing desk thing.

The desk I was using at the time was huge, solid and immobile - a single and secured wooden boat in a 10' by 6' L-shape. There was no freaking way I could elevate it to standing height. At first I thought I would just build something roughly hewn from 2x4's but instead I noticed a blocky, simple accent table at Target. Worst case I thought I could saw off the legs to height.

In addition I had found an older storage ottoman sitting in my garage with a half-broken lid. It was quite nearly the same height as the two accent tables, so I thought I might see if it was of any use as well.

I placed both tables and the ottoman side-by-side on top of the desk. To my surprise the tables were the perfect height for a keyboard, mouse, monitor and a laptop riser. My arms were bent at a 90 degree angle, the top of my monitors were at eye level and I could shift my weight around to avoid locking my knees. For the first two weeks I alternated between sitting and standing... but soon I was standing 6 hours a day.

Here we are, over a year later. The only alteration I have made has been a comfort floor mat to stand on, and then only because I moved to a cement floor. I must admit that people mocked me for a good while... people kept coming into my office asking "what are you doing? Do you really stand all day? Really? REALLY?" And yes, co-workers sitting adjacent to me were treated to being eye-level with my butt all day.

People eventually got accustomed to me standing and even became interested in converting to a standing desk as well. A nice side effect is that it is easier to have conversations with people who walk over... no longer do I have to stare up their nose while someone asks questions from above. I lost (and kept off) ten pounds, which is nice. No more lower back problems either, which may have less to do with standing and more to do with me sitting like an crooked monkey.

Not bad for around $100, half of which was a nice standing mat. All I needed was busted furniture from home, tables made for a college dorm room and a laptop stand.

Re-Examining Development Platforms

I've been granted an opportunity for perspective lately. I've done my best to steer away from .NET development up until recently - not because I had any particular gripes, but the technology platform just never seemed to be a great fit. C# as a language is pretty great... delegates, tail closure and nullable references are a welcome respite. VisualStudio isn't bad. If you judge IDEs by how many times they make you swear in a given workday, I'd say my four-letter word tally is comparable to that of me using Eclipse. In fact, once my development environment was up and running I thought I might grow to enjoy .NET development.

And then I started to use the foundation classes. First off... I'll acknowledge that no platform has ever been able to get dates and times "right." This is ever more apparent with .NET... who for some inexplicable reason have no real sense of this "epoch" thing that EVERY OTHER PLATFORM USES. Milliseconds since year zero are expressed as... integers? On top of that, time span arithmetic is only accurate when math is done using "ticks," which themselves are not really accurate to 1/10000 of a millisecond. I didn't go for the caesium clock upgrade in my current laptop.

Java developers often complain that anonymous classes are inelegant or verbose within Java. C# has abandoned anonymous classes in lieu of their delegate-based event handling system. However, C# developers exposed to anonymous inner classes actually seem to like them. A common gripe actually turns out to be a nifty feature when you're talking about event handling. The C#/.NET event handling mechanism isn't that fantastic... it's largely just a loose convention for using delegates. No extras or nice GoF listener patterns provided like a PropertyChangeListener.

Zooming out from design patterns and looking at .NET from an enterprise integration pattern perspective, the .NET platform is definitely at a major disadvantage when compared to JVM-based platforms. I've already covered the state of .NET integration frameworks but to recap: it's still nearly five years behind. My time with EasyNetQ has been great, but I still find myself wishing I could use Apache Camel to construct bigger things using common EIP components.

Despite the current rant, I've been fairly complacent with my new development platform. What really stirred things back up for me was when I cracked open the JMeter-Rabbit-AMQP plugin so that I could do RPC-based load testing of EasyNetQ services. Being a JMeter extension, JMeter-Rabbit-AMQP was a Java app that required me to fire up NetBeans once again and do some Java hacking. Once I did... damn. Until I made that sudden switch back I didn't see the huge gap that existed between .NET and Java development. While C# has some advantage over the Java language, the JVM platform is still leaps ahead.

Once you begin talking about instrumentation the gap grows even wider. I have grown accustomed to the fantastic introspection and profiling offered by Java Management Extensions and VisualVM; by contrast Microsoft's laughable implementation of Performance Counters has caused me more problems than it has solved. If it works (and it often doesn't due to permissions issues or outright registry corruption) there is no instrumentation that allows for live modification of managed objects or details on garbage collection. The actual .NET API to create and maintain performance counters is actually not bad, but the Performance Counter UI is so clunky and ill-conceived that it is often difficult to make use of it.

In the end... it doesn't matter. You do the best you can with the tenured development stack because ultimately it's not about the underlying technology - it's about the squishy, business-logicy brain inside of it. Keeping that squishy brain... err... braining is the most important thing.

Enjoying Walled Gardens

For the fourteen years prior prior to last June I had consistently used some variation of a Dell laptop with Linux - initially RedHat but quickly switching over to SuSE. Occasionally I would dual-boot into Windows to do some gaming, but all of my work and development was done within KDE. I really had no desire to change, even when I was lugging a 12-pound behemoth Precision laptop with openSUSE and Windows 7. Muscle strain aside, I was content with the setup.

The next job I moved to was going to be a switch from my conventional Java development to .NET, so I decided to take the opportunity to change up my workstation as well. I moved away from Wintendo/Linux and moved wholesale into OS X, since my new position delightfully coincided with the release of the latest MacBook Pro’s. Gone was my comfortable NetBeans environment, and instead I had to use VisualStudio within Parallels. No more Linux... and at the time I was confident I would hate OS X and re-build the laptop with KDE 4.

That was eight months ago.

The .NET ecosystem hasn't been fantastic... immersion has not worked in its favor there. The move to OS X has been considerably more pleasant. By and large OS X is still *nix at its heart and ports abound. Even my favorite photo management / digital darkrooom Digikam works within OS X. I haven't really been missing any apps as of yet other than Pidgin. Yeah, there's Adium... but... meh.

The ease of use is much greater as well. The application-based firewall works well, instant messaging integration works (albeit without OTR support), calendaring and mail integration works without issue. By and large things work without any futzing. This is doubly true with the rest of the family; while my attempts to get the household at large on openSUSE + KDE 4 failed miserably, the hand-me-down OS X workstations we've been reusing have been adopted with great enthusiasm. All walks of life have been happy to use 4+ year old iMacs with Snow Leopard, no complaints.

Apple's lustre has been slowly seeping away with the stylish kids of today, but it's hard to deny they've built a solid platform. Yes, it does this by sacrificing your freedom of choice and reducing your hardware upgrade paths. But for now... the walled garden is a damn nice place to just chill out and get some work done.

The State of .NET Integration Frameworks

I have been increasingly working with teams that are largely operating within the .NET platform, so I’ve needed to abandon my old favorites in the Scala / Java / Python world. An interesting thing that strikes me (and many others I’ve discussed the topic with) about the .NET ecosystem is how there are no good analogs for .NET frameworks to Java or Python frameworks. Yes, there are projects such as NHibernate, Log4Net or even Spring.NET - but these often are exercises in trying to shave a square peg down so it can fit into a round hole. Even SpringSource seems to be pulling back from .NET support, as its support has becoming increasingly shallow when compared to Java.

Gone are the days of relying on integration frameworks such as Apache Camel. Instead I needed to survey the .NET landscape for integration frameworks that would allow for the development of loosely-coupled service components that can be horizontally scaled and messages intelligently routed. While there were a few frameworks that helped with RPC-style message passing to private queues, not a whole lot implemented the whole EIP enchilada.

Following is a brief survey of what I found for .NET frameworks or libraries that can help build a distributed message-based architecture. I would love any feedback - so much of this is subjective, and I’m sure I’m still only scratching the surface. I tried to weigh the weaknesses and strengths of each framework to see which would best provide implementations of Enterprise Integration Patterns.

Windows Communication Foundation

Windows Communication Foundation (WCF) was released by Microsoft as a way to unify different ways the .NET framework communicated with other services. WCF abstracts SOAP and REST API calls, queued messaging, .NET remoting and other technologies under a single framework for service-oriented development.

Strengths

WCF is integrated natively within .NET 3.5 and beyond as a supported tenant of the Microsoft .NET framework. Since WCF is a first-party solution it enjoys the full support of Microsoft, receives regular updates and is in tune with the .NET release cycle.

WCF is a straight-forward integration framework that provides a familiar interface for remote connectivity using well-known messages. Its endpoint design is analogous to the messaging endpoints defined by Enterprise Integration Patterns (EIP), and it appears message construction follows EIP as well.

Weaknesses

While WCF provides a working implementation of EIP messaging endpoints and message construction, it does not necessarily provide concrete implementations of the message routing, messaging channels or message transformation. Instead WCF provides bindings to other routing and channel implementations such as MSMQ, datagram sockets, REST and SOAP.

NIntegrate

NIntegrate appears to have been discontinued and is no longer eligible for evaluation.

NServiceBus

NServiceBus is a nearly decade-old SOA framework built to a custom set of distributed service patterns. In its own words: "If you design your system according to the principles laid out [as part of its design patterns], NServiceBus will make your life a lot easier. On the other hand, if you do not follow these principles, NServiceBus will probably make them harder."

The principles of NServiceBus are largely aligned with those of asynchronous messaging, which is referred to as the "one-way messaging" pattern. It also offers several stand-alone helper processes for distributing tasks, managing timeouts, and providing service gateways.

Strengths

NServiceBus does not appear to be as complex as an Enterprise Service Bus, but is more robust than WCF. While WCF does not provide a queue-based message bus by default, NServiceBus does dictate that queued messages be used.

Both Publish/Subscribe as well as Request/Reply messaging is supported by NServiceBus, which is another strength over WCF’s focus on remote process invocation. In addition NServiceBus appears to provide transaction support and durable messaging for guaranteed message exchanges. Transactional and durable messages are persisted via RavenDB (an ACID-compliant document database).

Messages within NServiceBus are well-defined constructs that include headers for routing message exchanges. Marshalling is provided by NServiceBus to automatically convert objects into documents for submission to message queues.

[update 2013-08-16: Udi Dahan has brought to my attention that, at the time of this blog post, NServiceBus also supported sagas as a way for workflows to be defined as long-lived transactions. Unit testing libraries were also provided. Both appear to have later inspired similar implementations within MassTransit.]

Weaknesses

NServiceBus is strongly coupled to its underlying implementation. MSMQ is currently the only supported messaging channel, albeit with a good deal of custom bindings for transaction support, durable messaging, quality of service and publish/subscribe support. Support for other channels such as AMQP has been considered as part of NServiceBus’ roadmap, but has not yet been implemented.

Durable messaging is only provided via RavenDB, which is a freely available open source project. A RavenDB instance must be separately maintained and managed.

MassTransit

MassTransit is "a lean service bus implementation for .NET," not unlike NServiceBus. Instead of working on top of WCF, MassTransit has created an endpoint and message abstraction of its own. In doing so, MassTransit allows for a wider breadth of implementations for messaging channels and document marshalling.

MassTransit message handling and channels are defined using Dependency Injection and Inversion of Control, which allows for better protoyping and unit testing of components.

Strengths

One of MassTransit’s primary strengths is its flexible channel architecture. Implementations can leverage MSMQ, ActiveMQ, RabbitMQ and TIBCO for message channels and JSON, XML or other document types for marshalling. MassTransit also supports a wide variety of IoC containers (or no container at all, in theory).

Messages can be simple POCOs without annotation or implementation of another interface. Likewise message handlers (i.e. message consumers) can be simple classes that require a minimum of instrumentation.

Transactional and persistent messaging is supported. This also has multiple methods of implementation depending on preference.

MassTransit has an interesting implementation of .NET state machines for managing workflow and transactional messaging, called “sagas.” Sagas provide a way for workflows to be succinctly defined and efficiently process long-lived transactions. This could possibly be an efficient way to implement an Aggregator or Scatter-Gather Enterprise Integration Pattern.

So that developers can more easily create unit tests over consumers and handlers, MassTransit offers a testing library that more easily allows a local message bus to be constructed within the context of a given unit test. This greatly eases the development of unit tests by providing a supported framework for local unit testing.

Weaknesses

The MassTransit project itself can give the appearance of not being well maintained and entirely documented. The project has shifted from Google Code to GitHub while documentation is hosted within a stand-alone site. On occasion one will find dead links, outdated examples and incomplete documentation.

The AMQP routing implementation (the wire protocol also used by RabbitMQ) used by MassTransit is sub-par. The documentation indicates that routing keys are often lost from the message headers, and it appears only fanout exchanges are supported. MassTransit’s routing engine appears to compensate for this, however this obviates the efficiencies often gained when using AMQP bindings. In particular this hurts interoperability, since reply queues can be difficult to create based upon a fanout exchange. While MassTransit can use AMQP as a transport layer, it does not fully take advantage of AMQP’s capabilities.

Concurrent consumption is determined based on the entire message bus, not on each message consumer. This does not allow developers or operations to throttle the precise number of concurrently running threads permitted to respond to incoming messages. These sorts of quality of service levels are determined for the entire message bus.

Rhino Service Bus

The authors of Rhino Service Bus are the organization Hibernating Rhinos, who are also responsible for the document store RavenDB. There are many similarities between NServiceBus and Rhino Service Bus, most notably the emphasis on "one-way" asynchronous messaging. Also not unlike NServiceBus the primary queueing mechanism is MSMQ, however Hibernating Rhinos’ own Rhino Queues service is also supported for durable and transactional messages.

Rhino Service Bus was designed to be architecturally very much like NServiceBus and MassTransit, but more lightweight and with a slightly different design for private subscription storage.

Strengths

Rhino Service Bus implements both request/reply as well as publish/subscribe message exchange patterns using "one-way" asynchronous messaging. Message handling within Rhino Service Bus is very succinct; processing an inbound message requires only the implementation of an interface with a known message type.

While most integration frameworks in this survey appear to favor programmatic configuration by defining configuration variables in code, Rhino Service Bus defines settings with an external configuration file. In environments already accustomed to using external configuration files as part of their build process, this can reduce implementation cost.

Since the Rhino Service Bus’ persistent messaging schemes are directly integrated with Rhino Queues, it appears that installation, maintenance and patching will be more straight-forward than with NServiceBus.

The method for the selective consumption of messages is interesting - Rhino Service Bus uses generics to match endpoints with inbound messages. Only inbound messages of a declared type will be dispatched to a local endpoint.

Weaknesses

Rhino Servie Bus can only use either MSMQ or Rhino Queues as the message channel. More advanced brokers such as RabbitMQ are currently not supported, and there appears to be no proposed support on Hibernating Rhinos’ roadmap.

Once again, Rhino Service Bus configuration is XML based. While this is also cited as a strength, I know several engineers who instead consider this a liability. Personal preference on behalf of many developers seems to favor programmatic configuration over static configuration files.

Rhino Service Bus appears to have limited quality of service properties, likely due to an emphasis on transactions and stateful messaging via MSMQ. The service bus itself seems to have a central focus of maintaining stateful messaging, as opposed to routing stateless exchanges.

Spring.NET Integration

The Spring framework was one of the premier IoC frameworks for the Java Platform, and the SpringSource team has released a similar IoC framework for the .NET platform as well. Current incarnations of Spring (including those on the .NET platform) span beyond just depdency injection; it also includes data abstraction layers, messaging frameworks and service integration components.

A large initiative released by the SpringSource team in 2008 was Spring Integration, an integration framework that was meant to simplify the development of service oriented application development. Just as the initial Spring container was meant to be a concrete implementation of the "Gang of Four" (GoF) Programming Design Patterns, Spring Integration was meant to be a strict implementation of Gregor Hohpe’s Enterprise Integration Patterns.

Strengths

Adherence to the Enterprise Integration Patterns is one of Spring Integration’s primary strengths. An integration framework with a focus on pattern-driven development helps designers and engineers develop applications with well-defined solutions to problems. Spring Integration relies heavily on the Spring IoC framework, which also enforces GoF Design Patterns. By matching both of these well-proven enterprise design patterns developer productivity can be increased and code can be more easily tested.

Spring Integration offers an abstraction of message routing and message channels that allows the integration of best-of-breed implementations to more easily be performed. Broker implementations can range from AMQP, JMS, flat files, peer-to-peer, etc.

Weaknesses

Currently Spring Integration for .NET is in “incubator” stage within SpringSource. This can be considered to an open beta release of a project, where development is heavily underway and the given framework is subject to change at any time. Judging by the activity stream on the project’s issue tracker, it appears that there is currently only one engineer assigned to the project who last committed changes in March of 2010.

Since Spring Integration is closely tied to the Spring Framework, no alternate dependency injection containers are supported. While many integration frameworks surveyed rely on Castle Windsor, Spring Integration instead relies on the Spring IoC framework.

Spring.NET AMQP

The Spring framework was one of the premier IoC frameworks for the Java Platform, and the SpringSource team has released a similar IoC framework for the .NET platform as well. Current incarnations of Spring (including those on the .NET platform) span beyond just depdency injection; it also includes data abstraction layers, messaging frameworks and service integration components.

Spring.NET AMQP is not an integration framework, but instead provides "templates" as a high-level abstraction for messaging. While this doesn’t provide message routing directly, Spring.NET AMQP allows developers to establish routing keys and bindings that permit an AMQP broker to properly route messages.

The scope of Spring.NET’s abilities is roughly analogous to that of Microsoft’s WCF, with the addition of dependency injection and inversion of control containers.

Strengths

Within the Spring.NET AMQP project, adherence to the AMQP feature set is the primary focus of SpringSource. This allows engineers to fully leverage brokers such as RabbitMQ and perform topic exchanges, content-based routing and multicasting.

Weaknesses

Spring.NET AMQP is a lightweight templating library which does not provide any adherence to Enterprise Integration Patterns. Routing and message transformation is not supplied by the framework, however it can more easily be implemented by the developer.

EasyNetQ

When users encounter difficulties when integrating .NET integration frameworks, a common alternative cited is EasyNetQ. While not necessarily an integration framework, EasyNetQ allows for messages to be more easily routed via RabbitMQ, not unlike the Spring AMQP project.

The EasyNetQ was inspired by MassTransit and created for use by the airline travel company 15below. The primary goals of the project are maintaining a very simple API and requiring minimal (if not zero) configuration. Code required for requests, replies and subscriptions are fairly minimal.

Strengths

EasyNetQ was created from the ground-up to take advantage of the AMQP specification’s message routing capabilities, and attempts to fully leverage the routing capabilities inherent to AMQP brokers. By adhering to the AMQP standard and expected behavior, interoperability between .NET and other platforms becomes an easier task. To accomplish the goal of zero or little configuration required to start using EasyNetQ, the framework accomplishes most setup tasks by convention. This allows for safe defaults to be easily chosen and reduced the amount of code that needs to be written.

Just as MassTransit implements "sagas" for long-running business processes, EasyNetQ provides a saga framework for workflows to be succinctly defined and efficiently process long-lived transactions. In a very similar way the EasyNetQ saga support could be an efficient way to implement an Aggregator or Scatter-Gather Enterprise Integration Pattern.

Connection management to the message broker has been implemented in a fail-safe way within EasyNetQ. Broker connections are performed using a "lazy connection" approach which by default assumes the broker will not be available at all times. If a broker is not available, messages are queued locally and the broker is polled until it comes online. Once the message broker is available, messages resume transmission. Bear in mind publishers of message are not aware of this outage; they continue without halting. This works in contrast to most other integration frameworks which throw exceptions or halt when the broker goes offline.

Weaknesses

Many integration frameworks under review directly integrate with another dependency injection framework such as Castle Windsor, however EasyNetQ does not have a direct integration with any such dependency injection framework. This has the benefit of requiring less dependencies, however it also makes the management of endpoint objects a bit more convoluted. This is not to say that a dependency injection framework cannot be used - indeed a IoC container can be very easily used alongside EasyNetQ. Other integration frameworks, however, effectively use dependency injection frameworks to auto-discover endpoints by type and greatly simplify the registration of message consumers. Such auto-discovery is not immediately available from EasyNetQ libraries.

Limiting the number of concurrent consumers is currently not something readily supported by EasyNetQ. While most integration frameworks allow you to rate limit consumption based on the number of concurrent threads, EasyNetQ does not currently offer this as a configurable property. Instead EasyNetQ provides the scaffolding for asynchronous responses backed by a blocking collection. Worker threads are added to the blocking collection, which them accept each request as they become available.

Time to Live settings are not easily accessible within the EasyNetQ framework, however facilities do exist to set message expiration settings directly within AMQP. Since this facility is not readily available, it makes quality of service a bit more difficult to establish for messages.

EasyNetQ does not offer a testing framework to assist with the construction of unit tests. Instead it is up to the developer to construct a mock message bus with which to test message production and consumption. This may not necessarily make the development of unit tests more difficult however, since endpoints are much more simply defined than with other frameworks.

Creating A New, Internal Framework

Creating a new integration framework is often regarded as a universally poor idea among developers. The YAGNI development assistant was created explicitly for monitoring an engineer’s behavior within an IDE and preventing the independent development of yet another framework for features "that aren’t necessary at the moment, but might be in the future."

If proper alternatives do not seem to exist as integration frameworks, it may be necessary to evaluate creating a new, first-party integration framework. If the sponsoring organization is committed to contributing to the open-source community, leveraging the community at large to develop an inter-operable solution may provide a unique framework that fills a need within the .NET ecosystem.

Can't Quit Qt

I’ve been dorking around with DeskBlocks lately, trying to find a way to get things to build & run within OS X. My ODE references were already many versions behind and the physics codebase is in need of an overhaul. Looming overhead was also the issue that Nokia was not entirely dedicated to the Qt4 platform that DeskBlocks is built upon, so I started to debate if I should find a new framework to build DeskBlocks within. I was pretty down on Nokia’s acquisition of Trolltech back in January of 2008, although it kinda made sense for the maemo platform. Still, with Nokia going all-in for Windows Phone it seemed like maemo and Qt was heading for the dustbin.

Sure enough, last week Nokia announced it was selling off Qt to Digia Oyj, even reportedly taking a significant loss in the process. Digia had already been running Qt licensing since 2011, so it’s not a huge surprise for it to take over the whole kit and kaboodle.

I’m of the mindset that this could potentially be a very good thing for Qt. I’ve seen Candyland get paved over so many times in the past decade, with big industries wrecking my favorite technologies over and over again. Going smaller may make things more agile - and it seems like many in the Linux community might agree. In a very smart move, Digia has already written an open letter to the KDE community emphasizing their commitment to Qt going forward. If they actually follow through on this dedication to the ecosystem, this could be a huge win for Digia Oyj.

Finland wins yet again.

Tired of The Good Old Desktop

It seems like people are getting tired of the good ol' "desktop" user interface we've all grown accustomed to within our window managers. A generation that has grown up with Windows, MacOS and X11 has seen the desktop metaphor used and abused over the past fourty years, and it seems like developers and user experience engineers are bent on shifting the dominant paradigm. Traditional users have grown accustomed and comfortable however, making the user experience shifting without a clutch. Instead, the desktop OS is forcing the change instead of building concensus.

Last year John Cook had a great blog post about why forcing users to change against there will may actually be a good thing for everyone. John cites a fantastic example - Microsoft's infamous "ribbon" toolbar. Microsoft Office introduced a new user experience element where menu and tool bars changed based on the document's context, causing elements to magically appear and disappear based on what Office thought they were going to do given the user's previous actions. This was a disruptive change that made many long-time users (including myself) particularly incensed. The resulting usability metrics have proven Microsoft correct however - especially given that prior to the ribbon "90% of the feature requests the Office team received were for features that Office already supported." After the ribbon was forced upon users they began using over four times as many features as they did pre-ribbon. The new user experience helped users discover features that they never knew existed.

This is happening on a more global level now that designers are bucking against the Xerox PARC GUI of old and abandoning the desktop metaphor. Apple's iOS and OS X could be considered the first shot across the bow to re-do the personal computer's user interface, although this wasn't so much forcing users into a new paradigm as it was creating a whole new product line. The first "forced" modern redesign might be considered KDE 4 - and people indeed got maaaaaaaaaaaaad. As soon as 2008 hit KDE 4 was released early, even though the development team publicly admitted that it might still be a bit immature. However the desktop concepts had radically changed and dependent applications needed to be re-written... and it seemed like the best way to get this to happen was to discontinue 3.5 development entirely and publicly release 4.0. The Plasma Desktop and the new widget-based design was something that caused major hiccups with hardware support, driver bugs and subordinant applications. History seems to have justified this change however, and now KDE 4 is a fantastic desktop interface.

Gnome 3 has made radical changes as well and is seeing similar community push-back, just as KDE 4 received upon release. They will likely have the same path however, and gradual acceptance and re-thinking the desktop interface will eventually take root and help users become more productive. I recently installed Ubuntu's take on Gnome 3 with Unity and was disappointed to see a number of features unceremoniously dropped. Even choosing a screensaver doesn't exist as an option. However - despite my own personal bias - the UI worked astonishingly well for a grade-schooler's laptop. I've tried openSUSE a number of times and kids just don't like it... but Unity was a hit. It was organized just as their minds expected it, especially with regards to instant messaging and e-mail.

Many people are now decrying Windows 8's Metro user experience, with its touch-based gridbag layout and full-on contextual menus. I'm guessing Metro will eventually win similar acceptance, although it will cheese off a fantastic number of users in the process.

I've been pretty psyched about KDE 4 and openSUSE as of late but I do have to admit... things are starting to get a lil' shakey. Bear in mind I HAVE NO RIGHT TO COMPLAIN since I haven't debugged a thing or filed a single bug report, but it feels like the gears are getting stripped a bit. Evolution has started to freak out with Exchange servers (and even IMAP), and calendar events don't sync across platforms. KDE 4 is starting to hit odd mutex hiccups where nothing happens (e.g. no applications launch) and suddenly they all launch at once. Sometimes it seems like kio is smacking the entire desktop around. All in all I do love working within Linux and KDE 4 - it has made me far more productive of an engineer. Issues with package building and distribution, however, are cause for concern that the platform can remain nimble enough. This issue considered alongside Attachmate's acquisition of the SuSE platform gives one pause.

Recently, and for the first time ever, I bought a Mac. Never before have I owned or operated an Apple desktop/laptop but I needed to legitimately develop within xcode, which requires OS X, which of course requires Apple hardware. I'm not a fan of paying an "Apple tax" to write software, but when my time to purchase coincided with Apple's refresh of the MacBook Pro line I decided that if I was ever entertaining trying the Mac platform, now was the time. Well, at least until the next refresh.

Whenever I jump around platforms I try to force myself to use it exclusively to do productive work. No jumping back and forth based on the task... force yourself to really work around those odd 10% problems that often plague your user experience. See how well edge cases are handled. Try to bend your mind around the nuances of the user experience. When the MacBook Pro arrived I tried to switch all of my work from openSUSE 12.1 to OS X Lion and move all my correspondance and communication as well. I am going to force myself to bend my brain around a new way of doing stuff... and hopefully I'll march past my frustration and objectively see if I'm more productive or not.

NVIDIA's Linux Fate

Just like everyone else on the globe, I heard/read/watched Linus telling NVIDIA, in absolutely no uncertain terms, that they were bad Linux citizens. At first I thought Linus was telling them that they were "number one!!!" but sadly this turned out not to be the case.

The topic came up from someone in the audience who was trying to get NVIDIA's Optimus tech working on a Linux notebook. This is a bit of a different bird than NVIDIA's usual discrete GPU market, as this requires co-operation between onboard (or on-die) video adapters, the discrete graphics card and the northbridge to turn off the accelerated GPU when not in use. To my knowledge only Windows 7 drivers can accomplish this, and Linux doesn't quite understand the management architecture to leverage it. The open-source Bumblebee Project appears to be doing a great job of trying to set ACPI & BIOS flags to accomplish the same thing, but the entire effort could be greatly accelerated with NVIDIA's help. While NVIDIA has more-or-less sanctioned the Bumblebee project by releasing installers that work with Bumblebee, they haven't contributed to it directly.

Linux called NVIDIA's lack of Linux support the "exception rather than the rule," and extended that comment to mention that "NVIDIA been one of the worst trouble spots" in kernel support and the "single worst company we've ever dealt with." This of course was punctuated with the infamous line: "so NVIDIA, f**k you." He didn't actually speak in astrisks, I'm just experimenting with censorship.

Linus, of course, just "like[s] being outrageous at times... [because] people who get offended should be offended." It was probably meant to be a thrown gauntlet more than a haymaker, and I can definitely see why. I've been a big advocate of NVIDIA, even with their proprietary stance on GPU drivers and increasingly as the nouveau driver has increased its feature parity. These kind of ACPI hacks however are not the same as keeping tight control over intellectual property tied to their GPU hardware. Given how good drivers are the true value behind discrete graphics cards, I understand not wanting to open-source them to competitors ("Hey AMD! Here's how we did tessalation!"). Controlling power management within the OS kernel is a different matter however - and everyone benefits (including NVIDIA) with broader support. What's the barrier to entry of AMD creating something similar that idled the discrete GPU when no accelerated calls were being made? For that matter, what's to stop Microsoft from coming up with a vendor-agnostic solution that does the same thing? It seems not unlike what OS X Lion is doing right now...

NVIDIA's response basically focused on the fact that they permit Bumblebee to co-exist, their proprietary GPU drivers still offer fantastic performance and support (which they do) and they are one of the largest contributors to Linux on ARM. In short: they don't feel like Optimus Linux support is worth doing right now. However there are signs that even this isn't working well - NVIDIA just lost a quarter million dollar GPU deal due to poor MIPS support.

NVIDIA, this is a great time to step up and contribute the solution to the Linux kernel. Show your competitive edge continues to be broad platform support, otherwise people may meander elsewhere.

Post script: If you're interested in getting a sense of how crazy GPU/graphics driver development is on Linux, read Jasper St. Pierre's post on The Linux Graphics Stack. Terrific outline on the steps a polygon goes through to get to your monitor.

Contrasting MIT's MITx with Stanford's Coursera

I've been really interested in the user experience of highly interactive sites - webapps where the user must interact directly with the site and stay within it for a good chunk of time. Courseware sites are a great example of such user experiences - web applications that engage students in interactive learning. Some big examples have launched within the past few years; Stanford University began providing open access to not only course materials but actually began to engage the public at large with interactive courses offered entirely online through Coursera. This year MIT has followed suit by creating MITx - and they upped the ante not only in student interaction but in how much content was released to the public. I enrolled in both MIT's 6.002x and Stanford's Game Theory class and gave them a spin for a week.

Screenshot of MITx 6.002x Courseware

MIT's 6.002x has been far more intense in comparison to the pace that Stanford's Coursera classes usually take. The class asks for 10 hours a week for study, lectures, exercises, labs, homework assignments and exams. Some students report that 40 minutes a day is sufficient to get through the lectures and exercises, however there are a fair number who are taking the full two hours a day.

Piotr Mitros was introduced as the lead software designer for MITx, and the user experience provided within the site really shines. The rather voluminous textbook is fully available within the site (apparently rendered as an HTML 5 canvas), and renders beautifully on a laptop as well as tables such as the Kindle Fire. In fact, the textbook was actually easier to read on a Kindle Fire than Amazon's own e-books. Lectures are interspersed with interactive exercises that ask you to submit answers to key concepts presented throughout the hour-long video series.

Both quizzes, homeworks and exercises are presented as forms submitted to the site, validated in JavaScript. There appears to be a rather nice algebraic interpreter behind the courses, as it takes a flexible set of inputs (e.g. V1, 1/3, 0.33333, 0.33) and evaluates them to a uniform solution solved within x decimal places. At times it refuses to acknowledge parenthesis or variables and throws syntax or evaluation exceptions, but for the most part it works surprisingly well.

Learning is provided through a number of facets. "Tutorials" are given in laboratory format, where one of the MIT professors walks through a live-action example of things such as the KCL rule or Ohm's Law. This hands-on style serves to underscore the series of lectures, given two per week, in a format that mirrors a classroom. Unlike the classroom however, you must respond to the open questions the prof asks of the class. A video lecture segment may proceed for 90 seconds and then halt until you respond to an open question that builds upon preceding concepts. The web application itself was built to have a natural flow of textbook -> lecture -> examples, however often links for the text pointed to a wildly incorrect chapter. Links are also provided to the open (albeit loosely moderated) discussion forum where students posit solutions and questions amongst themselves.

For as many ways to learn the material as MITx offers, it is often difficult to navigate the course itself. I was often lost trying to understand the sequence professors wished us to follow - should we read Chapter 2 first, then the lectures, then the labs? Often I would be deep in a lecture series, get completely lost and only later find we were halfway through a chapter within the text. I didn't even discover the importance of the poorly named "tutorials" (they're more akin to lab lectures) until very late in the game. There were also several algebraic errors made throughout the lecture and even within the text... and for someone such as myself who already had a fragile grasp of the subject matter, it could get frustrating to find out the error only later in the discussion forums.

The MITx platform is amazing - I can easily see it becoming the standard for online courseware going forward. If they open-sourced the stack, it could very well lead to an explosion of education opportunities to the lay audience. As far as MIT's 6.002x... the pace was just far too intense for me. I already work 60+ hour days, and the extra 10 wasn't feasible.

Screenshot of Coursera's Game Theory Lectures

Stanford's Coursera is an entrant that many are already familiar with - it seems last year's Artificial Intelligence class was a HUGE hit with everyone I talk to. I can't throw a pumpkin without hitting an engineer that raves about Stanford's online courses last year... and trust me, I've tried.

Coursera is a bit more low-key than MITx. A simple list of video lectures are provided, a discussion forum, quizzes / problem sets and... that's about it. A 90-ish page textbook is available for $5 from a separate publisher, but is not key to completing the assignments. Contrast that to 6.002x where there was generally 100-150 pages of reading a week, and you get an idea of how different the scope is. If 6.002x requires 1-2 hours a day, Game Theory requires 15-30 minutes a day.

There are some similarities between MIT and Stanford's approaches. Just like MITx, Coursera injects comprehension exercises within the video lecture stream. However, instead of being an HTML form the exercises are displayed as Flash forms within the video player itself. On one hand this is a bit more streamlined an experience, on the other hand you lose a lot of interactivity and features. One major annoyance was that exercises can sneak up... and often I want to rewind 30 seconds to make sure I understood the key concepts being asked. However, backing up from an exercise causes a 30-60 delay in the player while it re-buffers video (or somethin'). Backing up often takes the entire lecture off the rails.

One thing Stanford is doing well is that there are weekly Screenside (read: Fireside) Chats where the professors provide an open forum to ask questions. This shows a great level of dedication by the professors offering the class, and I applaud that level of interactivity especially when there are so many students enrolled for a free course. On occasion associate instructors for MITx would answer questions, but there was no regular schedule.

The very fact that I'm contrasting freely available, online courses I'm taking from both Stanford and MIT is enough to make me flip my lid. To have such staples of industry like MIT 6.002 or Stanford's vast catalog of courses open to the general public can make you excited about what the future holds. If MIT were to open their courseware platform and if stellar CompSci foundations like Stanford continued to offer a battery of courses on such interactive foundations we would have an entirely new workforce of software engineers on our hands.

Filling The Pipeline

My past few work engagements have been centered around cloud computing and big data - doing stuff from managing large data centers to machine learning to map/reduce clusters. When I was at VMworld 2011 last year I took the opportunity to ask the "Big Compute and Big Data Panel" about leveraging vector processing hardware such as NVIDIA's Tesla to do data processing. The five panelists (Amr Awadallah of Cloudera, Clint Green of Data Tactics, Luke Lonergan of EMC, Richard McDougall of VMware and Paul Kent of SAS) largely agreed on a few main sticking points in vector processing for massively parallel systems:

• The toolset is still relatively immature (maybe three years behind general CISC architectures)
  
• The infrastructure has not yet reached commodity level
  
• Big Compute works well with vector processing clusters, but not big data, since the latter is all about locality rather than in-memory processing
  
• Commodity GPU processing is greatly constrained by memory paging - there's too much latency in transferring large in-memory datasets to GPU memory.
  

AMD had a few interesting announcements over the past few weeks that may pave the way for making cloud and big data/compute clusters more efficient and more "commodity." The first is their acquisition of SeaMicro, whose emphasis is around massively parallel, low-power computing cores with high-speed interconnects. This addresses one big issue brought up during the panel - that interconnects on big data clusters are going to become a prevailing issue as data needs to be transferred across nodes more rapidly to keep otherwise idle compute resources busy. CPUs can't crunch data sets if the data takes forever to arrive over the wire.

The next big announcement, which may be a huge sleeper hit, is AMD's unified memory architecture that's supposed to arrive in 2012. The slide on AnandTech shows that in AMD's 2012 product line the "GPU can access CPU memory," which is a HUGE development in vector processing. Imagine a data set being loaded in 64 GB of main memory, having 8 CPU cores clean the data using branch-intensive algorithms and then that same in-memory dataset being transformed by 512 stream processors. That kind of compute power without the need to stream data across a PCI-E bus could be a really, really big deal.

Still, the issue that remains are the tools that are available to make this happen. Very likely a developer would need to write generic C code to do the branching and then launch a separate OpenCL app to transform it but still share memory pointers so that nothing has to be swapped or paged out. In a world full of enterprise software developers, this kind of software engineering agility isn't exactly easy to find. If Cloudera were able to unleash this kind of power, AMD would have a big hit on their hands. Maybe AMD needs to start looking towards Cloudera as the final stage in the pipeline - an open-source framework that unlocks the potential of their infrastructure.

Let Cooler Heads Prevail with Security

A long while back I wrote about my attempts with passwordless logins and how, instead of people trying to understand the use case, they attacked me under the auspices of security. This is by no means a new argument... the same brand of venom is spewed on behalf of X11's olde tyme conventions, SysV initialization or LSB compliance. Bear in mind the distribution maintainers themselves actually understand these use cases, but often well-intentioned users assume that the same profiles that work for a server installation should also apply to a kid's laptop.

And then they woke papa bear.

I noticed in my Google+ stream that Linus Torvalds had a similar rant, this time also around network & printer administration. Not only that but the focus of his ire was aimed at openSUSE, the same distribution I alternate ranting and raving about. His arguments were not unlike my own - not everything needs to be locked behind a root password (or even a password for that matter). True, you can circumvent or correct this behavior by changing access levels and permissions, but that can be a hairy proposition. By the time your kids / wife / sales people run across this issue, they're already frustrated and you've lost 'em.

Of course there was backlash... and Linus' parting shot didn't quite help things either. The Register even carried a story about the "tantrum" and interviewed openSUSE director Andreas Jaeger, who admitted Linus' had a valid use case but that ultimately "there are bugs but it's not as simple as [Linus] states."

It kinda is and kinda isn't "as simple." User-level access to print queues should be a doable thing. I believe time zone changes can happen just fine - I tested in KDE and while the system timezone did not change, the time displayed on the desktop switched to US Central just fine. NetworkManager no longer needs administrative access (unless you want it to) and yes, passwordless logins have been working for some time now.

In the end, I understand Linus' frustration but have to say that Ubuntu and Fedora are just as bad. Yes, it would be nice to have CUPS admin normal users as admins, definitely. I can't help but wonder if this angst is due to the Linux desktop being so close, and yet so far away. There are still rough edges to iron out, but for an internationalized effort of distributed engineers coming together to make an open-source desktop OS for the love of the game, things are looking pretty damn peachy.

Why Organizations Should Open-Source Projects

I can understand if traditional businesses often struggle with the concept of contributing to open source projects or maintaining OSS projects of their own. If a business manager looks at software like a physical, inherently valuable object it is often hard to make that same object freely available. After all, if the business spent $5,000 on development of the software, why should it give the result away?

I believe the more appropriate view is that the value of an application isn't realized upon publication like a book or a movie. While each may be products of creative labor, software is never really "finished." A well-kept application is always in a state of flux, adapting to new use cases and fixing defects. The only time you are actually done with an application's codebase is when you abandon it.

When you realize software is never completely done, the big question is how a business effectively maintains the codebase. How are bugs found and then resolved? How are new features prioritized and implemented? How do you keep things going without draining your existing engineering staff? Once those questions come to light, releasing software to an open source community makes a lot more sense.

I'm not saying that all applications a business writes should be publicly released as open source. Apps laden with business logic, code that epitomize your core business, will likely not be re-usable for others in the community and may disclose sensitive business practices. However "glue" libraries such as utilities, messaging or scalability frameworks can be highly re-usable and can be isolated so as not to disclose any core business use cases.

If an application is re-usable and adopted by others in the open-source community, they begin to rely on your app and apply their own critical thinking to its codebase. The larger community may conceive of use cases you haven't yet encountered, or find esoteric bugs that you haven't run into yet. Even better, OSS developers will often contribute code or bug patches to resolve issues or add much desired features. At this point the maintenance efforts for your codebase are distributed among a large and very knowledgeable public, significantly reducing the expense in maintaining the code. By adding multiple points of view new ideas and a pool of developers an app may become more reliable than if you attempted to maintain it on your own. Both Netflix and Twitter have released such projects as open source with great success and community support.

Recently I helped foster a similar initative to release a Spring AMQP component for Apache Camel. There was absolutely no business logic within the codebase, it was a glue component used for message transport and so it needed to be as rock-solid and dependable as possible. Not only that, engineering resources were scarce and AMQP best practices were still new to the team. The more eyes that could review the component and the codebase the better it would be.

After some evaluation we chose GitHub to host the source and Sonatype's OSS repository to host the resulting Maven targets. After the intital import into GitHub was performed, we signed up for Sonatype's OSS repository access and began publishing snapshots. Jenkins, our continuous integration server, would check out the codebase from GitHub and then publish snapshots to Sonatype on demand. Internally we started using the snapshot builds, ensuring we pushed our local changes into GitHub whenever we were ready to distribute another snapshot build.

One question that arose early was what the company's "sponsorship" of the project should be. The company managers and directors wanted to ensure the project didn't carry any organization artifacts with it - for example, packages or classes that carry the company name. However it does make sense to have a single steward of the project, one that monitors submissions and maintains the pipeline. To that end we used GitHub's "team" concept to create a team repository where software engineers were added as owners of the team code. The core team would merge pull requests, monitor issues being submitted and ultimately be responsible for pushing artifacts to the Sonatype repository. If an employee would leave the company they could possibly be removed from the team itself and no longer be granted rights to publish to the Maven repository, however they could still create forks and provide pull requests. This was an added benefit - engineers could continue contributing to the project long after they left the company itself. By open sourcing the project we could both open maintenance to a sea of new developers as well as prevent losing the historical knowledge of old ones.

While the code itself was freely available and the binaries were actively published, the project couldn't necessarily be considered "released" to the community until we began promoting it. The project spanned many other popular projects including Spring, Apache Camel and RabbitMQ. Once the component was in a stable-ish state that could be tested by other developers, posts were submitted to mailing lists for each project. There were varying levels of response, but a few individuals began to express interest and even volunteered to write How-To documents and include it in peer presentations. At the same time I also started to see if I could share lessons learned with the StackOverflow user base, and offered snippets from the component's codebase if I though it could be useful. As a result the GitHub project began to get ranked higher with search ranks, which also helped greatly with visibility.

Once the component started to be used within production we released the initial one-dot-oh release. The Maven release was promoted to the Sonatype stable repository, tweets were tweeted, posts were submitted to mailing lists and we tried to invite as many people as necessary to kick the tires. Once it became easy to include the component in Ivy and Maven dependencies, adoption greatly increased and more people tried the component. As a result we started to see an increase in pull requests, suggestions and bug fixes. There were a few deviations from the AMQP specification that we wouldn't have noticed had not the community taken a critical look at the component and provided patches. Use cases for asynchronous production made for very helpful unit tests and helped prioritize new features. The 1.1.0 release of the component was markedly more robust than the 1.0 release but required less engineering effort on behalf of the team.

To my mind everyone won through the open-sourcing of the camel-spring-amqp project. The company was able to deliver high-quality software hardened through peer review and a global pool of developers were able to re-use a collaborative codebase. Overall cost went down, business value went up and high-fives were copiously distributed to all involved.

Less Cash Makes More Money

I went on two spending sprees this season - one hosted by Google's Android Marketplace, the other hosted by The Humble Indie Bundle. The two shopping sprees combined cost me less than twenty bucks, but out of them I acquired no less than 48 (yes, forty-eight) commercial applications. Four of them not only included source code, but previously undisclosed version control access to said code and remastered soundtracks. That's a pretty freakin' good deal.

One might think that my mad app grab put these software studios at a loss... but that appears to be far from the case. Introversion Software (a personal favorite of mine) noted that the opposite effect occurred:

As we pass 180,000 sales of the Humble Introversion Bundle, I can’t help but feel astonished. This is the biggest single sale Introversion has ever done on any platform, even beating the epic Steam promotions we run from time to time. It roughly equates to one sale every six seconds. And the best part of all - the part that makes me most happy, is that this promotion has doubled the number of people who have played our games. 180,000 is more copies than our best selling game Darwinia ever sold, and the Humble Bundle includes all four of our games, so that means twice as many people have now played each of our games than before the bundle. That’s pretty cool. As the principle game creator here at Introversion, that’s the biggest thing for me.

And let’s not mess around, the revenue is pretty good too. When it’s all divided up we’ll see about one quarter of the total revenue you see on the Humble site - and I’m sure Humble won’t mind us revealing that, because it’s obvious if you’ve look at the default sliders. That’s one quarter of a pretty big number, and that revenue will keep us going for a long while. And we’ve done it while raising over $200k for some very worthy charities.

As for those publishers who participated in Google's 10 Apps / 10 Cents / 10 Days promotion? I haven't heard of any testimony from that front, however watching the Android Market itself has been very interesting. During the 3 weeks spanning the promotion and the week immediately following I've seen more activity and application updates pushed than I've seen in the past 3 months - including apps that were not part of the promotion. It seems the promo not only featured some titles whose volume might have otherwise tapered off but it also increased interest in adjacent titles as well, enough so that publishers wanted to have updates waiting in the wings for them.

I've noticed that this kind of title distribution is gaining steam, enough so that Rock, Paper Shotgun has almost been driven insane by the sheer number of bundle announcements. This has the leading indications of being a potentially disruptive market force... as much as I might loath the cliche. This method of catering to all price points has somewhat been used by recording artists and game studios years before, but the emphasis was selling at cost and then giving premium content to the upper 10% of the fanbase who would pay for it. For some artists lowering the cost barrier wholesale can pay amazing dividends and grow a loyal user base in short order.

openSUSE 12.1 - Good Enough to Blog About

I've been away from the blogging scene for over eight months now. My language skills have languished likewise. It seems only fitting that my return to posting is launched on the same topic I departed with.

For too long it seems like openSUSE has been stuck within an odd, "uncanny valley" of user experience and desktop usability. It did so much well that the stuff that worked poorly really stuck out like a sore thumb. I've never really had the problems with YaST that others of my kin seem to decry ("Omigosh, you use SuSE? Ugh, is YaST any better?"), but other issues like poor integration points (e.g. Eclipse) and the banishment of SCPM and SaX2 were frustrating.

openSUSE 11.4 ended up looking okay however had issues with wireless, the noveau driver wasn't ready for prime time and LibreOffice ended up being crashtastic. Luckily (I guess?) I had a hard drive crash right before openSUSE 12.1 launch and was able to start clean when the mirrors sync'd on November 16th. As soon as openSUSE 12.1 launched I grabbed an ISO and began installing on my ThinkPad W510.

Man, what a difference.

First off, the noveau driver is still a bit buggy but has nearly reached parity with the proprietary NVIDIA driver on desktop compositing and UI effects like wobbly windows. I can't tell you how jazzed I was about this lil' facet. Don't get me wrong, I absolutely adore NVIDIA and the work they've done to bring driver support to the Linux desktop. Still, an OSS solution that has native integration into XOrg provides a seamless experience - especially with xrandr and KDE 4. Now KDE (and to a lesser extent, the bootloader) can natively manage resolution, orientation and multiple desktops. I was a bit apprehensive given I frequently switch between projectors, dual-screen setups with larger monitors and road-warrior setups with just the laptop display however noveau and KDE 4 are able to switch between these environments without so much as a mouse click. Plug in a VGA cable and boom - I'm broadcasting a second desktop over the monitor. Sweet.

LibreOffice is much more stable. No huge features I've been able to discern, but then again I'd likely not even notice them given my superficial use of any office suite. Evolution seems a bit smoother and more responsive - hopefully someday soon my office can switch to Exchange 2008 so I can leverage its new SOAP API via Evolution. That should make a big difference over the webdav connectors currently being used.

I've been trying out new features such as brtfs on alternate workstations, since encrypted brtfs partitions are not supported yet. So far it has been quite nice having snapshot capability and b-tree balancing to a file system, especially given how many crazy files I generate or move around.

Still very happy with Digikam; it has become my mainstay for post-production of RAW photos. It somewhat softens the harsh reality that my wife's iPhone 4S take better photos than my Canon DSLR.

All in all, a major tip o' the hat to the openSUSE team on all fronts. Very happy with the usability of the latest release.

Amazing openSUSE 11.4

First of all, I'll admit that I've waxed gloom-and-doom when talking about openSUSE lately. I'm now saying not only that my worries turned out completely wrong, but openSUSE 11.4 is probably the most amazing OS I've run so far.

I'm not claiming this assertion lightly, either. For the first time in the history of any OS I've ever installed hardware was detected and installed correctly the first time right away. 3D acceleration worked out of the box. My Intel/Broadcomm chipsets functioned immediately without me even having to edit a config file. I dual-boot with ease. All in all, an effortless installation.

Performance increases were immediately evident as well. Apps loaded swiftly, content panes refreshed immediately and window compositing worked fantastically. And in this sprint of performance I found that I didn't need a single drop of proprietary software. The open nouveau driver for NVIDIA GPUs now has sufficient 3D support to provide desktop effects such as transparency, and Broadcomm wireless chipsets are now supported within the latest kernel. No joke! Package management is many, MANY times faster now due to more simple and more concurrent HTTP operations. Updates and package installation is in a whole new league now, and wins hands-down over any other distro (including Windows).

But wait! There's more! openSUSE 11.4 also is the first distro with Firefox 4.0 and LibreOffice, giving me the freshest builds of two suites I use frequently. Evolution has been updated so that IMAP operations are non-blocking (oh thank the heavens) which makes it appear much faster as well.

Reading down the list of openSUSE 11.4 product highlights is like looking at my own personal want list for a Linux desktop. If the coming work week goes as smoothly as the past couple of days have, this will be the distro to beat all distros. I even find myself wanting to use openSUSE 11.4 over Windows 7 for random tasks at home, simply because the user interface is much more elegant and flexible.

I owe openSUSE 11.4 so much that I've finally given up on typing the distro as "OpenSuSE" in homage to the original distribution. SuSE has now grown up and is now truly a fantastic product of the open distro's team - and is now truly openSUSE in its own right.

Unbridled IDE Anger

It's midnight as I type this. I have spent the past four hours attempting edit a single freaking Grails project within an IDE. Any IDE.

I switched to NetBeans almost nine months ago and I've been a very happy NetBean'er. Everything worked out of the box with minimum futz and complied quite nicely with accepted standards. Maven 2, Subversion, minimalist Java runtimes and Spring hummed along without complaint. Looking to accelerate development of simple maintenance web applications I turned to Grails... and then things started to fall apart.

Grails actually has some great tooling within NetBeans, but one bug has been a huge thorn in my side. An acknowledged bug causes Grails projects to not notice any external libraries, even if they are your own and open within NetBeans. This is beyond annoying because features like syntax highlighting, error detection and auto-completion go completely awry. The bug is reportedly resolved in the upcoming version 7 of NetBeans, however I installed Beta 7 and the problem continues to occur.

Not being beyond switching back to Eclipse, I decided to try Spring's own IDE. Spring's IDE is a slightly customized version of Eclipse, with a bit nicer front-end dashboard for plugins and Spring product support built-in. Since Grails is a Spring project I figured it would work well.... and indeed it did. My other projects... well...

SpringSource Tool Suite uses the latest beta of Maven 3 for dependency resolution... and you cannot change that. Yes, you can specify alternate Maven installations, but STS completely ignores those settings for dependency resolution. Maven 3 will also refuse to load transient dependencies if it doesn't like your POM as well - which means that you simply cannot load your application with all required libraries if your Maven 2 pom.xml is rejected by Maven 3.

I could go back to my old Eclipse installation... and then I need to find a way to load the Grails plugins into the old Eclipse install. That sounds like loads of fun.

So now I have three choices:

1. Use NetBeans and have absolutely no auto-complete or strongly typed variables
   
2. Use SpringSource Tool Suite and re-do all my Maven 2 POMs
   
3. Use my old Eclipse installation and try to install the necessary Spring plugins
   

Now, bear in mind I've burned four hours trying to STS to work. Another four to get NetBeans to work. How interested do you think I am at trying to get MORE plugins shoved into my old Eclipse installation?

I'm cheezed because NetBeans Grails support is rendered nearly useless by a fairly blatant bug. And I'm cheezed because Eclipse continues to be an albatross to every normal project I have. IntelliJ IDEA - you might just be next.

Until then, I've got a Grails app to blindly maintain while NetBeans continues to yell about imports not being found.

Candyland Gets Paved. Again. Repeatedly.

I'm getting a bit tired of being so curmudgeonly. I can't help it tho. All my toys are being taken away.

First Novell acquires SuSE Linux, which I have been fairly skeptical of since the purchase was announced. Then Nokia ate Trolltech, the place where my favorite Qt was grown. And then Oracle ate Sun Microsystems, the ones who kept expanding the boundaries of software engineering and releasing such leaps to the public (usually (sometimes)). Now Oracle is smacking around robust and growing projects that have served the software engineering good for many years now.

And now Novell is bought by Attachmate, with 882 patents being absorbed by a Microsoft subsidiary. The openSUSE team says everything is "business as usual,", but there are very strong indications that this will not always be so.

Maybe this is just the ebb and flow that is technology capitalism... but all my power tools seem to be disappearing. If Apache and JBoss are bought by Philip Morris, I'm going to freakin' lose it.