Linux has a long history of software development, and a flourishing range of applications that meet even the most bizarre of needs.
A quick poke around freshmeat.net is testament to this, with its rich tapestry of useful, wild, and often wacky apps. Distros such as Debian, Fedora and Ubuntu bring these applications to their users with a quick apt-get or a yum install.
As this work was going on, Apple released the iPhone and made a major breakthrough in computing; it made the application developer platform attractive, consistent and accessible to consumers. Anyone with an iPhone could download a range of apps, many simple and silly, but all consistent in their presentation, and often new and innovative in how they used features of the phone such as the GPS, accelerometer and other facilities.
Apple built this platform from scratch, but thanks to the work put in by the GNU project and countless others, we already have a platform. We have a rich set of development tools, a range of desktop environments and a wide range of development forges packed with source control, bug tracking and other features.
Let's get better
While we have had the tools for a long time, what we have done less of a good job at is pulling these tools together into a consistent experience aimed at application authors. This is something that the Ubuntu community is working on, and we're going to look at some of this work and how it's useful for the wider open source ecosystem – that includes you too, non-Ubuntu fans!
Developers are funny beasts. Although from the outside they seem like a fairly consistent menagerie of code-writing, beer-drinking, pizza-eating creatures, their motivation and desire around the art of writing programs varies.
Within this variety though we can discern two sub-sets: systematic and opportunistic developers. Both write code, but each motivates themselves and approaches their work in slightly different ways.
Systematic developers are typically professional developers. They're the kind of people who will write a feature, and then immediately document it, write an automated test suite for the feature and will always be thinking structurally about their application today and how it will scale and grow with future considerations.
Systematic developers are the very definition of professionals, and they are commonly conservative, cautious and resistant to adding features unless they are implemented with completeness and precision. Systematic developers commonly commit themselves to a single project for long periods of time and often grow significant generalist knowledge of the codebase, becoming almost tribal elders in their respective projects.
Opportunity hacks
Opportunistic developers are the inverse of many of the properties of a systematic developer.
Opportunistic developers are interrupt-driven hackers who are in the business of scratching itches. They feel a problem or smell an opportunity and will often have a first cut of code ready within a matter of hours. They are often much more shoot-from-the-hip types of folks, their code is sometimes not well commented, and unit tests are often a low-priority item on a to-do list somewhere, but these attributes don't necessarily mean they are inferior coders.
They are often excellent coders, but they are reactive, energetic hackers who love to solve problems that they feel personally and are passionate about.
Opportunistic developers are the very lifeblood of Linux. When we talk about the basic building blocks of open source we often talk about 'scratching your itch', this is what opportunistic developers do. Our goal in the Ubuntu world, and the goal of many others, is to ensure that the barriers to itch-scratching are as low as possible.
Optimising for opportunity
When developers want to produce software they enter into a four-step process: Imagine > Create > Collaborate > Publish. This broad high-level set of steps can be broken down into a more detailed set of steps, which break down into the following elements:
DEVELOPING STEPS: The common steps involved in building a Linux application.
1. Ideas: This matches the Imagine step of the higher-level process; thinking of ideas of software to work on.
2. Gnome/KDE: This matches the Create step of the higher-level process; using a preexisting development platform to create your application with.
3. Launchpad/BZR: This matches the Collaborate step of the higher-level process; using Launchpad and Bazaar to work with other developers to make the application better.
4. Debian Packaging/PPA: This matches the Publish step of the higher-level process; packaging your application and then publishing it to a Personal Package Archive, which enables others to download and install it like any other package.
Let's now take a look into some of the work and projects that have been going on to simplify and improve this process.
Imagine
The very first step is to imagine a solution to a given problem.
At this very first stage the developer needs to feel empowered to have the motivation, tools, skills, and determination to implement the vision that they see in their minds. Although this sounds like a simple first step, it is a challenging one.
To optimise it there needs to be a wealth of positive stories of how developers have dreamed up solutions to problems and effortlessly implemented them because the platform was a help rather than a hindrance.
In the Ubuntu world we have tried to build an atmosphere around the concept of Ubuntu providing a complete and comprehensive platform for implementing whatever solution the developer dreams of. We have done this by organising events such as Ubuntu Application Developer Week and creating support resources such as the #ubuntu-appdevel IRC channel on the Freenode IRC service.
With the motivational element of opportunistic developers a story of encouragement and outreach, the following three steps in the four-step process are very much about technology, and the goal is about lowering the barriers to get people up and running as quickly and easily as possible.
Create
Over the years a vibrant developer community has formed, complete with a vast array of tools, languages and functionality. Unfortunately, while powerful, many of these tools are awkwardly complex, and many developers have let their ideas and creativity get buried under an avalanche of confusion around how these tools fit together.
Part of the cause of this problem is that many developer tools only cater to systematic developers; the kind of codewriting workaholics we mentioned earlier who hack for a living, with a fervent attention to detail backed up by unit tests and other hallmarks of the professional programmer.
For many opportunistic developers, if the tools needed to scratch their itch require too much effort or investigation, the itch can quickly disappear and what was once a creatively excited hacker has now moved on to be a couch-bound excitable PlayStation gamer who grew bored with Linux as a platform.
A solution to this overt complexity in the toolchain was a simple tool called Quickly. Quickly was the brainchild of the now director of Ubuntu engineering at Canonical, Rick Spencer. Quickly gets you up and running quickly (it's not just a clever name) writing an application from scratch.
Traditionally, writing desktop applications has involved a not-insignificant amount of faffing required with build systems, source control, packaging frameworks, graphical interface tools and other things that get in the way of writing code. Quickly is a tool that simplifies how these different things fit together.
Quickly provides a framework with a series of templates for creating different types of applications. With each template a series of decisions are made about the tools involved in creating that application. By far the most popular template, and the one that Quickly itself was created to satisfy, is the Ubuntu template.
This template uses a set of tools that has become hugely popular in modern desktop software development, and tools we have harnessed in Ubuntu. They are:
Python: A simple, easy-to-learn, flexible and efficient highlevel language.
GTK: A comprehensive and powerful graphical toolkit for creating applications, and the foundation of the Gnome desktop environment.
Gnome: The desktop environment that ships with Ubuntu, offering many integration facilities.
Glade: An application for developing user interfaces quickly and easily, which can then be loaded right into your Python programs.
GStreamer: A powerful but deliciously simple framework for playing back and creating audio, video and other multimedia content.
DesktopCouch: A framework for saving content in a database that is fast and efficient, hooks neatly into Ubuntu One and is awesome for replication.
Gedit: For editing code, Quickly assumes you are going to use the text editor that ships with Ubuntu, which provides a simple and flexible interface for writing your programs.
With this core set of tools you can write any application you could imagine and know that it will run effortlessly on Ubuntu and other distributions. The elegance of Quickly is that it understands a common platform for Linux but does not try to complicate the desire for simplicity by being tempted down the slippery slope of investing months of energy into an Interactive Development Environment (IDE), when many Linux users are in fact comfortable with the command line.
Collaborate
While Quickly is fantastic for getting users up and running with a new application, there is a much wider challenge around how developers can collaborate together around code. Producing software and providing an environment in which contributors can work together on it requires a large number of tools and the integration of those tools.
In the bad old days of open source it was a nightmare to set up and integrate these tools, but these days we have a variety of free websites with ready-to-roll development environments for creating and maintaining open source projects easily. One such example is Launchpad.
LAUNCHPAD: Launchpad is Ubuntu's site for hosting code and fixing bugs, and is where you should upload your new app.
Launchpad is a powerful, simple and comprehensive development forge that has become hugely popular over the last few years (it currently sports over 17,000 projects) and is right at the heart of how Ubuntu is developed. The site provides a range of useful facilities:
Code hosting: Launchpad knits together neatly with the popular Bazaar distribution version control system. Together they provide a fantastic method for contributors to work together on code, merge changes into a main code base and host code online.
Bug tracking: Although traditionally a complex and ugly part of software development, the bug tracker built into Launchpad is simple and effective.
Translations: Many projects struggle with providing multiple language support, but Launchpad provides a simple means for anyone who knows more than one language to translate applications without ever touching code.
Blueprints: This feature provides a means to produce specifications for ideas and features in your project.
Support: Launchpad provides a question-and-answer facility that is well suited to give support for your users.
Package building: A hugely popular feature in Launchpad is the ability to have your very own Personal Package Archive (PPA), which enables you to automatically build and deliver Ubuntu packages to your users.
If you want to find out more about the range of facilities in Launchpad, you should take a look at the online tour at https://launchpad.net/+tour.
Launchpad offers a simple and effective experience for creating applications, and much of its simplicity is in how the different components of Launchpad can link together. As an example, you can create a blueprint and specific, attach bugs to it, attach Bazaar branches to bugs and more. This interconnection of information helps simplify visibility of information and ensure that developers always know what is going on.
Launchpad is not perfect though, and some developers have tried to simplify its use in application development. One such example is the way that Quickly enables you to publish to Launchpad (more on this later). Another interesting example is a tool called Ground Control by Ubuntu community member Martin Owens.
GROUND CONTROL:A more specific set of steps to produce an app for Linux.
Ground Control takes an innovative approach in turning your file manager (Nautilus) into your development environment. Imagine you want to fix a bug. The process for fixing a bug is typically the one highlighted on the opposite page. It works like this:
Choose a bug to fix: You find a bug on Launchpad that irritates you enough that you want to fix it.
bzr branch: Download the code for the project that's afflicted by the bug.
Fix bug: Perform the fix in your local branch of the code.
bzr commit: You commit the fix to your local branch, ready to push.
bzr push You push the code to Launchpad so the maintainer of the application can take your fix and apply it.
Attach branch to bug report: For completeness, you attach the branch to the bug report. This ensures that anyone subscribed to the bug report is aware of the fix.
Propose for merge: You then follow the Launchpad 'Propose Merge' process in which you notify the original developer of the fix so he/she can review it and merge it in if suitable.
When you are a new developer starting this process, all those commands and the correct order and syntax can be a little confusing. Many developers have gone so far as to create a sticky note outlining the process until it becomes rote.
Martin Owens' Ground Control project provides an entirely graphical way of performing the same process...all within a file manager. The way it works is that you load up Nautilus and browse to a Projects directory in your home directory. In there is a 'Fetch Project' button. Clicking on it pops up a dialog box in which you can search for a project (for example the Ground Control project mentioned earlier).
When you perform the search a list of matching projects will be displayed, and you can click on one to select it. Doing this creates a new folder in the Projects directory in Nautilus with the same name as the project (eg groundcontrol).
If you click inside this new folder another button called 'Fix Bug' appears. Clicking it pops up another search dialog box which enables you to search for a bug number or bug search term inside that project. When you search, a range of bugs are displayed, and you can double-click on one to grab the latest code from Launchpad and automatically create a folder called bugfix-lp-123456.
QT Creator: The KDE team have an excellent set of Qt development tools for building apps.
Ground control
You now go and hack on the code in that folder and fix the bug in question. When you have changed some of the files in that folder a new button appears in Nautilus called Upload Fix. Clicking that button opens a new dialog box where you can describe the changes you made to the code.
Clicking OK then pops up a final dialog box asking you to enter a merge message (this is the message that you send the developer asking them to merge your bug fix into the main code). When you click on OK, your changes are pushed to Launchpad, the branch is attached to the bug report in question, and a merge proposal is automatically made.
The entire process simply involves clicking buttons in a logical set of events, and at no point do you ever need to enter a command or create a note to remind you of the process. Projects such as Ground Control demonstrate the desire to simplify the process of collaborating on development, and the project was made possible by the flexibility of the Launchpad API, which enables developers to provide alternative interfaces to the date inside Launchpad.
Publish
With a simple method of creating applications, and a simple method of collaborating around applications, the next step is to get your application into the hands of users. This process is typically broken into two steps:
Packaging the application: Making the installation and removal of the application compatible with different distributions by using either the Debian packaging system (Deb), Red Hat Package Manager (RPM), or other system such as Gentoo's Portage.
Uploading to a distribution: Unlike with Windows, we don't expect users to go to random websites and download executable files. We instead expect distributions to have large archives of pre-packaged software. As such, we need to get the application uploaded to the archive.
Unfortunately, both of these steps have traditionally been quite complicated. The former has involved learning the relevant packaging systems, which in themselves can be fairly complex even for a basic desktop application. Part of the challenge with packaging has been that there are often many different ways to package an application, and the skills required to package your new app are often outside of the scope and interest of application developers.
Fortunately, Quickly eases this significantly. With a single command you can generate a Debian/Ubuntu package that's fully compatible and pulls in all required dependencies (much of this was made possible due to the excellent Martin Pitt).
In addition to this, Quickly includes a 'release' command that will automatically produce an Ubuntu package and upload it to your Launchpad Personal Package Archive, all in one command. This effectively makes it a one-command operation to publish new versions of your software, and saves you oodles of reading about packaging when you would prefer to be hacking on your app.
QUICKLY: A sample Quickly app just after it has been generated.:
The latter of the two steps above, uploading to the distribution, is the more complex element. All Linux distros have teams of developers who have worked hard to build trust and technical competence to be approved as an official developer; that is, having direct upload rights to add packages to the archives and future releases of a given distribution.
Gaining these upload privileges often requires significant skills, and skills that are traditionally designed for assessing operating system integrators. As an example, in Ubuntu, there are two broad types of contribution:
Core Dev: This is for developers who want to upload to the 'main' archive, which includes all of the officially supported software (such as the software in the official release ISO images and CDs).
MOTU: This is for those developers who want to work on the non-supported Universe archive, which includes thousands of applications imported from Debian.
Becoming a Core Dev requires significant generalist Ubuntu and packaging knowledge, and it also requires comprehensive technical competence to become a MOTU. With both there is an assumption that developers will be working on multiple packages, and these developerassessment processes rightly require a high level of quality.
The challenge with these current processes is that for app developers they are a little heavy. To help resolve this in the Ubuntu 10.10 cycle a new process called the Application Review Board was introduced, in which application authors can submit an application for technical assessment by a community board.
If the application meets a set of technical assessments around code and packaging quality, the application is approved and made available in the Ubuntu Software Centre. Details about the process can be found at http://wiki.ubuntu.com/AppReviews.
Wrapping up
In the last few years we have seen ever more competition in attracting application developers to different platforms.
While Apple and Google have done an excellent job with their respective platforms, there is a huge opportunity to make Linux a top-tier application platform, and this article outlines some of the work going on in the Ubuntu world to help encourage and motivate application developers and make their lives as easy as possible.
This article has not had the space to cover the many innovations happening inside the Gnome and KDE camps, other distributions or the wide variety of upstream projects that are seeking to make development easier. Fortunately, it seems that many in the open source community are passionate about enabling more people to contribute to free software, and if we keep stepping back and making our different tools, processes and systems easier to use, we can hope to see a wealth of additional applications available across different Linux distributions.
