Agile development

For a long time, I have been wanting to write a blog on development methodologies, but haven't done so because I usually write the blog when I am satisfied with my survey of all the recent developments in that area. But methodologies are endless research so here is a brief overview of Agile which will be updated soon.

A brief overview first. Strictly speaking, Agile methods are not a methodology but a philosophy for modern application development. So there are specific methodologies like XP, Scrum, Crystal, Lean Development (from the Google article) which belong to the Agile class. It is absolutely true that most of our development happens that way and we should use Agile philosophy. As for RUP, it is a broad set of practices like use cases, iterations, documentation. Hence really speaking, RUP is a framework that can be fitted with many methodologies. For example, RUP can be used with XP (incidentally called dX - XP turned round on its head :)

When it comes to choosing a methodology, it is easy to fall into the trap of mix-and-match. For example, many organizations claim to follow a variation of XP, but all they are doing is - following the part of XP which says developers may not question the requirements, but conveniently forgetting that XP strictly specifies that only developers may give the estimates for the work. For the same reason, if we are to really follow Agile, we need to watch for conflicts. Agile clearly says that it is heavily dependent on people rather than just process-oriented. So it does not think of people as resources but as individuals, each of which has its own strengths and weaknesses and it is no longer possible to think of developers as pluggable units. It also changes the process itself if people want that.

I have also studied a bit on outsourcing using Agile methods. There are some big differences. First of all, usual Agile does not rely much on documentation but on close proximity between technical and business people. In outsourcing scenarios, this needs to be handled by more documentation. Practitioners believe that the amount of documentation should be "just enough", although this is determined only by experimentation across several projects. Secondly constant visits from both sides are necessary. Third is that business requirements should be written down at the client side and then test scenarios be written at the outsourcing side. Most important is that continous integration systems be in place, so that nightly builds can be taken any time the customer wants to see the current situation. I also believe that Agile methods have certain weaknesses, like lack of an integrated architecture, which can be well addressed by the software architecture artifact from RUP.

My views are influenced by Martin Fowler (universally heralded for design patterns), Alastair Cockburn (of Crystal fame), XP proponets, RUP website and others. Some interesting links for further reading -

1. Misuse of Iterations
2. Various articles on methodologies by Martin Fowler
3. Internationally Agile
4. The new XP

Mp3 tagging through music fingerprints

I have a collection of few thousand mp3 files with unmeaningful names and tags. Sometimes I do try to rename the tags and filenames using iTunes, Windows Media Player or other editors but its just too huge a task to be completed, more so because the collection is increasing all the time. There is a huge number of tools for fixing, ranging from tools which can edit the tags, to tools which can write the tags using filenames. While learning Python, I wrote a small code to beautify the names, extending which is a Someday task in my GTD list :)

It is too difficult to edit tags using iTunes, even with the functionality of editing multiple files simultaneously. I found this great open source tool Mp3 book helper, which uses the good trick of exporting/importing tags and filenames to CSV files. It is much easier to edit the CSV files using Microsoft Excel, specially because of Macros and functions and then import the corrected file.

However, often the filename and the tags are completely incorrect making it impossible to correct it without listening. I have always wondered if there could be a way to digitize the music patterns and look the tags into a database of music patterns. Today I was searching if there has been some work on it, and found to my suprise that there are many such systems already implemented.

MusicBrainz is such a community which has such a database of music fingerprints and tags. It provides a freely downloadable tool called Picard that can search and correct the tags of your files by sending music fingerprints to MusicBrainz server, although highly manual and not automated. It uses music fingerprinting technology supplied by MusicDNS, which has a database and client side APIs for collecting music fingerprints of various audio files and searching the database. The use of this API is completely free for non commercial purposes. A real act of benevolence I would say, attempting to bring some sort of an order into the huge amount of audio information !

Looks like it can be classified as a Web 2.0 application...the elements are there all right. The database is unique and is constantly improved by users contributing music fingerprints and tags. Programming model is there too, only thing missing is a nifty user interface. It would be nice to write a service which constantly monitors your collection and automatically tags your files by running the MusicDNS api.

Web 2.0

Every client wants to enable Web 2.0 for their applications, and like Paul Graham, I initially disliked the term so much that upon its mention I used to immediately launch into a lengthy marketing talk about user experience, AJAX, collaboration and the semantic web. No wonder the term has been called marketing slogan and even amoral. Looks like there are still some folks out there in software who believe in honesty ! :)

Finally I decided that enough was enough and at least I need to try to appreciate it from a business perspective if not technical.

The term was popularized by O'Reilly Media and MediaLive International for a conference they hosted, the first Web 2.0 Conference in October 2004. Later on, an article on oreilly.net clarified the meaning in detail. This article tries to differentiate between Web 2.0 applications like Wikipedia, Blogs, Google Adsense, Flickr and their Web 1.0 counterparts like Britannica Online, Websites, DoubleClick and Ofoto, and uses the differences to come up with Web 2.0 principles -

1. Web as a platform - Software is provided as a scalable service. Examples are Google Search, Adsense, BitTorrent
2. Harnessing collective intelligence - Users are trusted to contribute to the collective material. Examples are Wikipedia, Amazon reviews, Flickr tagging, Blogs
3. Data as central - Application relies on a unique, hard to recreate repository of information that becomes more valuable as more users use it. Examples are Amazon, Imdb
   
4. Lightweight programming models - Programming models allow use of various platforms and integrate easily to form innovative applications. AJAX, REST web services, RSS are key components in this. Mashup is a good example of these put to work.
   
5. Software above the level of a single device - Devices can utilize the web to perform useful services. iTunes and TiVo are good examples.
6. Rich user experience - Gmail, Flickr
   

Dion Hinchcliffe examines the current state of Web 2.0 with a brave attempt at architecture of the web using people, machines and technologies. In a seperate article, he also lists the applications best qualified to be called Web 2.0.

After all that reading, I think I have at least a handle on the new web. I can't get rid of the feeling that technically, nothing has changed. AJAX means that now we can use Javascript safely and can get away with supporting a few browsers :) RSS has also been around for long. Web services have been there for a long time as well as as P2P. One thing that has really changed is the convergence of devices.

However, it is definitely true that these technologies are being combined in ingenious ways to create new applications that allow users to access relevant information and collaborate more effectively. And if you would like to call it the emergence of Web 2.0 and it makes business sense, I am fine with that !