Jon Atkinson

I'm a technologist, with a long and varied history in high-end technical delivery and infrastructure management. I'm particularly interested in managing software teams, rapid application development, and scalability challenges.

The biggest mistakes of my career

11th December 2008

I've just finished reading this post about horrible experiences in software development, by Eric Spiegel.

I've been working in the web industry for quite a while now, and like anyone, I've had my share of project which have gone bad, but I'd quite like to write about one of them; to my mind the worst sequence of mistakes which I've made in my career, and certainly the one which I felt the most guilty for.

I'd just quit university, I was in my first job as a proper web developer. I was part of a decent small team, we had a stable of undemanding but largely satisfied clients, and after a month long hand over period, the existing developer left and I remained as the sole developer at the company.

Life was good as I learnt the ropes, and for about a year, everything was going really well. I was increasingly going on pitches with my boss, who to this day is probably the best web salesman I've worked with; primarily because he was a salesman who came to the web, rather than a techie who moved into sales; the validity of that opinion is an argument for another time.

I became involved in the pitching and specification of a total website rebuild for a recruitment company who were servicing a niche in the recruitment market, but who had their eyes on moving up the web recruitment food chain. Like us, they were a small team, young, and we hit it off pretty much instantly.

In retrospect, the problems started early; I think that both we and the client lost touch with reality as we thought about the wonderful site which we were going to build together. We were going to be part of building a national brand! This client was going places! This was going to be the greatest recruitment site ever, and there were going to be dump trucks full of money rolling into the car park any second! We couldn't fail.

Of course, while we sat with the client, tossing around feature ideas, writing ridiculous specifications and generally talking about the work we were going to do, no code was being written. Repositories lay empty, Photoshop canvases were blank, and as the days slowly crossed themselves off the calendar, our six-month deadline slowly crept towards us.

Before I even began to become anxious about having written no code, we made the second big mistake; the client had solved all our problems! They'd bought an off-the-shelf set of PHP scripts from a software house in Mumbai. It seemed like a great way to leapfrog the pain of developing from scratch systems which any web application take for granted; login, signup, password reminders, search. All that was left to do was hack in a few custom features, insert our templates, and we would be done. The client didn't even want to negotiate a reduction in our price.

I couldn't believe my luck. I could coast for the next couple of months, and I could still deliver the project on time.

So, the code the client bought sat on my hard drive, I installed it, and every now and again I'd fire up my editor and tinker around with it. Tt seemed to fulfill 90% of the features we wanted for launch, the next mistake I made was telling myself that that meant I'd need to spend 90% less time on this project.

Of course, I couldn't have been more wrong. As the next few weeks passed, our designers delivered designs, we had yet more meetings with the client (adding more features along the way), but I was still convinced that I could pull this off. Eventually, about a month before the deadline, I finally sat down to work.

The details of what happened next aren't particularly interesting. It's the same story which most web developers have seen. The code we had bought was a mess. It was a nest of markup and PHP code, entirely procedural, with insane variable names and no coherent structure. Even modifying the smallest behavior of the system involved trawling through 10 uncommented functions. There were no tests to run, no way of ensuring the changes we were making weren't compromising the integrity of the system. I started turning out javascript and iframe hacks to make the code behave as I wanted. Database normalization was forgotten. The nights before client demonstrations turned into all-night patching and hacking affairs, and the demonstrations themselves ended up being heavily scripted to make it look like we had made some progress. I was too close to the deadline now to admit that I'd secretly known for a month that there was no way this projects would be done on time. I'd look like an idiot, the company would lose a client and I'd probably lose my job.

As we neared the deadline, I was working longer and longer hours, 80, 90 hours a week, just to maintain the illusion that I was in control of the project. I wasn't eating, I stopped talking to friends or colleagues. I stopped visiting my girlfriend, I slept at the office, and inevitably the amount of progress I was actually making steadily dropped to zero, as I desperately tried to fit six months of work into a few short weeks.

We were due to launch on a Monday. I worked through the weekend on no sleep. The site was still nowhere near completed, and I had the client calling me every hour to check on my progress. I wasn't going to meet the deadline, in fact, I was going to sail straight past it. I'd known this for a while, but now I couldn't hide it any longer.

On the day before launch, my girlfriend visited me at work. I remember her telling me that I looked tired (which was probably an understatement), and suddenly everything slowed right down. I had a moment of clear thinking, and what had been in the back of my brain for weeks finally started to resolve into the only action I had left.

I gave up.

I remember feeling relieved and ashamed in equal measures as I packed my stuff into a cardboard box. I sat down at my computer for the last time, and I composed a letter to my boss. I printed the letter, put it in an envelope, then put it on his desk. My girlfriend and I left the office, and I set the alarm for the last time, locked the door, then posted my keys back through the letter box. We both got in the car, then I turned my phone off and that was that.

Over the next few days, of course, I had plenty of calls. My employer started off by asking me to come in and talk, and when I refused that turned into anger and frustration. The client called me up and offered to pay me extra (and well!) to finish the site, but none of this really mattered to me. Looking back, I think I had burnt out, which is a pretty sad thing to have happen to you at 20 years old, made worse by the fact that it was entirely my own fault. I'd let a lot of people down, and I felt about as low as I ever have done, before or since.

None of this is anything to be proud of. This is a tale of naivety, laziness and stupidity. This isn't a tale about the perils of purchasing software from outsource firms in India. If it were, I would have written about this years ago, but while the circumstances are personal to me, maybe there are a couple of lessons buried deep within here which other people will find useful.

The more I work in this industry, the more I realize that you can't expect to coast and produce results. Writing code for a living is hard work, and programming is an industry in which action and results speak louder than bullshit. I made some really poor decisions, most of which could have been avoided or resolved by just communicating the problems with the project to the people involved. I have no doubt that assistance would have been available to me if I held my hands up and said "I can't do this". But I'll never know now.

I spent a long time regretting what happened on that project. For a few years, I told myself this could have happened to anyone. I post-rationalized everything that happened, and found a way to pin the blame on everyone but myself. I think I always knew that is was my fault, though. I found it very hard to let go of what happened. Every single problem that I faced on that project was brought about by a combination of pride and lack of communication. I didn't want to admit I needed help.

In the end, I think that lessons about communication are hard-won, and nearly impossible to teach. It is easy to find yourself in a situation where you have to work incredibly hard just to keep your head above water. As soon as the first fears creep into your mind about failure, that is when you need to address them. Ask for help, and it will come. Since I realized this, I've not made the same mistakes again, and I hope I never will.