Many software companies have adopted the idea of using forums for support.  One of the issues they face is getting everyone to go to the forum.  One mechanism I’ve found that really helps is to put a link back to the forum right on your app in some prominent place.  Make it a bit of a teaser.  For example, in my G-Wizard application, we call the support forum the “User’s Club.”  There’s a contrasting button for it at the top right of the window:

This attracts a very high percentage of users to go join the User’s Club and at least check it out before they have any problems, which is a good thing.  Aside from the user’s club, there is also an indication of whether the software is up to date that changes to a link to update the software when newer releases are available.

Debugging as Code Review

Posted: January 10, 2011 in Uncategorized

Bugs are never where we expect to find them.  If they were, we wouldn’t have made the bugs in the first place.

Most are not so hard to find, but periodically you will find a stumper.  These are bugs that are especially not where you expect.  They may be hiding in plain sight because you’ve convinced yourself things work differently than they do and are ignoring the bug as a result.  They may be in some nook or cranny that is wholly unexpected.  Object oriented and event driven programming seems to excel at giving bugs increasingly novel places to hide.

You’ll know you have a stumper from your soaring frustration level and the fact that finding and fixing it is taking far longer than you had expected.  When that happens, stop.  Start doing a code review on every portion of the program that might even remotely be associated with the problem.  You may not find the problem, but you will be surprised at how many other problems you do find in the process.

I finished up some work today on my G-Wizard software and went to commit back to Github and low and behold, no sign of eGit anywhere.  WTF?!??

No trace of it ever having been installed, and I have been nowhere near intalling or uninstalling anything in Eclipse since I can remember.  I have a vague recollection of it doing some sort of update quasi-recently, and that’s it.

So, I went and reinstalled it, and still no menus in Eclipse.  Clearly something is incompatible with something.

It’s times like these that I really hate Open Source tools.  They’re like the way the military refers to helicopters:

A bunch of parts flying in a loose formation.

As in, they don’t necessarily really work together all that well, all the time.

So, whatever.  No time to debug it.  Not productive.  I’ll just use git manually for a while from the command line and see if things recover down the line.


Nice NoSQL Overview, Heroku

Posted: September 13, 2010 in Uncategorized

Enjoyed this roundup of the NoSQL world from Heroku.

While I am fascinated by the topic, I am at least somewhat convinced that NoSQL is a premature optimization.  After all, I’ve seen some pretty high transaction levels (2m users or more) being handled by MySQL without even so much as Memcached for help.  Granted, web properties aspire to be way bigger than that, but unless you are blessed, and most aren’t, it’ll be a long time, if ever, before you get there.  We’ve also seen quite a few large web properties work just fine with MySQL given the appropriate discipline, which mostly boils down to eschewing the features (transactions, highly normalized schemas, yada, yada) that NoSQL doesn’t offer anyway.

All things considered, I wouldn’t be too keen to bet a new venture on a NoSQL technology, though I’d be fine with joining one that was already in motion and had proven their choice works.  I guess I want to hook up with someone that has the scar tissue and lived to tell about it before I risk my hide.

The US Army says that somewhere between 1/4 and 1/3 of soldiers can’t read maps and can’t be taught to read maps.  This metric holds true in times of draft or the volunteer army, so it isn’t just a function of who would want to be in the army in the first place.  It’s been a long time since I read that, so I have no link, but there are related links out there and I have no doubt that it is true.

Interesting.  An entire cognitive ability is missing from some fraction of the populace.

I have read from much less reliable sources that it is possible to teach anyone to have perfect pitch if you do it at the right time in their development while they are still very young.  As I say, consider that one unreliable, but it does make you wonder if the people who can’t read a map could’ve been saved if only we’d reached them early enough as children.

So it is with programming.

The vast majority of people cannot program and cannot be taught to program.

Consider Jeff Atwood’s post, The Non-Programmer Programmer, which put me onto this fascinating quote from Dan Kagel:

A surprisingly large fraction of applicants, even those with masters’ degrees and PhDs in computer science, fail during interviews when asked to carry out basic programming tasks. For example, I’ve personally interviewed graduates who can’t answer “Write a loop that counts from 1 to 10” or “What’s the number after F in hexadecimal?” Less trivially, I’ve interviewed many candidates who can’t use recursion to solve a real problem. These are basic skills; anyone who lacks them probably hasn’t done much programming.

I’ve said for a long time that great programmers are born not made, but I hadn’t really concluded it was at the level of the Army recruits who can’t read a map.  I’ve had PhD Computer Scientists work for me that couldn’t code their way out of a paper bag.  These were people from great schools including MIT and Stanford.  At the same time, I’ve known programmers with nothing but a high school diploma who were among the very best I have ever worked with.  I’ve know old programmers (luchadors, no doubt), who were kick ass coders.  One of my university professors joined my first startup and wrote the Modula-2 compiler and toolset we wrote all our software on single handedly while he was no spring chicken compared to the rest of us kids.  I’ve known youngsters with no apparent experience whatsoever who did amazing things.  One of my QA hires at a startup showed up in my office after one of our morning meetings.  I’d been asking the team to scour their network of contacts for someone who could build us a scripting language for our product.  He wanted the job.  After barely suppressing a desire to laugh in his face, my inner dialog said, “What harm could it cause?”, and I gave it to him with the proviso that we would keep looking to hire and when the new guy came on, this QA guy would apprentice with him to learn programming.  We never got the chance to hire the new guy because this “QA guy” got what I had budgeted as a 6-8 week project all built, and built extremely well, in about 30 days.

So how do you find great programmers?

I’ll give you my standard recipe, which I have recited at endless interviews and put into practice throughout my career.  To find great programmers I look for people who’ve done something great.  They need to have completed at least one posting working on a product that I respect. Their role needs to have been to produce something essential for that product.  I will start out looking for this on resumes and in referrals.  I will confirm it by talking to the candidate.  And, most importantly, I will get in touch with the team that developed the product and find out what they think of this developer.  Did they play a pivotal role of some kind?

There are some corollaries to the scarcity of great developers to consider.

First, I don’t like to hire entry-level developers.  It’s too hard to tell whether they have the programmer gene, and it is just as easy for an entry level developer to screw up a release cycle as a veteran.  Back in the old days, when I was a young luchador, we used to put guys like this on install programs.  Well folks, you can wind up shipping late or delivering a lousy User Experience if your install program sucks too!

Second, if you are an entry-level developer, go build something.  We live in a time where you can contribute to Open Source or just build something amazing yourself.  When I was in Grad School, I did three things that got people’s attention:

–  I consulted on CAD/CAM software.  I was porting pretty big company’s CAD products from PDP-11 and PDP-12 to the then-new Motorola 68000 workstation world.  I got my first job through a random set of contacts as a sort of apprenticeship that blossomed.  So check your network and go get some consulting.  Don’t charge them unless you succeed. 

–  I built a screen saver for our school’s Sun workstations (brand new at the time and the graphics library was written by some odd company I’d never heard of called Lucasfilm) that showed where the stars and planets were in the sky based on current time and location.  It constantly updated.  Other than the lack of augmented reality, it was not unlike the very cool Star3map application for the iPad.

–  I built a faithful clone of the Apple Lisa desktop UI for the Suns.  At the time, the workstations came with the C-shell in a window, which just seemed incredibly siller for a megapixel display and a mouse.

Lots of people saw those three pieces of work, which were completed in a remarkably short time (less than 18 months), and that was my resume.  That got my first startup funded, in fact.

Third, think about the ecosystem that is developers and the impact of this scarcity.  Once upon a time, corporate IT built all their own software.  Now they build very little.  The suits will tell you because it’s too expensive, it’s not our distinctive competency, yada, yada, yada.  BS.  It’s because there weren’t enough real programmers to go around.  By this time, it isn’t clear there are enough for IT, or for VARs and SI’s like Accenture.  So they started to cluster at ISV’s. 

There is no monopoly Silicon Valley or anyone else has on great developers, but they are mobile.  They vote with their feet.  They will go where they are valued the most, even if they have to travel across oceans and move their families to do it.  Maybe that’s why there are very few places that have enough of them to have built very many great software products.

Fourth and last thought.  While I don’t believe in hiring the entry-level, whether you do or you don’t, maybe a programming test is a good idea.  Just because a lot of smart people out there can’t program at all.  It’s not their fault, they just don’t have the gene.  They may not even realize it.  I’ve been surprised at how far I have seen some people go cutting and pasting code.  Such people have passed my “worked on something significant with a pivotal role as perceived by co-workers” test.  Maybe they’d pass the simple programming test too, but I think it does no harm to try. 

I would focus on a simple test and forget making it a brain teaser.  Don’t go all elitist.  Something a little more than Hello, World and a lot less daunting than reimplementing an entire arithmetic library using only increment, decrement, if, and recursion (been there, done that).  Forget about why man hole covers are round, too.  Use a test like Dan Kagel’s counting from 1 to N.  If you want to get fancy, ask them to print the first N prime numbers.  But not much more than that. 

Use the test as an ice breaker.  For their first contact, sit them down in the conference room.  Make sure the white board has the test all written up.  Give them a laptop with an editor.  Tell them to accomplish the task on the board in their favorite language and send the result to a particular email when they’re done.  That email is their first interviewer, and that person should review the result and think about where they want to go in the first interview.  I can’t imagine a good programmer being offended.  It’s almost humorous if the test is so simple.  The reasons why (because most can’t program at all) is an interesting conversation starter.  The thing is, great programmers can’t help but enjoy programming even simple stuff.

Another thing I like to do if they do well in the first round, is bring them back to present some piece of work they’re proud of.  Do a deep group code review.  Get them to present the architecture, why they made the decisions they did, and walk the team through it.  It’s important to gauge their communcation skills too.

I decided to start /dev/luchador with a repost from my Smoothspan blog.  Call it a common touchstone from which the two can diverge over time.  This blog is intended for posts that are too technical for Smoothspan or just not right for CNCCookbook (an audience of machinists there, not software people).  It’s not a particularly technical post, but it should resonate with the developer audience /dev/luchador is intended for.

I love that line from Paul Graham’s post about what went wrong with Yahoo:

In technology, once you have bad programmers, you’re doomed. I can’t think of an instance where a company has sunk into technical mediocrity and recovered. Good programmers want to work with other good programmers. So once the quality of programmers at your company starts to drop, you enter a death spiral from which there is no recovery.

That observation is so true, as is this one:

Theirs was not to reason why; theirs was to build what product managers spec’d.

That’s in reference to Yahoo’s totally Product Managment-driven decision process.  It has seemed to me at times like Microsoft errs dangerously on this side too.  Product Managers are essential, and can be great, but if you place in a role where their job is to hand down stone tablets for the developers to slavishly implement, it doesn’t work.  It is not empowering to the developers who have tons of great ideas, insights, and vision themselves.  But ultimately, it’s just not a good idea to have guys in Ivory Towers thinking great thoughts they have no responsibility or accountability to deliver on.  It’s certainly not much of a team effort when things work that way.  I see great Product Managers as being the ultimate Customer Advocates.  Who else can say that they spend 100% of their time understanding the company’s customers and translating that into product requirements?  The trick is to balance those requirements against other sources of requirements, for example the need to innovate and differentiate, which far more often comes with vision rather than customer feedback.  It takes both as even the brilliant Steve Jobs discovers when his visions get too much at odds with what customers want.

The final great point from the post for me was:

In the software business, you can’t afford not to have a hacker-centric culture.  So which companies need to have a hacker-centric culture? Which companies are “in the software business” in this respect? As Yahoo discovered, the area covered by this rule is bigger than most people realize. The answer is: any company that needs to have good software.

Hacker culture often seems kind of irresponsible. That’s why people proposing to destroy it use phrases like “adult supervision.” That was the phrase they used at Yahoo. But there are worse things than seeming irresponsible. Losing, for example.

Yes indeed, there are worse things than seeming irresponsible.  Though I think the appearance of irresponsibility is only an appearance and comes from suits not being able to understand the hackers.

So let’s say you’re running a company that has Bad Programmers.  Are you really doomed? 

It’s an interesting problem to think about fixing it.  How did you get there to start with?  Here are some thoughts:

–  Like Yahoo, you had a culture that didn’t value Technology.  Great programmers can smell that a mile away and will general avoid it.  I have seen cases where most of a company didn’t value the developers much, but there was an elite core group where great developers could flourish.  It’s often hard for overly sales-driven cultures to value developers as much as they should.  As one friend put it, sales-driven cultures see the Customer as God and Sales as the Church.  There were certainly elements of this at work for Yahoo.

–  You outsourced or isolated your Developers.  This is not a guarantee you have Bad Programmers, but it is a virtual guarantee they’re not really plugged into your corporate culture the way they should be.  Developers can smell this too.  Sometimes they like being off by themselves, but its too easy for it to turn into a Stone Tablet scenario at which point they’re unhappy.  The opposite extreme is the Dev Group that has no customer contact and is just implementing whatever they feel like.  Also bad!

–  The fish rotted from the head.  Bad Leadership is a problem for Developers.  They have little patience for a lot of politics or a weak leader that’s easily snowed.  Dilbert is not a recipe for a great Hacker Culture.

–  Lazy hiring.  You started out with some great developers, but you did a poor job hiring and building an organization.

–  Lousy process.  Smart people abound, but the processes in use are not productive, or worse are destructive.  This is usually a function of poor change control.  If left to fester, you can get spaghetti code from the brightest of teams.

So back to the question of fixing it, and while we’re at it, let’s also look at avoiding it in the first place.  Consider these proactive steps:

–  Make sure your corporate culture values what Developers do.  They don’t call them Software Companies for nothing.  They’re not Sales Companies, Marketing Companies, or Finance Companies.  Make sure you act like it, while also recognizing the huge value these other groups bring. 

–  Keep the Developers well plugged into your corporate culture.  If they’re remote, work overtime looking for ways to make sure they’re still plugged in.  Rotate them through corporate.  Send the right people out to the remote locations frequently.  Never let it turn into a case of micromanagement from afar (Stone Tablets) or no management at all.  Make sure there is strong leadership in the remote locations and make sure the leaders there most of all are plugged back into corporate.  Let me tell you, this will be a lot of work.  You outsourced and offshored to save money.  Welcome to the first of many hidden costs that will make it seem less attractive, though not fatally so.

–  Get great Engineering Leadership.  You need leaders who are inspired, visionary, extroverted, technical, and downright fun to hang around.  They need to be great not just with the developers, but with their peers in the other functional areas.  The leadership, more than anything, has to move the cultural values both ways across the blood brain barrier that often separates Developers from the rest of the culture.  They’d better speak both languages (Geek and Business) to do that well.

–  Realize that Hiring is critical from day one, and it never stops.  I could write many posts on hiring, but I will leave it to one simple observation.   99% of the time people seem to regard hiring as a temporary distraction from what’s really important, such as delivering a product.  But hiring the wrong people will cause you to deal with more kinds of Hell for longer than any architectural mistake you can possibly make.

–  Keep tuning the process.   Start out with the obvious Best Practices, but don’t rest on your laurels from there.  Development organizations have two responsibilities: delivering Great Products and building a superior Software Factory.  They spend most of their time focused on the Products, but like Hiring, the Software Factory is not to be overlooked.  Any given product release is just a battle in the war.  If your Software Factory releases faster than the competition, if it builds better product every cycle, and if it does this more cheaply than the competition, it’s a huge advantage.

One last piece of advice.  If you already have a lot of Bad Programmers runnning around, find a way to isolate them.  Don’t just beg Good Programmers to come in and drop them piecemeal into a pit of Bad Programmers.  They won’t be effective and they won’t hang around.  Start out making sure you have great leadership, and build new groups with Good Programmers.  Preferably do this product by product, and through acquisition of Teams that are known good entities who know how to work together to accomplish great thinks.  Don’t hire Good Programmers and term them into Bad Programmers inadvertently, or drive them away altogether.