Sunday, January 20, 2013

Developer Metrics

In this blog post, I'm going to talk about developer metrics.  What I need to know is how much work a programmer can produce and how many man hours am I getting from this programmer per week.  The purpose of these metrics is to use them to estimate how many man hours it'll take to complete a project and predict when the project will be completed. 

Collect raw information

I currently require my programmers to track their time and tasks (I don't track start and end times, just time spent).  At the end of the month I have a list of what was completed and how long it took to complete.  I also have a list of how many hours were dedicated to programming, how many were dedicated to bug fixing and other tasks.  My programmers are only required to track time to the nearest hour.  This gives a less accurate, but easier to estimate total time.  Now I take each category of their time and divide it by the total hours they have worked (by subtracting vacation time, personal time and holidays from their normal 40 hour a week schedule).  This will give me the % of time spent on each task on average.  I use this only for predicting deadlines.

In order to determine their production rate, I have to rate the tasks they've done into difficulty.  There are a hundred ways to categorize software tasks.  They all depend on how you divide your tasks.  Let's just skip the details on this and admit to ourselves that this part is really subjective.  The trick here is to stick to the same "ruler" when measuring performance in the future.  Once I have a list of tasks with difficulty ratings, I can determine how fast each programmer can perform each of these categories of tasks (assuming I have given them at least one of each category of task).  As time goes on, I average these together to get a more accurate picture.  Sometimes programmers improve (this would be normal), and I have to adjust my numbers to match.  The point is, I need the production rate of each programmer in order to estimate software development times.  I also need these numbers to determine how many programmers I need on a team to complete a project by a specified deadline.  In the examples below, I'm going to just ignore the difficulty part of this and pretend that all tasks are equally difficult to perform.

There is another method of estimating that is used by the Menlo Innovations.  Their method requires the programmers assigned to the team to do their own estimates.  I have used this technique in the past and it works pretty good.  Most of the time I don't use this method because I know my programmers really well, and I have a good feel for how long it takes them to complete a particular task.  My company is also a bit lax on deadlines.  I set a deadline to make sure that projects get done, but we're never under any serious pressure to meet that deadline (other than my own self-imposed deadline).  If your company requires a hard deadline, then this method puts the pressure on the developer to make sure they estimate their own time and stick to it.  It also may require some negotiating between you and your programmer (otherwise, they could technically, estimate a crazy long amount of time just so they can slack off).

A product that I have been eyeing for quite some time is FogBugz.  This software has an estimating tool that can track performance of each programmer and adjust future estimates to reflect actual times that programmers took on previous projects.  They call this Evidence-Based Scheduling.  This product is a bit pricey for my current development schedule, but I'll be using it in the future if our development cycle gets too hot.

Keep your statistics

OK, so now you have a few numbers.  Let's say that you have 3 programmers and their numbers break down like this:


OK, first, I need to mention that we're going to pretend like the time dedicated to each task is just a function of their job in the company.  If your company has dedicated developers, they might spend time fixing bugs, or you might have a dedicated team of programmers that do bug fixes.  In this example, I'm just trying to show that it's rare to get 100% utilization out of a human and attempting to estimate time schedules with that expectation is going to result in great disappointment.  The "other" category might include things like office paperwork, meetings, etc.

Apply your statistics


User your programmer development percentage to calculate the daily rate in hours (i.e. devtime * 8).


Here is an example list of projects and their rough estimates:


Now we want to know how many work days it will actually take to complete these projects.  Just use the daily rate of each programmer to determine how many hours per day will be completed.  For this example, I'm going to just assign all three of my programmers on this project (I'll assume it can be neatly divided, or that I'm just getting a rough estimate).  So the daily hours of my 3 sample programmers totals to 15.2 hours.  Divide the total hours by that number to get the total days:


At this point, you can just round your estimates to the nearest day.  To obtain a deadline date, you will need to drag out a calendar and "x" off the days from the starting day to the number of days in the above spreadsheet.

Summary

The important facts to remember here is that we need raw numbers from the programming staff.  From these numbers the daily production rate can be calculated.  The daily production rate can be used to produce an estimate of days to complete each program on your list.  This can be used to prioritize projects or just to determine if your department needs more resources, or maybe just to determine the schedule of deployment dates for each project on your list.

I hope this information is helpful.  Drop me a comment if you have questions or other ideas.