Thursday, November 29, 2012

Database Design: Part III

As promised, I'm back.  Part II covered scheduling and estimating our sample project.  What I didn't cover is the final part of this project, the meat of this example where the customer keeps their account information. I'm going to expand the overall web map to add in the new edit account interface:


The assumption here is that the "Select Customer" web page is the same page used for both purposes.  Nothing in this diagram indicates that it is the same web page and it can be built as two distinct pages, but any opportunity to reduce code should be looked at.

Now the schedule and estimate need to be updated to account for these two interfaces and I'll leave that for another day (as well as the actual web page design for the "edit account data" web page).  What I want to do right now is show that there is more than one way to skin a web server...er, I mean build a web site.

For instance: Should we use a select web page, or just include a drop-down at the top of the edit page.  If a drop-down is embedded on the same page as the data is displayed, then a decision must be made over what to do if there is no data in the system and what to display in the details before an account is actually selected.  So we just display the first persons data?  Maybe we should stuff a "-- Please Select a Customer -- " item in the drop down that indicates that no data is displayed until the user selects an actual user from the list. 

What about the customer edit page?  If we use a drop-down list on this same page for the customer name, then how do we handle changing the customer's name?  What happens if the customer list grows to thousands of customers?  Duplicate names possible?

This type of pontification should occur any time you create a design.  You should make a list of notes for everything that comes to mind while you are designing and when you review a design.  When you are done with your design, you should have a huge list of questions that need answering.  Answer these questions before starting any code.  Have your best programmer look over your design and make him/her ask questions.  Run it past marketing/management, see if they have questions.  Get answers before starting to code.

Next, you need to ask yourself questions about your database design.  Do you want to have records deleted from the database when the customer is deleted?  I'm assuming no.  So if we want to maintain some sort of history, do we just flip a flag in the customer table and filter out those records in the main screens and design new screens to view history information?  Or do we create history tables and copy those records to the history table before deleting them from the primary tables.  In the first example, the data in each table will continue to grow forever.  In the second example (using history tables), there is a possibility of data loss unless transactions are used.

For any data entry web page, we need some sort of data validation.  How to handle duplicate data input.  Maybe the customer number needs to be assigned by the computer when the user is created the first time.  Then this number should never be changed.  That would prevent the possibility of duplicate numbers.  If there are multiple web users entering data into this database, then the unique customer number should be generated by the database and not the program.

Do the web pages look aesthetically appealing?  Is that important, or is this just a utility used by your company's employees?  What about access to the website?  Is this going to be used by customers, or only by the company?

As you can see the whole process of designing software can take multiple iterations.  The bigger the software package, the more iterations you're likely to encounter.  Rushing into the coding phase of any software project will typically end in disaster, or at least regrets about something that could have been done better if only the right questions were asked.

One other point here is that a software package can be "grown".  If you choose to grow an application, you must first decide what the most important features are going to be, then build those first.  Multiple iterations can be used to expand the features of your software using this technique.  If you've never heard of this technique I would refer you to SCRUM extreme programming.  Here's the wiki page and the graphic in this page pretty much sums up what I just described.

That's all for now.  I'll get expand on this subject in the future.

Doing Research

As an IT manager it's my job to stay on top of things.  It's impossible to know everything about the computer field, so pick your battles.  I usually research subjects that I believe will improve my company standing among our competitors.  Of course, I keep tabs on my competitors, everybody should.  Otherwise, one day, you wake up and your customers are running for the door and you suddenly find out that your competitor has launched a killer product based on a technology you never heard of and it has a coolness factor that attracts your customers like flies.  Oh yeah, anybody that has been in the IT business for long has experienced that sinking feeling before.  That feeling, when you suddenly realize your company is at least 6 months behind your competition.

To avoid this problem, I like to keep tabs on my competition and do a lot of research in my field.  What am I talking about?  One example is knowing what is out there.  My current company surveys roofs, collects raw data, then produces budgets and construction plans.  That's the short-short version of what we do.  In reality, there are a hundred little pieces that are performed from step 1 to step N.  One of the "features" of our software that our competition is yet to catch up on, is the dynamic building viewer interface.  Our customers can log into our website and view their roof information visually.  The roof areas can be color coded according to data points that they choose.  Similar to what ARC View does.  The way we do it is we parse AutoCAD data into our database and form the correct polygons, text, symbols, etc. and they we use this data to reform the output in PDF format (for reports) and using a viewer.  In the past Adobe provided an SVG viewer plug-in and we ran with that for a couple years (until Adobe announced they were ceasing support for the plug-in and FireFox still did not have SVG capabilities).  So we stumbled onto Microsoft Silverlight.  We've been using Silverlight for about 4 or 5 years now and we've been lamenting the problem of plug-ins not working on the iPad and iPhone.  Which leads me back to my original title: "Doing Research."

I initially looked at HTML5 canvas.  I started a test projects to see what features can be ported from our Silverlight code into the new canvas.  When I discovered that Silverlight produced a static output (i.e. the lines, ellipses, squares and polygons don't exist after they are drawn into the buffer) I knew I had a problem.  Then I re-visited SVG, which to my delight is available on Chrome, IE 9, FireFox, Opera and Safari (not to mention Safari for iPad and iPhone).  There is an additional benefit, no plug-in required.  So it's back to SVG!

Another example is the ability to import and export data into our database.  Excel is one of the obvious choices and we use the Apose product for excel input and output (this is a natural choice since we are using C#).  One of our newest enhancements is regarding our bid forms.  We have customizable bid forms for roofing contractors to print out.  Typically, a contractor will receive a user id and password from our company when they are invited to bid.  They log into our system and type their bid prices into the project that we have setup.  Then they submit their bid (which locks out their ability to change anything in that project from then on) and they can generate a bid form in PDF format.  Aspose also produces a product called Aspose Words.  This is an MS Word import export library that can also produce PDF output.  So we have created a WSYWIG interface allowing our designers to customize the formatting of our bid form and use special tags to insert real data into the text.  The roofing contractor clicks a link that runs the Aspose Words library to generate a PDF output of the data that was previously uploaded by the designer using MS Word.  If you're interested in these third-party libraries you can go to Aspose.com for more information.  They make Java components as well as .Net components.

Another product I'm researching is iText.  This is a PDF library.  Our reports are written directly in C#.  I've used a lot of report generators in the past, many different versions of Crystal Reports and I have to admit that they are lacking.  I see the appeal, but my department tends to write very complex reports and it seems to go smoother using a PDF library that we can just paint onto rather than trying to get around a rigid report generator.  We are currently using PDFLib, which has served us well in the past, but iText has some features that make things easier (like mixed styles in one paragraph).  iText can be found here

This is but a small sample of the type of research I do on a weekly basis.  Sometimes I'll stumble across something that I'll assign one of my programmers to research.  Typically, I'll make them write some simple demo program to see how feasible the product is and how we can incorporate it into our product.  Sometimes weaknesses will present themselves.  Sometimes I am sufficiently amazed at the technology to suddenly change direction of our product (which normally occurs after I "sell" the idea to marketing/management).

So never stop researching.  Don't get left behind by your competitor!


Book Recommendation

If you're looking for a good book on dealing with legacy code, "Working Effectively with Legacy Code" is my recommendation.  This book is organized in a way that allows you to identify the problem you are having in converting your legacy code into unit testable object oriented code.  Each type of difficulty has it's own section and you can skip around, or as I've done, read straight through.
 

The entire theme of this book is that legacy code must be modified or refactored to work with unit tests.  The reason for incorporating unit tests is to make sure the code is verifiable when completed.  Verifiable code is easier to modify later because the unit tests can be re-executed after the modifications to make sure the original functionality still works as expected. 

The author starts with a legacy code change algorithm (copied without permission):

1. Identify change points.
2. Find test points.
3. Break dependencies.
4. Write tests.
5. Make changes and refactor.

Then he proceeds to provide chapters for performing these tasks in situations that a programmer might run into if they are attempting to modify legacy code.  An example is: Chapter 9, I Can't Get This Class into a Test Harness, or Chapter 23, How Do I Know I'm Not Breaking Anything?  Those two examples are part of breaking dependencies.

If you're interested in this book, I would recommend going to amazon and thumbing through their "look inside" feature (click here).

Sunday, November 25, 2012

Database Design: Part II

This is the second part of my software design article.  I mentioned in my last post that I'd do some coding.  Before I get to that part, I need to explain a little more about "why" we should build software in the first place.  There are only two reasons why a business should build software:

1. It adds value to the company product.
2. It makes the company more efficient.

There really is no other reason to build software.  Now you're probably thinking, "Gee, a program I built last year makes Jill's job easier."  Easier is nice and it can be quanified as making the company more efficient, if it reduced the amount of time it takes Jill to perform a task, or it allows Jill to perform more work in the same amount of time. 

All right, so this is all Econ 101 stuff, but I thought I'd point it out to remind everybody why software is built in the first place.

Next, I want to remind everyone that building software costs money.  It doesn't matter if the person building the software is already working at the company, or if they are full-time saleried employees, it costs money to pay that person to write code.  So there is a cost associated with every piece of software built.  Also Econ 101 stuff.

In the first part of this blog post I dug right into a software design.  Technically, this design will only be performed if someone in charge of the company approves of it.  If this is the idea of an IT person, then that person needs to get "buy-in" from management.  Then the detailed design needs to be completed, and finally an estimate and maybe a schedule must be created.  Keep in mind that management is going to want to know how many programmers will be dedicated, what hardware and software needs to be purchased, how many total man-hours are going to be expended and when will it be completed.  This will boil down to a total cost in dollars and a delivery date that can determine if this project will be delivered in time to be of use to the company.

OK, so how do we develop an estimate?  I've read a lot of books on estimating software and some books get into intricate details of estimating how long it takes a programmer to write software.  My experience is it depends on how well you can break the problem into definable chunks.  If the chunks are small enough, then a rough guess is good enough for each chunk and any error in estimating should come out in the wash.  Also, it just takes years of practice.  Knowing the ability of your programmers is good and understanding which algorithms are going to be complicated and time consuming and which ones are going to be easy is just good ole experience. 

So let's do an estimate for the parts we've already designed.  I know that this project is trivial and it could be built by any seasoned programmer in a day or so, but I'm going to "pretend" like this is a big production and we are going to employ some resources to building the software.


There's the estimate.  In reality, as I said earlier, it would probably take a couple hours to throw something trivial like this together, but I've made it into a big production and rounded each task up to the nearest hour.  In more complex applications you can break down each web page or screen into multiple tasks.  You may also notice a line for integration testing.  When everything is complete, you'll need some extra testing time for testing how the web pages interact with each other.  If you prefer, you can add in beta testing.  How much beta testing you perform will depend on how complex the application is and how high you want the quality to be.  If you're using test driven development, then each web page should be mostly tested before they are integrated together.

Anyway, now we have an estimate.  Next, we need a delivery date.  In this trivial example, we estimated 13 hours total, so that equates to two work days.  Or one work day for two people, assuming it can be coordinated between two people.

This project will not require additional hardware (I'm making this assumption) or software.  If you were working for a startup company, and they don't have any computers yet, you'll have to estimate the cost of a web server, the cost of the developer's work station(s) and the development software that will be used (i.e. visual studio or equivalent).

There is still one piece missing.  Your schedule.  If you're going to use a gaant chart, you'll need to decide which tasks can be performed in parallel and which tasks are critical.  In this project I'll assume that the main web page must be completed before any other page can be created.  In reality, any web page can be created in a vacuum and then they can be connected at the end.  But that would be a boring gaant chart and I wouldn't be able to show an example.  I'm also going to assume one programmer will spend two days on this project.


OK, now we have a simple gaant chart.  If you have Microsoft Project 2010, you can copy data from excel into Project and link the tasks together.  If you're using viso or another project management tool, you might need to re-type your tasks into the software.  Either way, this tool is what management will want to see.  It shows that this project will start on the 26th and end on the 27th of November.  It shows that the critical path includes every task in the project (some projects can have multiple paths and the longest path will be the critical path).  This chart shows that the previous task must be completed before the next can be started.  If there were two people working on this project, we might restructure this project to look something like this:


I have used P1 and P2 to designate programmer 1 and programmer 2.  Microsoft Project has a more sophisticated resource management capability, but as I've mentioned before, this is a trivial example. 

If you look closely at the second gaant chart, something interesting has happened here.  It will still take two days to complete this project.  But, only programmer 1 will be working on the final task on day 2 (and technically, he/she will only need half a day to do it).  This is why a CPM schedule can be helpful in determining schedules.

That's all for now.  Next I'll get into some details of what questions you should be asking as you build an application like this.



Saturday, November 24, 2012

Database Design

I'm going to design a database as an example.  This is a very tiny project, but I'm going to use it as an example of what to do and what not to do when designing database connected software.  On with the story problem...

You work for a new bank.  Your job is to write a program that tracks saving accounts for any number of customers.  I'm going to make this painfully simple and ignore stuff like interest and user authentication.  So don't expect to use this in the real world.

The first step is to decide what data to store.  Let's focus on the customer information.  For this example, we'll design one table with the customer name and a customer number: 


You can use any field name conventions you want, I just gave each field the smallest name possible so the queries are smaller.  At this point you might be wondering what the "pk" field is used for.  If you attended any database design course, you'll recognize the primary key field right off the bat.  The purpose is to designate a field in the table that is guaranteed to be unique, no matter what.  This can come in handy when we want to delete just one record since this field will not depend on user input and will never by viewed by the end user.  Only the software will use this field.  This will also be the field that will be used to connect to other tables.

Next we'll need a table to store the checking and saving account information.  I'm going to make this really simple, but it will not be very useful in the real-world.  Later, I'll expand this database to add features to make it more useful.  For now, we're going to need an account table:

 
OK, this table contains a primary key too.  This primary key is unique for this table only and can be used for the same purpose as the primary key in the customer table (to refer to or delete a unique record).  The "customer_key" field is what will be linked back to the customer table's primary key.  This is called the "foreign key".  I usually index this key because queries are going to join these two tables together using this key.  One example query will be:

SELECT
    SUM(amount) AS account_balance
FROM
    account,
    customer
WHERE
    account.customer_key = customer.pk AND
    customer.name = 'joe'

This query will get the balance of the account for Joe.  Now we can design the integrity constraints for this database.  Obviously, we're going to join these two tables together at some point.  We also don't want the possibility that there are account records existing for a customer that is deleted from the database.  So we'll need a one-to-many relationship between the primary key of the customer table and the foreign key of the account table:


As you can see from this diagram, each customer primary key (pk) can have zero or more associated records in the account table.  The associated records will have a matching number in the "customer_key" field.  In addition to putting a constraint on these tables I would recommend adding an index to the customer_key foreign key in the account table.

Now it's time to build some software.

One of the major considerations when designing an input screen for this database is that the customer must exist before you can create account records.  So the first screen will need to be a customer select screen before entering the customer account information.  Before a customer select screen can be tested, we'll need some data to display.  So it makes sense to design the customer add and edit screen first.

The main screen will look like this (I'm designing this as a web site):

 
The screen to add or edit a customer record will look like this:


When we edit a customer record, a customer select record screen must be used to get to the edit screen shown above:


OK, so now I'll diagram a map of the web site as it exists now:


Now you can see how these pages are all connected together.  At this point, there is enough design information to build enough of the web site to be able to maintain user accounts (minus the capability to delete an account).  These web pages can be used to populate the customer table only.  Constraints on the database, at this point, will not affect any software that you write (assuming you auto-generate the unique primary key for the customer table).

I'm going to end this blog post for now, but in the next exercise, I'm going to do some coding, and then I'm going to move on to the account portion of this project.  When this project is complete, I'm going to ask a few questions that you should be asking yourself when you design software.  I'm going to basically, critique my own system and show that there are many considerations that can be taken when designing even the simplest program.



Designing Database Driven Applications: The basics

I'm going to talk about the basics of designing an application that connects to a database.  I'll assume you already have an interface idea and some notes on what to store in the database.  I'm also going to assume this is your first experience (i.e. for those who just came out of college or this is a senior class project of some sort). 

The first step is to design the tables that will store your data.  You'll need to draw an Entity Relationship Diagram (I'm betting you thought you could just forget that information after your database class final... LOL!)  Once you're confident that you have all the fields and tables designed you'll need to decide how the integrity constraints are going to be setup.  This is more important than the actual data fields.  If you forget a data field and have to add it later, it will not affect your code as much as a mistake in connecting two tables together.  So focus your efforts on getting the constraints correct!

After your relational integrity constraints are designed, you can create a test database.  Create all your tables and make sure you create all the constraints.  You can add indexes at this time, or leave them off until the end.  You'll need to analyze your data once you're done with your software in order to determine if you need additional indexes anyway.  Normally, I'll index all foreign keys and any fields that are specifically used for sorting (like fields I named "sequence" or "order", etc.).

Now you can design and build the software.

OK, why did I stress this ordering of events?  My experience is that programmers will make mistakes (yup, it happens).  The best way to make sure that the final software will not crash because of an integrity error, is to create a database with the constraints first.  Also, it's easier to change the program while writing the initial copy, than to add the constrains after all the software is written.  Which will require you to redesign the software to make it work with the added constraints.

Make sure you have all the foreign key constraints you need before finishing your project.  If you miss one, it will show up later when you query your data and records drop out, or you total records and end up with a larger number of records than you see on your program's list (these are known as orphaned records).

I'm talking about this subject because I have experience with it.  I inherited a database containing more than 400 tables designed by a person with no formal database experience.  This database did not have any integrity constraints.  When I ask the "programmer" why he didn't put any constraints on his database, he replied that he tried, but it just broke his software, so he left them out.  Apparently, he saw no reason for constraints, I slapped my head and decided that I better check the data.  It didn't take long to discover all types of data integrity problems.  Attempting to add constraints and fix the software at that point was an uphill battle and the entire system was rebuilt from the ground up.  I wouldn't recommend rebuilding an application because of missing constraints, but the lack of knowledge in database design also translated into a lack of knowledge in software design.  I only wish that he could have spent the time to get the database part right so I could have redesigned a front-end to connect to the existing database and not worry about constant database problems showing up as bad customer reports.

Maybe I'll do a subject on basic database design and stress the important stuff that is not taught in colleges or tech schools.

Microsoft Surface

So Microsoft has a new toy (announced back on October 26th).  It's called the surface (see here for more info).  As an IT manager, it's my job to find out about any potential paradigm shifting devices on the market.  Sometimes it's a joy, sometimes it's a headache.  I haven't decided which type of device the surface is (joy or headache).  I have determined the following basic information about the product that makes me scratch my head:

There are two models, one with Windows RT and one with Windows 8.  Initially, I thought that they were both the same hardware that could have two different OS's installed.  Nope.  They are different CPU's and the RT model cannot run Windows 8 programs.  The Windows 8 version will have slightly different dimensions (It will be thicker and heavier) and will not be available until after Christmas.

For specifications on the Windows 8 version I will refer you to the CNET article here.  This article mentions that it will ship early 2013.

My initial reaction that Microsoft was going to produce a tablet got me a bit excited.  I assumed that the reason Windows 8 had that added touch device interface was to compete on the smart phone market.  But a touch pad that can run windows apps?  Why that was like a thin laptop.  As soon as I found out that there was going to be a version called RT, I was a bit confused.  I suspect that confusion is what will happen to a lot of people who expect the new tablet to run windows apps and find out it doesn't.  At least, not the one they are selling before Christmas.  I understand the whole marketing angle behind getting it out the door before Christmas.  My issue is the fact that somewhere down the road there will be a mixture of these two tablets floating around and many people will not realize the difference between the tablet that they own and the other tablet.

To compound matters, as a software designer, I must make a choice going forward.  Do I create my apps to run on the iPad because it's currently number 1 and take the chance that it'll remain the market dominant tablet?  Do I write software to run on Droid devices and incur the pain of software that must adapt itself to a large variety of hardware with different features?  Or do I write my software to run on the surface, and which surface?

One other note: The RT version runs with apps from the Microsoft Windows Store.  I suspect the selection is currently small.  Maybe people will port their windows applications to RT and make them available in the Microsoft Windows Store.  Only time will tell.

That's my two cents.  I hope people take the time to research what they are buying.  Impulse buyers beware!

Sunday, September 30, 2012

Hardware Inventory

Hardware Inventory

If you are an IT manager and your department manages a network and/or computer hardware and software, then you need an active inventory.  Even if you are just tracking this inventory on a spreadsheet, you need to keep it updated constantly.  This may require some skull-cracking, but it is a top priority.  Your inventory "system" needs to track the purchase dates and costs of hardware and software, who it is assigned to and when it is expected to be replaced.  You'll need to assign a fixed number of years for each type of hardware and software (i.e. like 5 years for desktops and 3 years for laptops, etc.).  Once you have completed an entire inventory with expected replacement dates, you can produce a report that shows how much you expect to spend over the next 5 years and you should be able to report how much total hardware and software inventory you currently have on hand.

Another metric you might want to track is repair tickets.  My company is so small that my network admin/PC repair guy just fixes each machine as the complaint is lodged.  However, if your company has hundreds of PCs and you have one or more designated PC repair technicians, you'll need some method of prioritizing requests and measuring progress. 

Other options

As I've stated before, I work for a small company.  Unfortunately, over half of our computers are laptops.  It's the nature of our business.  To reduce our repair costs, we have used the 3-year on-site repair plan from Dell (not that I'm selling Dell computers, here, you can check into maintenance plans for your favorite vender and use that).  Then we replace our laptops every 3 years.  In other words, we never repair laptops.  Ever.  I like it that way, it's cheap, management is happy, the end user is happy when a technician shows up on-site and fixes their laptop right away.

That's all for now.  I'll go into more detail in a later post.

Enhancement Requests/Work Requests

Enhancement Requests/Work Requests

Everybody want a features.  There isn't a software package on this Earth that I don't desire an extra feature or two.  Every time someone uses my software, they think of a feature that they would like to make their job easier.  It's natural.  It's also frustrating.

Story Time..

When I started my first IT manager job, my company used to treat features as something that was necessary.  If an employee "needed" a feature, some programmer "needed" to jump on it and program it into our software.  This worked initially, when a small hand-full of employees used the software and the size of the software was small.  As the software grew into a huge tangled mess of conflicting features, things got out of hand.  Soon the number of bugs were uncontainable.  Then the complaints about how the software was changing every day started coming in.  To top it off, managers who expected their people to get work done at a fever pace expected features to be implemented in time for them to get that important Excel spreadsheet done by Friday.  It was ugly.  I had to literally change the culture of the company to fix this particular problem.

So my first step was to make sure there was a log of all feature requests and work order requests.  I had to separate these two to make sure that people knew that a work order was a custom output that the IT department did once per request.  A feature was something that was permanently coded into our software.  So there I was with a hundred features, many conflicted with capabilities already in the system, and some features would require serious programming hours to incorporate.  Then there was the "software changes too fast" problem to solve.  So I educated the company owners on the proper way to build software and what the pros and cons were to building new versions every week and building them once per quarter or once per year.  I also got buy-in from management that we would have an enhancement meeting to decide which enhancements would go into the next version.  This ensures that people requesting the enhancement didn't complain that I was playing favorites. 

What is the pay-back on an enhancement?

There are only two reasons you should ever incorporate an enhancement: It will reduce your production costs (i.e. enhancement "A" saves marketing 5 hours a week tracking customer data) or it add value to the product.  Value is something that makes your product more valuable.  If it's more valuable, then you can either charge a higher price, or you should be able to sell more copies.  Never incorporate an enhancement that will cost more to implement than save money on the bottom line or not add value.

Gilding the Lilly

I just love that term.  Gilding the lily means that you are gold plating your own enhancements.  Even developers will want their own enhancements and many times they'll spend fruitless time enhancing software to do what they want it to do.  Don't let this situation continue.  This is wasted man-hours for a feature that most users will not need. 

Next I'm going to talk about hardware inventory.

Software Development Progress

Software Development Progress

If you're developing software you will need to measure progress.  Management will need to know your progress.  They will want to know what to expect and ultimately how much it will cost.  You will need to know if your software is making progress or if you need to shut it down and cut your losses. 

Years ago (in the late 90's), I was promoted as IT manager to a company that had two IT departments merged into one.  The previous IT managers were sacked and I was left with a huge mess to clean up.  If I had to do it all over again, I would have done a lot of things different.  One thing I would have done early on is track software progress.  It took me a year to get a handle on what progress was being made on each of the software packages that we were producing.  This lack of measure in my department opened the door for office politics to get ugly fast.  Part of the problem with the company that I work for is that they were unfamiliar with the nature of software.  In their minds, software was just a big can of play-doe that can be molded into anything needed.  Which is how the previous IT managers treated it.  I knew better, but in order to communicate realistic expectations, I needed to get my documentation ducks in a row.  This included measuring. 

The first step is to nail down a design.  I won't go into details of software engineering techniques in this blog post.  In fact, there are entire blogs written about this subject alone.  Suffice it to say, that the more detailed information you can get down on paper the easier it will be to estimate how long it will take to develop your software.  Initially, you'll need to boil it down to man hours.  This can be a rough estimate at first.  Next you'll need to know what resources you'll have available for the programming, debugging, user manual, help file creating, etc. (whatever you'll need to complete before putting the product on the end user's PC, or publishing to the website).  You may have to present various scenarios to management.  They are going to want to know the overall cost of the project and that comes down to man-hours times cost per man-hour.  Once scenario is one programmer working non-stop until the project is done.  This is easy to manage and is normally done for small programs.  Most of the time, you'll need to use multiple programmers.  Don't forget about contract labor programmers.  Sometimes it's easier to hire a hand-full of programmers to develop the meat of your application and use your best programmers to manage and connect the pieces together.  What you will present to management is a time-frame, maybe a start date and a maybe delivery date.  I have discovered through years of experience that management will have a desire to set the priority of your development. 

Your project starts, now what?

So you presented your estimate and costs to management and they are excited about the product.  Now it's time to roll up your sleeves (or time for your programmers to roll up their sleeves) and get to work.  Each week you should know how much progress is being made.  You can do this by counting the number of completed features, or what percentage of a feature is completed.  Your technique is your choice.  I use the Menlo method of tracking projects, it's simple, it's effective and I can shut down a project in the middle and still end up with a functioning product (with only some of it features, but a functional product none the less).  There are many methods of running a software development project, I won't go into details here.  At this point, I'm assuming you're a new IT manager with limited experience and you need to get the ball rolling.  Anyway, you'll still need to track the total percentage of completion of your project.  This needs to be tracked at regular intervals and presented to management so they don't begin to wonder if their money went down the rabbit hole.  Your progress should indicate a rate if you plot this on a line graph.  Such a graph should indicate approximately when you expect to complete the project.  If your progress is missing your expected completion date then you need to take action as soon as possible. 

My project progress is missing the expected completion date, what do I do?

First, you need to prioritize.  Technically, you should prioritize all the features of your application early on, and focus on the most necessary features first and organize the features down to the least necessary features.  If you're going to miss your delivery date, and marketing already published a delivery date that cannot e changed, then you'll need to cut some unecessary features from your product.

If you can add programmers to your project, that is an option.  This will need to occur early on.  Towards the end of your project is too late to add resources.  Such a plan will only increase the amount of time it takes to complete the project due to organizational issues.

Prepare for Roll-Out (aka Deployment)

Make sure you prepare ahead of time for the roll-out of your product.  You need to think about how you are going to transition from your current product to the new product.  If this is the first iteration of your product, then you are in a unique position.  Most of the time, you'll need to provide some sort of data conversion from an older version.  Make sure you send out notifications early.  If you're going to need an outage, make sure you announce the expected outage times early. 

Next I'm going to talk about enhancement and work requests.

Software Quality Measuring

Software Quality Measuring

Measuring the quality of your software is similar to measuring bugs.  You need to prove that you are making progress.  This will keep management satisified and it'll determine how much time you will be spending with help desk calls.  The first thing you'll need is a minimalist objective way to track bugs.  Bugs can be tracked during development time, and it should be a goal to solve all bugs before releasing the software (although it is sometimes impossible to solve obscure, non-repeatable bugs).  You need a log where you can put your bug information.  I normally use one sheet per bug (and staple screen shots to this sheet if necessary).  You can use a bug tracking software package if you like, but it's necessary to get things off the ground as quickly as possible.  Once you have a book of bugs, you need to have metrics to measure how many bugs are being found per week (or per month, depending on how you plan to report the information).  You'll also need to know how many bugs are fixed in the same period of time.  These two numbers are going to give you a rough idea of progress.  I normally plot these numbers in Excel and show a line graph of how many bugs have been found each week, then I can find a trand line (or curve) and predict what I expect over the next few weeks.  When you're done, your chart had better point downward. 

What if my bug chart is going up?

Oh boy!  You're doomed!  OK, not quite.  There could be many reasons for an increasing number of bugs.  Maybe your product use has increased.  Many of my company's software is used for end-of-year accounting purposes.  Near the end of the year, near close-out time, bugs increase.  Not because there are more bugs, but because now they are being discovered in sections of software that was not previously used or not used as vigorously.  It may also be because of newer users.  New users are very good at finding interface bugs.  Things like typing a dollar sign in the price input box.  Trained users might be trained not to type in the dollar sign, and the software developers who (for this example) forgot to accept only the numeric part of the input were careless due to a rushed production schedule.  None of what I'm describing gets you off the hook.  When managment sees a chart showing the number of bugs rising, you better have a plan, not just an excuse. 

So what's your plan?

Initially, you should find more resources to throw at the problem.  This technique doesn't work when developing new software, but it does work when fixing bugs.  You should have an estimate of how long (on average) it takes to fix a bug.  Then assign enough programmers to the bug fixing task to get the bugs fixed as quickly as possible.  Your goal should be to fix all the bugs on the books before the next report to management is due.  Utlimately, it will end up coming down to fixing as many bugs as you can with the limited resources you have.  You should also prioritize your bugs to hit the most critical bugs first.  Then further prioritize to fix the easiest bugs first.  This technique is also reversed from software development, but your goal is to get the overall quantity of bugs reduced as quickly as possible.  Remember: Always fix the most critical bugs first!  Don't try and reduce the number of bugs quickly by just fixing the easy bugs.  If you have multiple developers, you might be able to get away with this strategy if you assign one developer to fix all the easy bugs, but ultimatly, and work-stopping bugs should get the most resources assigned.  Any bug that causes a work stoppage is costing the company money in production as well as developer time to fix the bug.

I'll get into more detail about bug tracking in a future post, but for now, this is all you'll need to keep your bosses happy and keep the progress of software development flowing.

Next I'm going to talk about Software Development Progress.

Software Version Numbering

Software Version Numbering

As I mentioned in the introduction post, you must track the version number of your software.  You can choose any method of tracking, such as a sequential numbering scheme, or you can follow the stardards used by most IT products (such as Visual Studio).  Initially, I would recommend starting a spreadsheet for each software project that is under development.  The spreadsheet needs information about the project, when development was started and when it was first rolled out for use by the customer/employees.  If a bug is fixed an the application is reintroduced, then the version number must be incremented and noted.  Do not release a version without incrementing the version number!

Another name for tracking version numbers is revision control.  There is software available to track version numbers and there is software available for maintaining source code versions.  Initially, you will want to just track the version numbers just to get control of your softare development.  In other words, it's better to implement a quick and dirty system and get rolling than to stop progress and make a big production out of it.  Later, when time becomse available, you can research products and other more complex methods.

I cannot stress enough how important it is to get version control in place early on.  There will come a day when a dispute breaks out between the customer/employee and one of your developers or between your department and management.  Version control can help resolve disputes. 

Now you're probably thinking "Frank?  What are you talking about?"  OK, it's example time.  I took over as an IT manager for a small company that had two IT departments that were combined into one department (the previous IT managers were both sacked).  None of the software had version control.  The company had a dozen different software applications used by employees and customers for various sub-tasks that the company did.  I was young and green at the time, it was my first IT manager job.  I knew about version control, but I didn't think it was high on my priority list because the department I took over had many other issues that I had to solve as well.  Unfortunately, my version control problem became a major problem.  One program that was used by a customer to enter their data contained a small database for storing local information.  When the customer was done entering their information, they would go to the export menu and export their data to a zip file which they subsequently emailed to our company to be incorporated into our master database.  The problem occurred when a customer sent us a zip file containing a miss-matched database configuration.  It took hours to figure out what the difference was and we determined that it was an older copy of our software that the customer was using (apparently, they didn't install the new version when they performed their work for the new season).  Once the differences were identified, then the hard part of conversion started.  Our other option was to call the customer back and make them install the lastest version and re-enter their data, but we didn't have an knowledge of which version they actually had because we didn't track the version numbers.

In order to resolve any future problems with database miss-matches, we incorporated versions in our software and we always provided a data converter.  So each product has all the previous data converters built-in.  New software will have a new converter added to it if the database was changed.  For example: Version 2 of our software has a converter to go from version 1 to 2.  Version 3 of our software had a converter to go from version 2 to 3.  So if you previously had data from version 1 and you install version 3 of our software, then the software could detect that your data belongs to version 1 and runs the 1 to 2 converter.  Then it would detect that you have version 2 data and runs the 2 to 3 converter.  Voila!  You have the correct version of data to match your software.  The customer is happy, you are happy, all is well.

Example 2

We received a call from a customer that was having problems with bugs in his software.  We had never seen such a bug, but every computer is different and there could be a conflict in the version of their OS, their installed software, etc.  Unfortunately, it's difficult for the developer to look at the code of the latest version and try to fix a bug that may have been fixed in a previous version (I've seen developers work on a bug that was fixed by another developer months ago).  We were forced to send the latest copy of our software to the customer and make them install it and call us back.  Of course, we look like idiots at that point, because we should have been able to ask them for the version that they had installed.  If it was an older version, then we could have requested they install the latest version first, rather than crossing our fingers and hoping they had an older version with a bug that we had previously fixed but not tracked (I'll get into bug tracking later).  As it turned out, the bug was still there and we needed to fix it.  Do we know if the customer was running the latest version?  No idea.  Did we waste their time?  probably.

Version Control Software

As I said earlier, version control software is not necessary, but it can also be used a crutch for tracking bugs and assisting customers with problems.  If you are running a multi-programmer shop and more than one person is responsible for the source code of an application, you'll probably need some sort of version control software.  The purpose of this software is to keep copies of all your revision histories and control the merging of source code.  I can't tell you how many times I've compared the current version of a source file with a previous version to figure out what was changed and why.  It's one more detective tool that can assist in tracking down a bug.  I won't go into version control software at this time.  I'll save this for a future post.

Next, I'm going to talk about measuring the quality of software.

Measuring (Introduction)

Measuring (Introduction)

This will be a mutli-part posting about measuring.  I will list a short introduction to each subject in this blog post and then create one detailed post for each subject in the near future.

Let's say you're a new IT Manager.  You have never done this job before.  You were either hired directly to take over an IT department or you were promoted to IT manager because the previous manager was sacked or moved on.What do you do?  Where do you start?  What if the department you took over is new and standards are not set in place?

That's where measuring comes into play.  You need to get control of what your people are doing and make sure management knows that you are in control of the department.  Both can be accomplished by measuring and tracking.  Now the question is what to track and why?

Software Version Numbering

If your department develops custom software, maybe it's only for internal use, maybe it is a custom application for your customers.  Either way, you need to implement versioning.  I have ignored versioning in my early years to my own peril.  It's difficult to help a customer or employee troubleshoot a problem if there is not way to track which version they are using.  It is also a problem with web applications if multiple users are working on the site and may accidently post the wrong copy.

Software Quality Measuring

In order to make sure you are making progress in fixing bugs, you need to somehow track how many bugs you have repaired and how many more bugs are reported.  This information can be used to predict how many more bugs are expected and it can be used to show that the quality of your software has improved (assuming the number of bugs are dropping).

Software Development Progress

Any project in progress will need to be tracked.  You need to get an estimate together so that you can determine how many programmers will work on the project and how long it will take them.  You'll also need to be able to estimate the amount of bugs you are expecting so you can be more effective in quality control.  Management is going to want to know all of this information because they need to know how much it's costing them to build the software.

Enhancement Requests/Work Requests

You need to keep a list of requests from users.  These possible enhancements to software will sometimes conflict with each other.  That's ok, it's just a rough request list.  In my company enhancement request does not translate directly into "enhancement must be incorporated."  If your IT department is new then you must somehow involve your company management in the decision making about which enhancements will go into the next version of your software.  This gets you off the hook when enhancement requesting users fly off the handle because they didn't get their requests implemented.

Hardware Inventory

If you are in charge of the IT hardware (or in charge of a network administrator and/or PC repair person(s)), then you will need to carefully track your hardware costs.  You'll need a complete inventory of each PC, server, printer, etc.  You can keep it to items costing $100 or more, or in large companies, $500 or more.  I usually don't track mice, keyboards, etc.  You'll also need to track software licenses and hard copies.  Each piece of hardware and software must have a purchase date and the number of years before replacement.  This will determine an annual budget for IT hardware and software.


Tuesday, May 1, 2012

C# Books

For those who are new at C#, or those who want to expand their knowledge of the C# language, I would recommend the book "Illustrated C# 2010" by Daniel Solis (technically I own the 2008 and 2005 versions, but their both excellent). 



Why do I like this book so much?  I've purchased more books in my life time than I care to admit.  I have a habit of buying books on a subject when I get stuck on a problem.  Then discovering that the book I bought doesn't quiet have enough detail.  So I buy another book.  Then another book.  I have .NET books like crazy.  I think I've browsed through them a couple times, maybe gleaned some intelligence from a couple of chapters and that's it. 

This book, however, is not one of those 900 page, difficult to hold books.  It's convenient.  It's detailed.  It has a lot of really detailed information in it.  Is it about visual studio?  No.  Can you gain enough knowledge about delegates to be useful.  Yes.  This is an excellent book to just sit and read.  It's also an excellent reference.

Monday, April 30, 2012

ASP Front-Side Page

The front-side page used in a C# web application is an optional beast.  It's purposes is to divide you C# code from your HTML code.  The downside is that the front-page is not compiled at compile time.  It's compiled at run-time, on the server.

The second issue with this method is that there are two ways to inter-mingle code in a web application.  One method is to print the HTML from the back page, and the other method is to insert method calls and C# code into the front-side html.  Both methods can get ugly and it's worse when both methods are used on the same page.

A technique that I've used is to empty out the front page and only use the back page.  This is not for everyone and there are a few downsides to this method.  To use this method, create a new aspx page (add new item - web form):


Assuming you created a new web page, now you'll see the default front-side code.  Remove everything, except the first line of code:



Now you can print html code by using the Response.Write() method.  For example:

Response.Write("<html>");
Response.Write("<body>");
Response.Write("Hello World!");
Response.Write("</body>");
Response.Write("</html>");

OK, so now you're wondering "why go through all that effort?"  When I build data-driven web based applications I typically use a lot of web pages where data is listed in rows and columns.  Instead of going through the effort of printing out html (like the sample above), I write routines to print one cell of information or one row of information and the html output is inside the method that prints the cell.  Here's a sample of what I'm talking about:


namespace MyNameSpace
{


public class GridOutputFunctions
{
     public static void PrintTop()
     {
          Response.Write(“<table>”);
}
public static void PrintRowStart()
{
          Response.Write(“<tr>”);
}
public static void PrintRowEnd()
{
     Response.Write(“</tr>”);
}
public static void PrintBottom()
{
Response.Write(“</table>”);
}
          public static void PrintCell(string MyData)
          {
              Response.Write(“<td>” + MyData + ”</td>”);
}
}
}
 To use the above code:

GridOutputFunctions.PrintTop();
GridOutputFunctions.PrintRowStart();
GridOutputFunctions.PrintCell("Cell 1");
GridOutputFunctions.PrintCell("Cell 2");
GridOutputFunctions.PrintRowEnd();
GridOutputFunctions.PrintBottom();

 If you're querying a database and listing out rows of data you can use this method to output a list of information on a web page.  This is an over-simplified example of how it works, but this method can be used without binding to a data object.

The elegance of this method comes into play in applications where you can write low-level output methods to apply to every web page.  Then your code would only contain C# code, no HTML.  In practice, additional "special-case" HTML is needed.

The downside to this method of programming is that 3rd-party C# objects don't always work with your web page.  Especially if they need the front-side page.  Also, the designer window in visual studio is useless in creating web pages. 

Saturday, April 28, 2012

Designing a Database Driven Application

Some helpful notes on successfully building a database driven application.  I'm assuming an MS SQL server for this example, but Oracle and many other databases will work as well.


Before your write any code:

  • Define your tables first.
  • Define all relational integrity constraints.  If you make a programming error, your constraints will crash your program while you are still building it.  
  • Use cascade deletes.  This can be tricky because cascade deletes can get into a circular reference and MS SQL server does not like that.  If you use cascade deletes, then you can manually delete a record and guarantee that child records are correctly cleaned up.  Oracle's triggers are little more forgiving, and you can use triggers in MS SQL server, but the application of the cascade is very simple and should be used where possible.
  • Create your "add" functions first to populate the database.  This will avoid the necessary step of manually inserting data into the database.
  • Index every foreign key field.  This will speed up your joins.
  • Index any field you are going to use in a select query.  I normally do this after I write my code.  I look through my "ORDER BY" fields and apply an index to each field in the list that doesn't already have an index.

Miscellaneous SQL Server Queries

Let's say you have an MS SQL Server full of databases.  Assume you have a large number of identical databases representing your customer data.  One database per customer.  Furthermore, you decide that you want to alter a table in each database to make one change to all the databases.  MS SQL server has views that allow you to query the names of your databases, the tables in each database and the fields in each table, etc. etc.  I call it the database of your database.


To list all databases in your SQL server use the following query:


SELECT * FROM sys.databases


This will list every database in your server.  To list all the tables in a database, use the following:


SELECT * FROM yourdatabasename.sys.tables


All of the views available are listed under Databases\master\Views\System Views in your SQL server management studio.  See sample below:



C#, Extension Methods

There are times when it's convenient to add a method to a data type. An example of an extension is the "Trim" method of the string data type:

string s;

string TrimmedString = s.Trim();

OK, so let's say that we want a new method called "Keep" and we want this method to keep characters according to a list provided:

namespace StringExtensionNameSpace
{
    public static class StringExtensions
    {
        public static string Keep(this string s, string Keepers)
        {
            string Result = "";

            for (int i = 0; s != null && i < s.Length; i++)
            {
                if (Keepers.indexOf(s[i]) > -1)
                {
                    Result += s[i];
                }
            }

            return Result;
        }
    }
}

To use this example:

string s = "This is a test";
string ResultingString = s.Keep("hisaet ");

The output of this example would be "his is a test".  Notice how the capital "T" has been removed but all other characters remain.


About the code:
The first thing to keep in mind is that this must be a static method of a static class.  The first parameter of the method must use the "this" keyword to indicate that the data type will be passed in as a variable named by the "this" statement.  


Possible gotchas:
- Make sure you use "public" or "internal" to be able to access this from other classes.
- Make sure your data type matches the data type you are extending.  In this instance, this extension will extend a string type and will not work on an integer data type.
- Don't forget to check your inputs for nulls, or empty values.


Here We Go!

I've finally done it. I've started my own blog. My intent is to publish information that I have learned over the years so software developers can learn from the mistakes that I've made. Hopefully, I can cut someone's debugging time by getting to the nuts and bolts of how things work. I normally use Google to find information, which can be a good start. Many times I have to sift through multiple articles to get the complete information that I need. I'm hoping to collate a lot of this information into this blog in a fashion that allows programmers to go to one location and find everything they need on a particular subject. I have no illusions that this will be an easy task and I'm not blessed with a plethura of free time. But I'm going to take a stab at it anyway.

With that said...Wish me luck!