Sunday, December 7, 2014

Dear Computer Science Majors..

Summary

This blog post is directed at all those Computer Science students or recent grads who want to get a leg-up in their career.  From time to time, I talk about what resources are available for Computer Science students to get ahead of the game.  It's been a while so I'm going to give you a few resources to hone your skills and some hints on how to get your career started early.


Practice, Practice...

First of all, there are a few sites where you can practice solving programming problems.  Companies don't hire software developers because they know a particular language (though it's good to know at least 3 or 4 languages in current use such as C#, Python, Ruby, Java etc.).  The reason for hiring a person is to hire a problem solver.  Remember those word problems in math class that everybody hates?  Yeah, that's what companies need.  Only every software problem is a word problem that must be solved.  Some problems will come with very detailed specifications and you will be very lucky when something like that occurs.  Many times you'll get somebody's fuzzy idea of what they want to do with a computer and you'll be required to translate that into software.  You'll discover just how challenging that is.

One of the subjects that colleges don't teach very well is how to solve problems.  It's mostly due to the lack of time needed to get good at problem solving, but it also has to do with the fact that college students don't yet have a big bag of tricks.  I obtained a big bag of tricks by reading a lot of books and writing my own programs on the side as a hobby (mostly games).  Unfortunately for me, the World Wide Web didn't come into existence until I graduated and I didn't have the resources that students currently have at their fingertips.  One of those resources is called Hacker Rank.

Hacker Rank is a website where software developers can sign up for free, practice programming problems of all difficulties and subjects and build up a reputation (click here and sign up).  Programmers can compete against other programmers.  Hacker Rank has many languages to choose from.  The practice problems have automated testing so you can get your program tested immediately.  Remember how you learned Algebra?  Classroom time was just an introduction, you got good at Algebra because you did the odd numbered  problems in the back of the book (or sometimes books use the even numbered problems).  The psycho students did ALL of the problems and they earned their 4.0 because it takes a lot of sweat to get good at it.  The only way to get good is through practice.  Hacker Rank is the programmer's equivalent of the odd numbered problems.  Spend some time at this site and practice!

I have personally worked out a bunch of problems at Hacker Rank (over 50), so I know how enticing it is.  I usually use my Visual Studio to work out the basic code (in C#) and then just copy it into their on-line editor, but you can just type it in directly.  There are other sites that you might try: CoderByte, CodeChef, CodeForces, TopCoder, CodeSchool and the list goes on.  Pick a favorite site and practice!


Programming Competitions

If you're really good at solving on-line problems and you'd like to participate in a competition you should seek out your local ACM chapter and get involved in their programming competition (assuming your college participates).  Click here for information on their programming competitions.  Click here to obtain the previous ACM competition programming problems.  Participation in the ACM programming contests is something you can put on your resume.  It makes a statement that you went beyond the normal curriculum and you are serious about software development.


Programming as a Hobby

Sometimes you have to visualize your career path as a career that is similar to an artist.  When an artist interviews for a job, they are required to show some of their best works.  As a developer it's best if you have something you can demonstrate or show off.  Companies will not necessarily ask for a demonstration of your works.  Here's the trick: If you have something posted on GitHub or BitBucket, you can put a link on your resume to those sites and companies will look at your work.  Trust me, when I look at a potential hire's resume and I see a GitHub account, I usually drop the resume and go look at what they posted on GitHub.  I learn more about a programmer from what software they've written than all the technical words typed on their resume. 


Building a Web Presence

In addition to using GitHub and BitBucket, you might want to setup a blog or a website.   You should be able to setup a website with minimal knowledge.  You're not expected to be an artist (or web designer), so use of a template or WordPress is more than acceptable.  You can give some bio information and talk about your school projects, or just makeup a website about a subject you are interested in (hiking, photography, programming, etc.).  Be careful about what you put on your website.  Don't put your resume there (don't make the mistake I made!)  Most of the head-hunter websites will scrape resumes from websites and you'll never get your resume removed from their sites and they'll never stop hounding you.  Also, don't provide any personal information unless you feel comfortable with providing such information to strangers.  Don't ever provide information such as your social security number, etc.  But, you probably already know not to provide that information.

If you want an easier method of getting a web presence, just do a blog.  You can start a blog at blogger for free.  Just sign up for a Google Plus account and start a blog.  You can create as many blogs as you think you can handle.  It's best to keep your scope narrow at first and give your blog a subject that you can write about really easy.  If you want to attract readership, you'll need to provide some interesting content and probably a few pictures.  Blogger has an upload button that is a lot like Facebook.  I'm sure you have albums of photos on your Facebook account.  Why not load pictures of your hobby on blogger and write something about your experience.

Why do I tell students to setup a website or blog?  Because of web presence.  It's very important.  When I get a resume to look over, the first thing I do is type the person's name into Google or Bing and see what pops up.  I'm amazed at how many long-time developers have zero web presence.  That's a red flag.  Soon it will become a career killer (if it's not already a career killer).  If you go to Google and type in Frank DeCaire, you'll see that I have a bit of a web presence.  

Trust me, I hated English class.  To write a paper is like walking barefoot through broken glass.  My first few blog posts were scary.  After a while, I got good at it.  Now blog posts just flow from my keyboard.  Start blogging when you're young.  I wish I started in the early 2000's.  Fortunately, I signed up for my first domain around 2001 or so and I've had a website ever since.  So that has helped my web presence.

Sign up for a Stack Overflow account.  This is where you want to put your resume.  Stack Overflow is by invite only, but it's easy to get an invite, especially if you already have a blog or website.  Stack Overflow is like the cream of the crop.  It takes a bit of effort to fill out all the information on their site and get your resume set, but take the time and get it there.  You can keep your work history on this site and over time you can just keep updating your information, in case you need to switch jobs.  Stack Overflow has controls to allow you to switch on or off you information so you can indicate if you're looking for a job or not.  You can start filling out this information when you start college and just update it as you progress through your education.

I have a Linked-In account, and it's probably a good idea to sign up for one, but I haven't had much luck finding a job with Linked-In.  Linked-in is used by many recruiters.  Some recruiters are just looking to fill contract positions but some companies have hired their own recruiters and are looking to fill full-time permanent positions.  So it's at least good to have a Linked-In account with your work history so a potential recruiter can contact you if you're looking for a job.  It's free, just don't forget to update your work history information when you switch jobs.


Internships

If your college holds job fairs, spend some time at one.  Sign up for a summer internship.  Look for a small company that has a lot of technologies to work with.  If you get an interview for an internship, ask what you will be working on.  Pick the company that will throw you in with the developers and give you a project to work on.  Avoid the internships involving low-level busy work that a company wants to complete on the cheap.  You want to get some working experience in the developer world.  You need to learn the skills of version control, unit testing, handling legacy code, deploying software, etc.  All the skills that are not taught in college but are necessary for running an efficient developer shop.  If you have these skills, you're a better candidate for a programming job because you can be up and running quickly.

Also, having an internship on your resume is like having a job on your resume.  You can do multiple internships and get away with the fact that you jump around from job to job, because nobody expects an internship to be permanent.  So if your resume has two or three internships, that's just more hands-on experience that you can brag about.  Having a variety of real-world development knowledge is a definite plus.


Resume Workshops

If your college has resume workshops, then be sure and participate.  People like me will be at these workshops.  We have good ideas on how you can spruce up your resume.  Why?  Because we look at resumes.  A lot of resumes.  We look at these things from the perspective of an employer looking to hire someone.  Sometimes we're the first person to look at your resume, sometimes your resume will go through a machine or a recruiter first.  We have all kinds of hints that will help your resume get through the machine and/or the recruiter.  We also have hints on how you can get past us and get an interview.  Don't miss out on the resume workshop.


Summary

I hope I've helped some of you think about what you need to do to get your career ahead of the game.  My career got off to a slow start because I didn't have the resources that are available today, but also because I had no idea about the purpose of a career fair or a resume workshop.  You want to get your career going as soon as possible so you can get through the boring "new guy/girl" aspect of your career.  You also want to make sure you are the person that a company "needs" because you have talent and skills.  If you have skills, you want to make sure that those skills are properly advertised, so you get paid what you're worth and you get your foot in the door of the company that you want to work for.

So get busy and get your blog or website started today.  You're only a Freshman?  Starting your first class in the upcoming semester?  Don't let that stop you.  Get a blog started.  Blog about your college experience.  If you already know how to program, blog about your skills.  Demonstrate your skills by setting up a website.  I knew how to do double-linked lists and binary trees before I started college.  If blogging had been available, I could have blogged about that skill.  I could have blogged about my game development hobby.  Don't let this opportunity pass you buy.

Saturday, December 6, 2014

NHibernate Mapping Utility Update

Summary

For those who have followed my blog, I have a long-term project called the NHibernate Mapping Generator.  This application generates ORM table mapping C# code for the Fluent NHibernate ORM.  In this blog post I'm going to describe a few new features to this application.


ADO.Net Unit Testing

There is an ADO.Net Context that was created in the unit test helpers of this solution.  I have added a condition where this can be called form a unit test causing the connection to override and use the SQLLocalDB database.  If you use the ADODatabaseContext in your project, then you can connect to your normal database by passing a connection string.  If you create unit tests and include the start and end code for the SQLLocalDB database, then this context will use that database whenever your code is executed from the unit test assembly.  When using the context from a unit test, you will not need a connection string at all, but you might want to specify which database to default to.  In order to do that, you can pass a blank connection string and pass in the database name, or you can pass the database name in the connection string and the constructor will copy the database into the new unit test context (see ADODatabaseContext.cs source for more details).


Session Wrappers

I have added two session wrappers to the project.  To accommodate all the default methods and properties, I used partial classes with all the repeated code in the Wrappers directory.  A method for each table is created inside the other half of the partial class for each database that is generated.  This code is contained inside the database directory and Wrappers sub-directory.  This will allow you to change your code from:

var query = (from d in db.Query<Department>() select d);

To:

var query = (from d in db.Department select d);


Future Enhancements Planned

One enhancement that will be necessary is the ability to access databases that do not show up in the drop-down list.  The SQL Server Management Studio has the ability to type in the name of a SQL Server and I intend to mimic that feature.

Next, I need to store the SQL servers discovered in the registry so that the next time this utility is run, it just reads those entries.  If the user desires, they will be able to refresh their list just like Management Studio does.  This will cause the startup time to be quicker.

Next, I need to have the ability to use a user id and password instead of just integrated security.  Management Studio has a drop-down to select one or the other, I'll probably mimic this and only show the database list when the user clicks a button.

Next, I would like to have check boxes for the ability to generate code for non-NHibernate scenarios.  That way the NHibernate mappings will not be created, but the views, stored procedures, table creation code and the constraint code will be created.  That will make this utility handy for people who want to setup unit tests for projects that use EF, LINQ-to-SQL or just plain ADO.Net.


Where to Find the Code

You can download the entire solution including the unit test helpers, sample unit tests, sample projects and the NHibernate mapping generator at my GitHub account here.

Fluent NHibernate Session Magic

Summary

If you've spent any time with NHibernate or Fluent NHibernate, you'll know how annoying it is to use the .Query<Tablename>() method in every LINQ query.  It's especially a problem for situations where you must convert Entity Framework or LINQ-to-SQL to NHibernate, since you have to convert every LINQ query in your code.  In this blog post, I'm going to show how to get rid of this construct.  In the future I'll add this to the NHibernate mapping generator to allow this syntax to be used any time you auto-generate your table mappings.


NHibernate.ISession

I'm going to mention up-front that I was not the brains behind this discovery.  My colleague Samer Adra (Blog, Linked-In, Twitter, Stack Overflow) played around with this code to get it to work.  So I'm going to blog about it so everybody can start using this technique and get rid of the annoying .Query<Tablename>() syntax.

So the first that that needs to be changed is the ISession interface.  To do that, you need to default all the methods and properties that are used by the ISession object.  It starts like this:

public class SessionWrapper : ISession
{
  private readonly ISession innerSession;

  public SessionWrapper(ISession innerSession)
  {
    this.innerSession = innerSession;
  }

  public void Dispose()
  {
    innerSession.Dispose();
  }

  ... more
}


And it goes on and on (download the sample code).

I setup the project with two tables in it.  These table mappings connect to tables in MS SQL Server.  If you've followed this blog you know how to get the mappings, if not, then you'll need to download the NHibernate Mapping Generator from my GitHub account and generate the mappings automatically.  Then you can go into the data project and find the table mapping cs file and copy the code.  You can get the mapping generator here.

Anyway, I added these two lines of code to the bottom of the SessionWrapper object shown above:

// Table mappings follow
public IQueryable<DataLayer.Department> Department
{
  get { return innerSession.Query<DataLayer.Department>(); }
}

public IQueryable<DataLayer.Store> Store
{
  get { return innerSession.Query<DataLayer.Store>(); }
}



Next, I had to modify the context to handle the session wrapper.  That only required me to change the OpenSession method:

public static SessionWrapper OpenSession()
{
  return new SessionWrapper(SessionFactory.OpenSession());
}



Now for the final code inside the Program.cs file:

using (var db = MSSQLSessionFactory.OpenSession())
{
  var query = (from d in db.Department select d).ToList();

  foreach (var item in query)
  {
    Console.WriteLine(item.name);
  }

  Console.ReadKey();
}



That's it.  Obviously, creating a method for each table inside the SessionWrapper is the most tedious aspect of this process.  That's why I'm going to add it to the NHibernate table generator project and generate this session automatically.  Also, you'll need to support the stateless session (IStatelessSession) if you intend to use stateless sessions in your projects.  So the same code will need to go into a stateless session wrapper as well.


Cross Database Support

One other thing to note is that the database that this table is located in is the sampledata database.  If there was a Department table in another database and we wanted to be able to access both tables in one query, then we'll need a unique name inside the SessionWrapper object.  One solution is to create a naming convention that uses the database name with the table name.  I may use a scheme for that in the mapping generator or I might leave it to the end user to come up with a scheme of their own.  Just be aware of that possible issue.


Download the Code

You can download the code at my GitHub account here.  Don't forget to go into the MSSQLSessionFactory.cs file and change the server instance (search for "YOURSERVERINSTANCE").



Sunday, November 30, 2014

Entity Framework Unit Testing with SQLLocalDB

Summary

I've published a few blog posts on the usage of SQLLocalDB with NHibernate.  Now I'm going to demonstrate how easy it is to use with EF.  In fact, SQLLocalDB can be used with ADO queries and LINQ-To-SQL.  If you're dealing with legacy code and your methods use a lot of database access operations, it would be beneficial to get your code in a test harness before converting to your target ORM.


Modifying the EF Context

The first thing I'm going to do is create a helper to deal with the Entity Framework context.  If you generate a new Entity Framework database, you'll get a context that is configured using the App.config file.  This is OK, if your application is always going to use the same database.  If you need to switch databases, and in this case, we'll need to use a different database when under unit testing, then you'll need to control the context parameters.  Here's the helper class that I wrote to handle multiple data sources:


using System.Data.Entity.Core.EntityClient;
using System.Data.SqlClient;

namespace Helpers
{
    public static class EFContextHelper
    {
        public static string ConnectionString(string connectionName, string databaseName, string modelName, string userName, string password)
        {
            bool integratedSecurity = (userName == "");

            if (UnitTestHelpers.IsInUnitTest)
            {
                connectionName = "(localdb)\\" + 

                    UnitTestHelpers.InstanceName;
                integratedSecurity = true;
            }

            return new EntityConnectionStringBuilder
            {
                Metadata = "res://*/" + modelName + ".csdl|res://*/" + modelName + ".ssdl|res://*/" + modelName + ".msl",
                Provider = "System.Data.SqlClient",
                ProviderConnectionString = new  

                     SqlConnectionStringBuilder
                {
                    MultipleActiveResultSets = true,
                    InitialCatalog = databaseName,
                    DataSource = connectionName,
                    IntegratedSecurity =
integratedSecurity,
                    UserID = userName,
                    Password = password
                }.ConnectionString
            }.ConnectionString;
        }
    }
}


The helper class builds the connection string and replaces the one that is defined in the App.config file.  There is a check to see if the calling assembly is a unit test assembly, I will use this to override the server name and point to the SQLLocalDB that is defined in the UnitTestHelpers.InstanceName.  This will ensure that if an EF context is called inside a method of your program, it will connect to the unit test database if a unit test calls your program.

Once you have created the helpers class. you'll need to use it inside your context code.  To do that, you'll need to add the code into the Context.tt file (it should be named something like "Model1.Context.tt").  This file (also known as a T4 file), is used to generate the Model1.Context.cs file.  Modify the code to look something like this:

#>
using System;
using System.Data.Entity;
using System.Data.Entity.Infrastructure;
using Helpers;
<#
if (container.FunctionImports.Any())
{
#>
using System.Data.Objects;
using System.Data.Objects.DataClasses;
using System.Linq;
<#
}
#>

<#=Accessibility.ForType(container)#> partial class <#=code.Escape(container)#> : DbContext
{
    public <#=code.Escape(container)#>()
        : base(EFContextHelper.ConnectionString("SQLSERVERNAME", "sampledata", "Model1", "", ""))
    {


First, you'll need to add the "using Helpers;" line, then you'll need to add in the EFContextHelper.ConnectionString method as the base class initialization parameter.
Change the SQLSERVERNAME to match your actual MS SQL Server name.  Change sampledata to match your actual database name.  Change "Model1" to match the name of the edmx file.  Finally, you can use a user name and password if you're not using integrated security on MS SQL Server.  In my example, I'm using integrated security. 
Once you change the tt file, save it and check the matching cs file to make sure there are no syntax errors.


Using the Mapping Generator

Currently the mapping generator is used to generate NHibernate database mappings.  If you don't want the extra code, you can download my project and strip out all the Fluent Nhibernate code (or just leave it for now).  I have added a new section that will generate code to create tables that match the tables defined in your database.  But first you must run the generator to create the table definitions.

Go to my GitHub account and download the NHibernateMappingGenerator solution.  You can find it here.  Make sure that the NHibernateMappingGenerator project (which is a Windows Forms project) is set to the startup project.  Then run the program and select your SQL Server and database:


If your server doesn't use integrated security (i.e. you need a user name and password to access your SQL Server), you'll have to do some open heart surgery (sorry, I haven't added that feature to this program yet).  Once you click the "Generate" button, this program  will generate code that can create tables, stored procedures, views and constraints in your SQLLocalDB database instance for unit testing purposes.  You should note that your test database will match your production database from the time that you generated this code.  If you make database changes to your production system, then you'll need to regenerate the table definitions.


The Unit Test

I'll demonstrate one simple unit test.  This test isn't really going to be useful, except it will test the fact that you can insert data into SQLLocalDB and query the data using Entity Framework and it will not corrupt your production data.  Here's the code:

[TestClass]
public class EntityFrameworkTests
{
  [TestCleanup]
  public void Cleanup()
  {
    UnitTestHelpers.TruncateData();
  }

  [TestMethod]
  public void TestEntityFrameworkContext()
  {
    using (var db = new sampledataEntities())
    {
      Store store = new Store
      {
        Name = "Test Store",
      };
 

      db.Stores.Add(store);
      db.SaveChanges();

      var resultQuery = (from s in db.Stores 

                         select s).ToList();

      Assert.AreEqual(1, resultQuery.Count());
      Assert.AreEqual("Test Store", resultQuery[0].Name);
    }
  }
}


As you can see, using EF from a unit test is just like using it inside your application.  Just open a context, insert data, query data, done.  The magic is occurring inside the context itself.  If you put a break-point inside the EFContextHelper.ConnectionString method, you can see where the connection string is built to point to the SQLLocalDB instance instead of your production database instance (you should always test this to verify that it is functioning correctly).


Generating the Tables

One detail that is buried in this sample code that I haven't covered yet is how the tables are generated inside the SQLLocalDB instance.  In previous blog posts I always used the Fluent NHibernate database generate feature to generate all the tables of the database in the unit test assembly initialize method.  If you open the AssemblyCommon.cs file (inside the SampleUnitTests project) and look at the ClassStartInitialize method, you'll see where I call another UnitTestHelpers method called CreateAllTables.  Dig into this method and you'll see ADO code that creates MS SQL tables for a specified database.  This makes the entire unit test helper project agnostic to the type of database you might want to test.  


Using This Technique for Other Database Types

The unit test helpers can currently handle EF as well as Fluent NHibernate, but technically, any ORM or direct data access can use this set of helpers.  Make sure to create a database connection helper for the database type you will use in your project (or already use in your project).  Then you can apply the same UnitTestHelpers.IsInUnitTest check to set the instance to the SQLLocalDB instance.  Finally, you can add unit tests for each method in your application, no matter which database context it is using.  

As I mentioned in the summary, it would ease your transition to another ORM, assuming that is your goal.  Your first step would be to create unit tests around your current objects and methods, using your current database technique, then convert to your target ORM and make sure the unit tests pass.


Getting the Code

You can download the entire solution at my GitHub account: https://github.com/fdecaire/NHibernateMappingGenerator.




Saturday, November 22, 2014

GDI+ Graphics: Adjusting Gamma

Summary

I'm going to do something a little different here.  I'm going to start a series of posts about the GDI+ graphics interface (in C#).  If you have followed my blog for a while, you already know that I have a demonstration game I wrote Battle Field One.  The last version I blogged about uses SVG graphics to render the output in a browser.  In a future post I'm going to show how to use GDI+ graphics and replace the web interface with a standard windows forms interface.  So I'll cover a few GDI+ techniques along the way and then I'll incorporate these techniques in the game changes coming up.


GDI+

First, I need to explain GDI+.  GDI+ is the new version of the 2 dimensional graphics engine that can be accessed directly from the forms paint event (GDI stands for Graphic Design Interface).  GDI is not fast.  For fast, we would need to use DirectX (I'll blog about DirectX and Direct 3D later).  Since Battle Field One is a turn-based game, we don't need fast.  What we need is simple.  That's why I'm going to use GDI+.

To setup a quick and dirty example, you can create a new Visual Studio project of type "Windows Form Application".  Go to the code part of your form.  You'll need to add the following "usings" at the top of your initial form:

using System.Drawing;
using System.Drawing.Imaging;


Now switch back to your form design and switch to the events tab of your properties window:



Scroll down until you find the "Paint" event and double-click on it.  A new event called "Form1_Paint" will be created in your code.  All of your GDI+ code will be entered into this event for now.

Now let's put some png files in a folder of the project.  Create a new folder named "img", download and copy this image into that folder:


mountains_01.png


Now put this code inside your Form1_Paint event method:

Image Mountains = Image.FromFile("../../img/mountains_01.png");

Graphics g = pe.Graphics;

g.DrawImage(Mountains, 100, 100, Mountains.Width, Mountains.Height);


Now run your program.  You should see a hex shaped image with some mountainous terrain:



Of course, this program is just rendering the png image that is sitting in the img directory.  The reason for the double "../" in the path is because the program that executes is inside the bin/Debug folder.  So the relative path will be two directories back and then into the img directory.  If you don't specify the path correctly, you'll get a giant red "X" in your window:







Adjusting the Gamma

One of the capabilities that we'll need when switching the game to use GDI+ is that we need to darken the hex terrain where the current units cannot see.  In the game the visible cells will be rendered with a gamma that is 1.0, and the non-visible cells will be rendered with a gamma of 3.0 (larger numbers are darker, less than one will be brighter).

Now I want to demonstrate three mountain cells that represent a gamma of 1.0, 0.5 and 2.0.  In order to modify the gamma setting we'll need to use the ImageAttributes object:

ImageAttributes imageAttributes = new ImageAttributes();
imageAttributes.SetGamma(1.0f, ColorAdjustType.Bitmap);


The DrawImage object can accept the ImageAttributes as a parameter, but we also need to add a few more parameters.  So I'm going to show the code here, and then I'll discuss it:


Image Mountains = Image.FromFile("../../img/mountains_01.png");

Graphics g = pe.Graphics;


ImageAttributes imageAttributes = new ImageAttributes();
 
// normal gamma
imageAttributes.SetGamma(1.0f, ColorAdjustType.Bitmap);
g.DrawImage(Mountains,
        new Rectangle(100, 100, Mountains.Width, Mountains.Height),
        0,
        0,
        Mountains.Width,
        Mountains.Height,
        GraphicsUnit.Pixel,
        imageAttributes);

// lighter
imageAttributes.SetGamma(0.5f, ColorAdjustType.Bitmap);
g.DrawImage(Mountains,
        new Rectangle(200, 100, Mountains.Width, Mountains.Height),
        0,
        0,
        Mountains.Width,
        Mountains.Height,
        GraphicsUnit.Pixel,
        imageAttributes);

// darker
imageAttributes.SetGamma(2.0f, ColorAdjustType.Bitmap);
g.DrawImage(Mountains,
        new Rectangle(300, 100, Mountains.Width, Mountains.Height),
        0,
        0,
        Mountains.Width,
        Mountains.Height,
        GraphicsUnit.Pixel,
        imageAttributes);


This is all the code you'll need for this demo to work.  When you specify a rectangle for the second parameter, you will need to use the x,y coordinates of that rectangle to determine where the image will be plotted.  Leave the DrawImage x,y set to zero.  Use the image.Width and image.Height if you want to maintain the scaling of the image itself.  Otherwise you can adjust these parameters to blow up or shrink an image.

If you run the sample, you'll see that there is a normal mountain hex on the left, a lighter hex to the right of it and then a darker hex to the right of that:








Download the Sample Code

You can go here to download the zip file for this demo application: 

GDIPlusGammaDemo.zip





Saturday, November 15, 2014

Returning XML or JSON from a Web API

Summary

In my last blog post I demonstrated how to setup a Web API to request data in JSON format.  Now I'm going to show how to setup your API so that a request can be made to ask for XML as well as JSON return data.  To keep this simple, I'm going to refactor the code from the previous blog post to create a new retriever method that will set the Accept parameter to xml instead of json.


Changes to the Retriever

I copied the retriever method from my previous code and created a retriever that asks for XML data:

public void XMLRetriever()
{
    var xmlSerializer = new XmlSerializer(typeof(ApiResponse));

    var apiRequest = new ApiRequest
    {
        StoreId = 1,
        ProductId = { 2, 3, 4 }
    };

    var request = (HttpWebRequest)WebRequest.Create(

        apiURLLocation + "");
    request.ContentType = "application/json; charset=utf-8";
    request.Accept = "text/xml; charset=utf-8";
    request.Method = "POST";
    request.Headers.Add(HttpRequestHeader.Authorization, 

            apiAuthorization);
    request.UserAgent = "ApiRequest";

    //Writes the ApiRequest Json object to request
    using (var streamWriter = new  

           StreamWriter(request.GetRequestStream()))
    {
        streamWriter.Write(JsonConvert.SerializeObject(apiRequest));
        streamWriter.Flush();
    }

    var httpResponse = (HttpWebResponse)request.GetResponse();

    // receives xml data and deserializes it.
    using (var streamreader = new  

           StreamReader(httpResponse.GetResponseStream()))
    {
        var storeInventory = 

            (ApiResponse)xmlSerializer.Deserialize(streamreader);
    }
}


There are two major changes in this code: First, I changed the "Accept" to ask for xml.  Second, I recoded the return data to deserialize it as xml instead of json.  I left the api request to be in JSON.


Changes to the API Application

I altered the API controller to detect which encoding is being requested.  If Accept contains the string "json" then the data is serialized using json.  If Accept contains the string "xml" then the data is serialized using xml.  Otherwise, an error is returned.

Here is the new code for the API controller:

var encoding = ControllerContext.Request.Headers.Accept.ToString();
if (encoding.IndexOf("json",  

    StringComparison.OrdinalIgnoreCase) > -1)
{
    // convert the data into json
    var jsonData = JsonConvert.SerializeObject(apiResponse);

    var resp = new HttpResponseMessage();
    resp.Content = new StringContent(jsonData, Encoding.UTF8, 

        "application/json");
    return resp;
}
else if (encoding.IndexOf("xml",  

         StringComparison.OrdinalIgnoreCase) > -1)
{
    // convert the data into xml
    var xmlSerializer = new XmlSerializer(typeof(ApiResponse));

    using (StringWriter writer = new StringWriter())
    {
        xmlSerializer.Serialize(writer, apiResponse);

        var resp = new HttpResponseMessage();
        resp.Content = new StringContent(writer.ToString(), 

             Encoding.UTF8, "application/xml");
        return resp;
    }
}
else
{
    return Request.CreateErrorResponse(HttpStatusCode.BadRequest, 

           "Only JSON and XML formats accepted");
}


Compile the API application and then startup fiddler.  Then run the retriever.  In fiddler, you should see something like this (you need to change your bottom right sub-tab to xml):





Download the Source


You can go to my GitHub account and download the source here: https://github.com/fdecaire/WebApiDemoJsonXml

 

Web API and API Data Retreiver

Summary

In this blog post I'm going to show how to create a Web API in Visual Studio.  Then I'm going to show how to setup IIS 7 to run the API on your PC (or a server).  I'm also going to create a retriever and show how to connect to the Web API and read data.  I'll be using JSON instead of XML so I'll also show what tricks you'll need to know in order to implement your interface correctly.  Finally, I'll demonstrate how to troubleshoot your API using fiddler.


This is a very long post.  I had toyed with the idea of breaking this into multiple parts, but this subject turned out to be too difficult to break-up in a clean manner.  If you are having trouble getting this to work properly or you think I missed something leave a message in the comments and I'll correct or add to this article to make it more robust.



Web API Retriever

I'm going to build the retriever first and show how this can be tested without an API.  Then I'll cover the API and how to incorporate IIS 7 into the whole process.  I'll be designing the API to use the POST method.  The reason I want to use a POST instead of GET is that I want to be able to pass a lot of variables to request information from the API.  My demo will be a simulation of a chain of stores that consolidate their inventory data into a central location.  Headquarters or an on-line store application (i.e. website) can send a request to this API to find out what inventory a particular store has on hand.  

The retriever will be a simple console application that will use an object to represent the request data.  This object will be serialized into a JSON packet of information posted to the API.  The request object will look like this:

public class ApiRequest
{
  public int StoreId { get; set; }
  public List<int> ProductId { get; set; }
}


This same object will be used in the API to de-serialize the JSON data.  We can put the store id in this packet as well as a list of product ids.  The data received back from the API will be a list of inventory records using the following two objects:

public class InventoryRecord
{
  public int ProductId { get; set; }
  public string Name { get; set; }
  public int Quantity { get; set; }
}


public class ApiResponse
{
  public List<InventoryRecord> Records = new  

         List<InventoryRecord>();
}


As you can see, we will receive one record per product.  Each record will contain the product id, the name and the quantity at that store.  I'm going to dummy out the data in the API to keep this whole project as simple as possible.  Keep in mind, that normally this information will be queried from a large database of inventory.  Here's the entire retriever:

public class WebApiRetriever
{
  private readonly string apiURLLocation =  

             ConfigurationManager.AppSettings["ApiURLLocation"];
  private readonly string apiAuthorization =  

          ConfigurationManager.AppSettings["ApiCredential"];

  public void Retreiver()
  {
    var serializer = new JsonSerializer();

    var apiRequest = new ApiRequest
    {
      StoreId = 1,
      ProductId = { 2, 3, 4 }
    };

    var request = 

        (HttpWebRequest)WebRequest.Create(apiURLLocation + "");
    request.ContentType = "application/json; charset=utf-8";
    request.Accept = "application/json";
    request.Method = "POST";
    request.Headers.Add(HttpRequestHeader.Authorization, 

         apiAuthorization);
    request.UserAgent = "ApiRequest";

    //Writes the ApiRequest Json object to request
    using (var streamWriter = new 

           StreamWriter(request.GetRequestStream()))
    {             

      streamWriter.Write(
           JsonConvert.SerializeObject(apiRequest));
      streamWriter.Flush();
    }

    var httpResponse = (HttpWebResponse)request.GetResponse();

    using (var streamreader = new  

           StreamReader(httpResponse.GetResponseStream()))
    using (var reader = new JsonTextReader(streamreader))
    {
      var storeInventory = 

          serializer.Deserialize<ApiResponse>(reader);
    }
  }
}


Some of the code shown is optional. I put the URL location and credentials into variables that are stored in the app.config file.  You can add this to your app.config file:


<appSettings>
    <add key="ApiURLLocation" value=" 

       http://www.franksdomain.com/WebApiDemo/api/MyApi/"/>
    <add key="ApiCredential" value="ABCD"/>
</appSettings>


The URL will need to be changed to match the URL that you setup on your IIS server (later in this blog post).  For now you can setup a redirect in your "hosts" file to match the domain in the app setting shown above.

Navigate to C:\Windows\System32\drivers\etc and edit the "hosts" file with a text editor.  You'll see some sample text showing the format of a URL.  Create a domain name on a new line like this:

127.0.0.1        www.franksdomain.com

You can make up your own URL and you can use a URL that is real (technically, franksdomain.com is a real URL and it's not mine).  If you use a real URL your computer will no longer be able to access that URL on the internet, it will redirect that URL to your IIS server (so be aware of this problem and try to avoid using real URLs).  The IP address 127.0.0.1 is a pointer to your local machine.  So we're telling your browser to override www.franksdomain.com and redirect the request to the local machine.

Now you should be able to test up to the request.GetResponse() line of code.  That's where the retriever will bomb.  Before we do this, we need to download and install Fiddler (assuming you don't already have fiddler installed).  Click here to download fiddler and install it.  Now start up fiddler and you'll see something like this:



Now run the retriever application until it bombs.  Fiddler will have one line in the left pane that is in red.  Click on it.  In the right pane, click on the "Inspectors" tab and then click on "JSON" sub-tab.  You should see something like this:



In the right side top pane, you'll see your JSON data.  If your data is shown as a tree-view control then it is formatted correctly as JSON and not just text.  If you serialized your object incorrectly, you will normally see an empty box.  Notice that the store is set to "1" and there are three product ids being requested.


The Web API

The web API will be an MVC 4 application with one ApiController.  The API controller will use a POST method. 

So let's start with a class that defines what information can be posted to this API.  This is the exact same class used in the retriever:

public class ApiRequest
{
  public int StoreId { get; set; }
  public List<int> ProductId { get; set; }
}


Make sure this class is not decorated with the [Serializable] attribute.  We're going to use a [FromBody] attribute on the API and the object variables will not bind if this object is setup as serializable (I discovered this fact the hard way).  As you can see by the list definition we can pass a long list of product ids for one store at a time.  We expect to receive a list of inventory back from the API.


The response back to the calling application will be a list of inventory records containing the product id (which will be the same number we passed in the request), the name of the product and the quantity.  These are also the same objects used in the retriever:


public class InventoryRecord
{
  public int ProductId { get; set; }
  public string Name { get; set; }
  public int Quantity { get; set; }
}


public class ApiResponse
{
  public List<InventoryRecord> Records = new  

         List<InventoryRecord>();
}



The entire API controller in the MVC application looks like this:


public class MyApiController : ApiController
{
  [HttpPost]
  [ActionName("GetInventory")]
  public HttpResponseMessage GetInventory([FromBody] ApiRequest request)
  {
    if (request == null)
    {
      return Request.CreateErrorResponse(HttpStatusCode.BadRequest, 

             "Request was null");
    }
       
    // check authentication
    var auth = ControllerContext.Request.Headers.Authorization;
       
    // simple demonstration of user rights checking.
    if (auth.Scheme != "ABCD")
    {
      return Request.CreateErrorResponse(HttpStatusCode.BadRequest, 

             "Invalid Credentials");
    }
   
    ApiResponse apiResponse = new ApiResponse();

    // read data from a database
    apiResponse.Records = DummyDataRetriever.ReadData(request.ProductId);

    // convert the data into json
    var jsonData = JsonConvert.SerializeObject(apiResponse);

    var resp = new HttpResponseMessage();
    resp.Content = new StringContent(jsonData, Encoding.UTF8, 

                   "application/json");
    return resp;
  }
}


The controller is a post method controller that looks for an ApiRequest JSON object in the body of the posted information.  The first thing we want to check for is a null request.  That seems to occur most often when a bot crawls a website and hits an API.  If we're lucky, then bots will not find their way in, but I always code for the worse case situation.  The next part will check the header for the authorization.  I didn't cover this in the retriever, but I stuffed a string of letters in the authorization variable of the header.  This was setup to be "ABCD" but in a real application you'll need to perform a database call to a table containing GUIDs.  These GUIDs can be assigned to another party to gain access to your API.  In this example the shopping website application will have its own GUID and each store will have a GUID that can be setup to restrict what each retriever can access.  For instance, the website GUID might have full access to every store to lookup information using this API, but a store might only have access to their own information, etc.  I'm only showing the basics of this method in this article.  I'll cover this subject more thoroughly in a future blog post.

Next in the code is the dummy lookup for the information requested.  If you navigate to my sample dummy data retriever you'll see that I just check to see which product is requested and stuff a record in the list.  Obviously, this is the place where you'll code a database select and insert records into the list from the database.

Next, the list of inventory records are serialized into a JSON format and then attached to the content of the response message.  This is then returned.

Next, you'll need to setup an IIS server to serve your API.

 

Setting up the IIS Server

I'm going to setup an application in IIS 7 to show how to get this API working from your PC.  Eventually, you'll want to setup an IIS server on your destination server that you will be deploying this application to.

I'll be using IIS 7 for this demo, if you're using Windows 7 or earlier, you will probably need to download and install IIS 7 (the express version works too).  When you open IIS 7 you should see a treeview on the left side of the console:



Right-click on "Default Web Site" and "Add Application".  Name it "WebApiDemo".  You should now see this:



Now click on the "WebApiDemo" node and you'll see a panel on the right side of the control panel named "actions".  Click on the "Basic Settings" link.  Change the physical path to point to your project location for the API project (this is the MVC4 project that you created earlier or downloaded from my Github account).



You'll notice that the application pool is set to "DefaultAppPool".  We'll need to change this to a .Net 4 pool.  So click on the "Application Pools" node and double-click on the "DefaultAppPool" line.  Change the .Net Framework Version to version 4:



At this point, if you try to access your API, you'll discover that it doesn't work.  That's because we need to give IIS server access to your project directory.  So navigate to your project directory, right-click and go to properties, then click on the "Security" tab.  Click the "advanced" button and "Change Permissions" button.  Then click the "Add" button.  The first user you'll need to add will be the IIS_IUSRS permission.  Click the "Check Names" button and it should add your machine name as the domain of this user (my machine name is "FRANK-PC", yours will be different):



You'll need to give IIS permissions.  I was able to make the API work with these permissions:


I would recommend doing more research before setting up your final API server.  I'm not going to cover this subject in this post.


Now click "OK", "OK", "OK", etc.

Now run through those steps again: to add a user named IUSR, with the same permissions:





If you didn't setup your URL in the hosts file, you'll need to do that now. If you did this in the retriever section above, then you can skip this step.  

Navigate to C:\Windows\System32\drivers\etc and edit the "hosts" file with a text editor.  You'll see some sample text showing the format of a URL.  Create a domain name on a new line like this:

127.0.0.1        www.franksdomain.com

Remember, you can makeup your own domain name.

Now, let's test the site.  First, we need to determine what the URL will be when we access our API.  The easy method is to go into the IIS control panel and click on the "Browse urlname.com on *80 (http)" link:



Now you can copy the URL in the browser that popped up:

http://www.franksdomain.com/WebApiDemo

This URL is actually the URL to the MVC website in your application.  In order to access your API, your going to have to add to this URL:

http://www.franksdomain.com/WebApiDemo/api/MyApi

How did I get the "api/MyApi"?  If you go to your MVC application and navigate to the "App_Start/WebApiConfig.cs" file, you'll see this default setup:

config.Routes.MapHttpRoute(
        name: "DefaultApi",
        routeTemplate: "api/{controller}/{id}",
        defaults: new { id = RouteParameter.Optional }
);



So the URL contains "api" and then the controller name.  Remember the previous code for the API controller:

public class MyApiController : ApiController

Ignore the "Controller" part of the class name and you'll have the rest of your path.  Put this path string in the app.config file of your retriever application and compile both applications.  

Startup fiddler.  Run your retriever.  Your retriever should run through without any errors (unless you or I missed a step).  In fiddler, select JSON for the top right and bottom right panes. You should see something like this:


Notice that the retriever sent a request for product ids 2,3,4 and the API returned detailed product information for those three products (Spoon, Fork and Knife).  You might need to expand the tree-views in your JSON panels to see all the information like that shown in the screen-shot above.


Setting a Breakpoint in the API Application


Your API application doesn't really run until it is accessed by IIS.  So we need to run the retriever once and then we can set our debugger to attach to the application process.  Once you run your retriever, go to your API application, in Visual Studio click on the "Debug" menu, then select "attach to process".  Then make sure the "Show processes from all users" is checked at the bottom of the dialog box.  Then find the process named "w3wp.exe" and click on it. 




Click the "attach" button.  Then click the "attach" button in the pop-up dialog.  You'll notice that your application is now "running" (you can tell by the red stop button in your tool bar):



Put a break-point in your code.  Run the retriever program and you see your API project pop-up with your break-point waiting for you.  Now you can step through your program and inspect variables (such as the request variables that were received, hint, hint) just like you ran this program using the F-5 key.



Summary


There are a lot of little things that must occur correctly for this API to work properly.  Here are a few things to look out for:

- Your firewall could also block requests, so be on the lookout for that problem.  
- If you don't have the correct URL setup in your retriever, you will not get a connection.  
- The objects you use to serialize and de-serialize JSON data must match between your retriever and API applications.  
- The IIS server must have the correct rights to the API application directory.  
- You need to setup the correct .NET type in your IIS application pool.
- Verify that IIS is running on your PC.
- Verify your inputs on your API by attaching to the w3wp.exe process and breaking on the first line of code.
- Verify your output from your retriever using fiddler.


Getting the Sample Projects

You can download the two sample projects from my GitHub account:  
https://github.com/fdecaire/WebApiDemo