Monday, September 5, 2016

Entity Framework Core 1.0 Smackdown!


In this post I'm going to compare EF 7 or EF Core 1.0 with the previous version of Entity Framework.  I'm also going to throw in the scores for Fluent NHibernate just for kicks.

I've done a few of these head to head comparisons, mostly for my own curiosity, but also to provide myself with a chance to learn a new technology that I know I'll be using for years to come.  Core is the beginning of a whole new paradigm with Microsoft I I'm happy with their decisions.  First, I have to commend them on their choice to include dependency injection right into their new .Net objects.  In addition to that, they have a new in-memory Entity Framework that makes it so much quicker and easier to design unit tests.  


To get started, I have the free version of Visual Studio 2015 (2015 Community).  Previous Visual Studios came with a free version, but they have always been limited to a web version or a desktop version.  Community 2015 seems to have all the features of 2013 Professional.  The Professional version of 2015 has a lot of nice features, but the Community version is a great way to get students "hooked" on Visual Studio.  

You'll need to make sure you have all the updates for VS 2015 before you start.  You might also need to install PowerShell 3.0.  You can search Google (or Bing) for PowerShell and download the latest version for your OS.  I'm currently using Windows 7 on this machine.

To create a Core 1.0 application, your best resource is Microsoft's on-line documentation.  I would start here, and go to the .Net Core download site.  Follow the directions from there.  I'm currently using the Tooling Preview 2.

EF 6.0 vs. EF 7.0 Core

I used my previous smackdown code to test the performance of EF 6.  I have a new machine so I know the timing will be different to start with and I needed a base-line.  Then I copied the performance object and all the POCO and context stuff I used in the EF 6 project to my new .Net Core project.  That's when I discovered something fishy.  EF 7 was performing very bad for selects.  Inserts, updates and deletes were faster under EF7, but a basic select was 100's of times slower.  As I dug around with SQL profiler, I discovered that there was no EF performance problem.  The problem was in the .ToList() conversion.  Not sure why this has become such a slug, but I don't want the .ToList() to skew my performance measurements.

Another problem I discovered was my logic involving the best of 5 runs.  I had coded this incorrectly as thus:

double smallest = -1;
for (int i = 0; i < 5; i++)
    double result = TestUpdate();

    if (smallest == -1)
        smallest = result;
        if (result < smallest)
            result = smallest;

WriteLine("UPDATE:" + smallest);

Currently, I'm using the Resharper ultimate add-on by JetBrains.  This tool quickly identified that "result = smallest;" was unused.  That's because it's backwards, and should have been "smallest = result;".  This bug doesn't throw the result, since the results can vary by a small amount anyway, but it was not intended, so I fixed it in the old version as well as the new.  I also dragged out the old NHibernate test code and refactored to make it mimic the Core 1.0 code as close as possible and ran that to get a base-line for NHibernate.

Here are the results:

As you can see, Microsoft has done a good job of improving the performance of Entity Framework.  I'm impressed.  NHibernate still beats the overall performance, but it's very tiny and real-world differences could go either way.


I have revisited this problem and re-tested all three scenarios (EF6, Core, NHibernate) with the .ToList() method added to the select section.  One issue I ran into was a bug in my update, select and delete tests.  The first test was an insert test and I looped 5 times to get the best insert times.  That was inserting 10,000 records each time.  Therefore, the update, select and delete times were based on 50,000 records.  Not what I intended.  So I inserted code to delete the records for each loop of the insert and I also deleted and inserted 10,000 records in-between each test (in case I ever run one test without the others).  The above list was updated to reflect the times without using the .ToList() on the select statement.  The below list is with the .ToList():

This indicates that Entity Framework 6 selects and brings in the list of data faster than Core or NHibernate.  I have scrutinized the code endlessly.  If anyone can find anything that looks odd in my code, be sure to leave a message and I'll look into it and update this post to reflect any bug fixes.

Test for yourself!

I would recommend downloading my code and trying this on your own system.  First, you can download the .Net Core solution by going to GitHub here and downloading the source.  There is a SQL script file in the source that you can run against your local MS SQL server.  The NHibernate speed test code can also be downloaded from my GitHub account by clicking here.  The Entity Framework 6 code can be found by clicking here.  You can ignore the MySQL part, I didn't use that for this test.