Another battle in the stORM

In a fight, there will always be a winner and a loser, but what if this time, there will be none of them? What will happen if no one is better than another or worse? What if, in this story, there is no hero or villain? In this story, we will compare two fighters with distinctive styles, skills, and abilities. Neither the Entity Framework Fighter nor Dapper Fighter has a medal for the best ORM (Object Relational Mapper) yet, but they will compete together in this tournament for the pleasure of the audience.

Author: Gabriel Bârzu, Application Developer

Gabriel Bârzu

Application Developer

Most of the battles have one match, but this time it will be a multi layered quest where in each phase they will show their strengths or weaknesses. For a fair game, we will call on BenchmarkDotNet to help us with more accurate statistics. BenchmarkDotNet is a powerful .NET library that simplifies the creation of accurate and reliable performance benchmarks, enabling code optimization for maximum efficiency. For neutral ground, we will use SQL Server, a relational database management system (RDBMS) developed by Microsoft for storing, managing, and retrieving data using SQL queries. Here, each fighter will face multiple rows and will try to select all of them, find a specific row, and create, update, and delete a part of them. Now that everything is prepared, we still need a referee for the final verdict, but we will see that later.

1. Knowing the fighters

Before we start, let's get to know the fighters better.

Entity Framework

In the left corner, we have Entity Framework, a .NET ORM (Object-Relational Mapper) that lets you work with relational databases using your C# or VB.NET domain classes, minimizing the need to write manual SQL queries. It comes from a well-known family and was developed by Microsoft as an open-source project, with contributions from various developers within the Microsoft community. It has undergone several iterations and enhancements since its initial release, with each version introducing new features and improvements to make it easier for .NET developers to work with databases. It was originally part of the .NET Framework, and a more modern, cross-platform version called Entity Framework Core (EF Core) is now actively developed.

Dapper

In the right corner, we have a smaller fighter, from a "poor village", but appreciated in his branch, raised by the community, the underdog of ORMs (Object Relational Mapper), Dapper. It is a lightweight, simple, high-performance .NET micro-ORM that simplifies mapping database results to objects while letting you control the SQL queries directly for maximum speed and flexibility. It provides a simple and efficient way to map database query results to strongly typed objects, allowing developers to execute SQL queries and map the results to objects without the overhead of a full-fledged ORM framework like Entity Framework. Dapper emphasizes performance and simplicity, making it a popular choice for developers who require fast data access while retaining control over SQL queries and database interactions. Dapper was primarily developed by the team at Stack Overflow as a solution to address performance issues caused by the increasing traffic that the site was experiencing. Since its inception, Dapper has gained popularity among developers due to its simplicity, performance, and ease of use.

2. Their philosophy

Entity Framework

Trading some control for convenience: EF provides a level of insulation from the raw SQL. This can lead to some performance trade-offs but aims to enhance the development experience.

LINQ Integration: EF's support for LINQ (Language Integrated Query) enables you to express database queries directly within your C# code, offering a more natural way to interact with data.

Adaptability: EF is designed to support various database providers and allows for customization of its behavior if needed.

Dapper

Against ORM: Dapper can be seen as a counter-reaction to the complexity and overhead that can sometimes accompany full-featured ORMs.

Developers with strong SQL skills: Dapper assumes you are comfortable with SQL and want to leverage those skills when interacting with your database.

Practicality over purity: Dapper's goal is to get data from the database into your objects efficiently, rather than providing a theoretically perfect object-relational model.

3. Key comparison points

Select tests

These tests are benchmarks designed to compare the performance of Entity Framework Core (EF) and Dapper when fetching all records from an "Employees" table.
The tests were grouped by:

  • Get all the records
  • Get the record based on a random id picked before the starting of the test
  • Get the record based on a property of the employee, in this case the name

The last two suites of the tests were created to see the difference in fetching data when an index is used or not.

Dapper demonstrates a clear advantage over EF in terms of raw performance and memory efficiency, particularly for large-scale data retrieval operations.

  • Get all: Dapper significantly outperforms EF when retrieving all records from a table, with a much lower median execution time (3.158s vs.1.978ms) and less allocated memory (658 MB vs. 1.20 GB).
  • Get by ID: Both EF and Dapper perform similarly when retrieving a single record by its primary key (ID), with negligible differences in execution time and memory allocation.
  • Get by first name: EF is slightly faster than Dapper when retrieving records by a non-primary key field (first name), but Dapper consumes much less memory (14.79 KB vs. 1.11 MB).

Insert tests

These tests are benchmarks designed to compare the performance of Entity Framework Core (EF) and Dapper in two specific scenarios: single record insertion and bulk (multiple records) insertion into an "Employees" table. For the bulk methods it was used a random generated 1000 employees.

Dapper demonstrates superior performance and memory efficiency for both single and multiple-row insertions compared to EF.

  • Single Insert: Dapper significantly outperforms EF when inserting a single record. Dapper takes an average of 845.4 microseconds (µs). EF takes 23,849.0 µs. Dapper also consumes less memory (803.77 KB vs. 31.562 MB).
  • Multiple Inserts: Dapper again outperforms EF for bulk inserts. Dapper takes an average of 124,138.1 µs. EF takes 516,014.0 µs. While Dapper still uses less memory, the difference is less pronounced in this scenario (22.94 MB vs. 645.60 MB).

Update tests

These tests are benchmarks designed to compare the performance of Entity Framework Core (EF) and Dapper in updating an existing record in the "Employees" table. It was used by a random existing fetched employee to update a part of their properties.

Based on these benchmark results, there is no clear winner between EF and Dapper in terms of performance for updating a single employee record. Both technologies exhibit similar execution times and memory usage.

  • Mean execution time: EF and Dapper have very similar mean execution times for updating an employee record (1.921 seconds for EF vs. 1.908 seconds for Dapper). The difference is minimal and might not be statistically significant.
  • Error margin: The error margin for EF is slightly higher than for Dapper (0.0247 seconds vs. 0.0088 seconds), indicating that the results for EF are slightly less consistent across different runs.
  • Memory allocation: EF allocates slightly more memory than Dapper (855.52 KB vs. 809.84 KB), but the difference is not substantial.

Delete tests

These tests benchmark the performance of deleting a single employee record from a database using Entity Framework Core (EF) and Dapper. Same as the update tests a random employee was picked.

The results suggest that EF and Dapper exhibit comparable performance for deleting a single employee record, with a slight edge for EF in terms of mean execution time and consistency. However, Dapper is notably more memory efficient.

  • Mean execution time: There is no difference, suggesting that both technologies perform similarly for this specific operation.
  • Error margin: Dapper has a slightly higher error margin, showing its performance might be less consistent across different runs compared to EF.
  • Memory allocation: Dapper uses significantly less memory than EF for this operation.

4. Battle scenes

In the vibrant realm of .NET development, two mighty forces reigned supreme: Dapper, the lean and swift micro-ORM, and Entity Framework, the feature-rich giant. Their rivalry was legendary, developers whispered tales of epic battles over coffee and in dimly lit Stack Overflow threads. In a dimly lit coding arena, keyboards clack furiously and impatiently.

Dapper: "Ha! Entity Framework, you fake giant! Your endless layers and abstractions slow you down. I am lightning-fast, a minimalist masterpiece! I’m a new era."

Entity Framework: "Speed is only one metric, Dapper. I mastered the power of relationships, change tracking, and LINQ sorcery. Your raw SQL is just a primitive tool."

Dapper: "A primitive? My direct SQL control gives me unmatched precision and power! Your fancy LINQ spells trouble with those unexpected N+1 queries running in the shadows."
*The N+1 problem occurs when Entity Framework generates many separate database queries due to lazy loading instead of efficiently fetching all related data upfront.

Entity Framework: "N+1? A rookie mistake! An ORM fighter like me knows the spells of eager loading. But what about your silly mapping mismatches or lack of change tracking?"

The battle raged on. SQL queries flew across the screen, unit tests ran in the background. Performance benchmarks were called, and architectural patterns were debated.

Wise Developer sipping coffee: "Ah, the eternal ORM dance! But remember, dear fighters, the true enemy is not each other, but the poorly designed data access layers. Each of you has its place. You Dapper for a precision strike, and you Entity Framework for complex domains. The key my little fighters, is not the framework, but understanding your data and choosing wisely."

Confused, both fighters lowered their weapons and wondered if there was a place for both in that world. They called a truce, and a new era has risen, no longer rivals but allies in the quest to develop better applications.

But thinking that at what happened, should any of them win that battle? Are any of them braver fighters? As the wise developer once said, “It depends”.

Additional resources: