Repository+UnitOfWork With Focus on Atomicity. Building Software With The Right Design Pattern.

Sometimes ago, I was reading an article on what makes a senior developer different from an intermediate and junior ones, while the article stretches the fact that Senior Developers don’t think in terms of individual classes, methods, functions or low-level technical particulars, they think in terms of object oriented and application design patterns. I strongly believe in addition that the leap to moving up the developer rank is predicated on the tendency to keep asking yourself questions. Yes, you were able to make faster that response time but can it be further improved? You proffer an algorithm to solve the problem, what is the Time complexity?

I’ve developed this kind of personality over time and it raises it head whenever I’m unto something, no matter how simple that I most time feel wary getting engaged.

I started programming as a Backend Engineer, mostly using .Net. Each time I write codes to interface with the database, lots of questions always pops into my head. The biggest of them are:

Q1: How do I create a re-usable design pattern that is repository agnostic with support of plugging in implementation at will without any need to make any changes from the consumer.

Q2 :Is it possible to push as many entities into the database all at once? rather than writing them 1 by 1. Wouldn’t that vastly improve the response time.

Q3 :If a push to the datastore fails, how does one ensure that the datastore remains consistent by purging out current update?

I know you are saying “This is pretty easy to achieve”, a lot of framework already makes this possible. Well, I know too because I’ve used Entityframework in .net and with Entityframework, this is pretty easy except EF is not available for all datastores and the Lazy loading advantage of EF can be overly abused by most developer resulting in overloaded queries with a knock on performance.

Q4 : What if you find yourself in a situation where you needed 2 or 3 data sources? With a framework like EF? Well, maybe implement multiple DbContext each with it own connection string, but, what if you don’t want to be a slave to EF, what if EF doesn’t support one of the datastore?

Q5 : Also, say tomorrow, something bigger than EF is released, how flexible can you integrate without having to scrap off the entire project codebase? Many may be fine with getting married to a framework and sticking with it forever but I’m not.

All these questions led me to building an answer, a design architecture that uses repository and unit of work pattern with focus on ensuring Atomicity. In this article, I’ll go over the motivations for such design pattern and also discuss its implementation in a step by step approach. I’ll drop the github repo link at the end of article should anyone wants to go over the full code base.

To answer the first Question, “How do I create a re-usable design pattern that is repository agnostic” I’ve defaulted to using an interface called IRepository. The implementation looks like this;

public interface IRepository<T> where T:class,IEntity
{
Task<List<T>> GetEntitiesAsync(GetOption<T> option = null);
Task<T> GetEntityAsync(string entityId, GetOption<T> option = null);
Task<CrudResult<string>> PersistAddEntityAsync(T entity);
Task<CrudResult<bool>> PersistUpdateEntityAsync( T entity);
Task<CrudResult<bool>> PersistDeleteEntityAsync(T entity);
}

Here, I need to mention something. design pattern is all about design thinking. I need not have to pick the newest Google or Microsoft design pattern and then be glue with it for life. If you understand the need and the necessity of building a scalable solution, you’ll think of the best ways to write your codes to achieve it. This is exactly what the guys at Microsoft and Google know how to do best so, they come up with the best design pattern for us all.

So back to our little IRepository. How does it solve re-usability concern?

One of the rules of design pattern is to code to an interface and never to an implementation and that is exactly what we’ve done here. In order to create a functional repository that truly can persist into a data store, an implementation of the IRepository is required. Each repository can define how and what it used to store it data. However, the consumer of our service need not know anything about this implementations because we’ll expose only the IRepository type.

Assume a scenario where we need to log every signup on our system. Say we’re authenticating user from a datasource like SQL Server and making our log to a file. In this case, we’ll need two abstract classes that inherit from RepositoryBase. A UserRepositoryBase and a LogRepositoryBase both extending our RepositoryBase.

public abstract class UserRepositoryBase:IRepository<User>
{
}
public abstract class LogRepositoryBase:IRepository<Log>
{
}

With this derive extensions UserRepositoryBase and LogRepositoryBase, we’ve made it for life. To carter for our user authentication using SQL Server, all that is needed is to provide and implementation of UserRepositoryBase, maybe call it SQLUserRepository and for the log, provide an implementation of LogRepositoryBase that knows how to go about using the FileStream class.

Remember, all we’re exposing to our consumer is the Base class. In case of the UserRepository, We’ll never expose SQLUserRepository implementation, rather, we’ll always expose UserRepositoryBase. With this approach, if we decide on migrating our user data to another datasource tomorrow say Oracle, the consumer of our service need not make any changes. All we need do is to simply provide an implementation of UserRepositoryBase that knows how to talk with Oracle DB and simply inject our new implementation to displace the old SQLUserRepository.

Case 1 solved. But, there’s still a long way to go.

PS: I’ve seen lots of solution to Q1 shut away the model specific derivation of IRepository like the case of UserRepositoryBase and LogRepositoryBase. They do something like this:

public class SQLRepository<T>:IRepository<T>
{
}

My first opinion on this approach is that it’s a lazy approach and it cover for its laziness by making IRepository’s Get methods take an Expression<T>, forgetting that not every datasource engine will support LINQ to create Expressions for example, Azure Table Storage and Redis.

Most developers that go this direction do so because they do not want to preconceived the necessary behavior expected of each entity and then provide a method to execute them. Rather, they want to rely on the generic Get method that offers an expression syntax to pull just anything anytime. This makes the codebase very complicated and unstructured.

Now to Q2; Is it possible to push as many entities into all the datastore all at once? rather than writing them 1 by 1?

Let’s say in our case of user and log. Remember we’re pushing our user to SQL Server while our log goes to a File.

Frameworks like EntityFramework has pattern that answers this questions. EF UnitOfWork pattern, DbContext, allow adding multiple entities before a call to SubmitChanges. But the motivation to work on this architecture isn’t just to build a pattern that will be store agnostic alone, we also have it in mind to build a design pattern that will be Framework independent. Meaning you can still use EF within our design pattern, as well as other frameworks and will all work together with little or no work needed from developer.

So, to ensure multiple entities can be push all at once, our architecture needs to support the UniOfWork pattern. The UniOfWork pattern control the RepositoryBase by telling it when to persist. By successfully implementing a uniofwork pattern over our IRepository, multiple entities can be Queued using our UniOfWork pattern and then commit all at once.

Our UnifOfWork interface looks like this.

public interface IUnitOfWork
{
void RegisterDelete(IEntity entity, UnitOfWorkRepositoryBase<IEntity> entityRepository);
void RegisterUpdate(IEntity entity, UnitOfWorkRepositoryBase<IEntity> entityRepository);
void RegisterAdd(IEntity entity, UnitOfWorkRepositoryBase<IEntity> entityRepository);
Task<bool> CommitChangesAsync();
}

The introduction of the unitofwork to control all our IRepository also introduce something new, the UnitOfWorkRepositoryBase.

We have a IRepository initially, why UnitOfWorkRepositoryBase again?

Well, what’s happening here is basically a separation of concern. The main function of our IRepository is to persist and fetch data from our store. But with the introduction of UnitOfWork, any Repository that will work with it needs to define a way to store it entity using the UnitOfWork.

So, here is the skeletal definition of our UnitOfWorkRepositoryBase

public abstract class UnitOfWorkRepositoryBase<T>: IRepository<T> where T:class,IEntity
{
private IUnitOfWork unitOfWork;
public UnitOfWorkRepositoryBase(IUnitOfWork _unitOfWork)
{
this.unitOfWork = _unitOfWork ?? throw new Exception("IUnitOfWork instance is require for class initialization");
}
public abstract Task<List<T>> GetEntitiesAsync(GetOption<T> option = null);
public abstract Task<T> GetEntityAsync(string entityId, GetOption<T> option = null);
public abstract Task<CrudResult<string>> PersistAddEntityAsync(T entity);
public abstract Task<CrudResult<bool>> PersistUpdateEntityAsync(T entity);
public abstract Task<CrudResult<bool>> PersistDeleteEntityAsync(T entity);
//
public void QueueForRemove(T entity)
{
this.unitOfWork.RegisterDelete(entity, this as UnitOfWorkRepositoryBase<IEntity>);
}
public void QueueForUpdate(T entity)
{
this.unitOfWork.RegisterUpdate(entity, this as UnitOfWorkRepositoryBase<IEntity>);
}
public void QueueForAdd(T entity)
{
this.unitOfWork.RegisterAdd(entity, this as UnitOfWorkRepositoryBase<IEntity>);
}
}

First, We ensure that any implementation of UnitOfWorkRepositoryBase passes an instance of IUnitOfWork which would enable it takes part in a Transactional Operation. We throw an exception if null is passed.

The next methods passes the implementation of IRepository interface over to the derive class that would implement UnitOfWorkRepository

Then we have the real methods to enable our Repository participates in a unit of transaction. The UnitOfWork pattern is basically concern with write operations like Create, Delete and Update which is the motivation for the 3 UnitOfWork esque behavior QueueAdd, QueueDelete and QueueUpdate. Although, we could do without this methods on our UnitOfWorkRepositoryBase by only using the UnitOfWork instance to register entity changes before commit but with this flexibility, it means with both the unitOfWork instance and the UnitOfWorkRepositoryBase instance, we can register entities that will get committed by the unitOfWork instance.

Our pattern is shaping up but we have a problem, a small problem one we hardly notice. If you look closely at our UnitOfWorkRepositoryBase, you’ll realized all it derive classes will need to implement the public methods of IRepository interface like GetEntity, GetEntities, PersistDelete, PersistAdd and PersistUpdate. The problem is when you access an instance of UnitOfWorkRepositoryBase, you’ll still have access to PersistDelete, PersistAdd and PersistUpdate which consumer of our service can misused by abandoning the fact that by using UnitOfWorkRepositoryBase, we want to force you to ensure Transactional Consitency.

public class UserRepositoryBase:UnitOfWorkRepositoryBase<User>
{
.
.
.
.
}
public class SQLUserRepository:UserRepositoryBase{
.
.
.
.
}
static void main(){
IUnitOfWork unitOfWork= UnitofWork.instance;
UserRepositoryBase userRepository= new SQLUserRepository(unitOfWork);
//If this 2 users must be added together,
//A mis-used can come like this
User user1=new User(){...};
User user2=new User(){...};
userRepository.PersistAdd(user1);
userRepository.PersistAdd(user2);
//instead of  
userRepository.QueueForAdd(user1);
userRepository.QueueForAdd(user2);
unitOfWork.CommitChanges();
//or like this
unitOfWork.RegisterAdd(user1,userRepository);
unitOfWork.RegisterAdd(user2,userRepository);
unitOfWork.CommitChanges();
}

So, our best bet is to hide behavior like PersistUpdate, PersistDelete and PersistAdd from UnitOfWorkRepositoryBase instances and to do this?…. we have to make them “protected” which implies only classes that inherit UnitOfWorkRepositoryBase will only be able to provide implementation for PersistAdd, PersistDelete and PersistUpdate. However, if we change the modifier from public to protected, we violate the interface implementation rule that says the access modifier of an interface cannot be changed in derive class. So, how do we go? Well, for now, we say goodbye to IRepository interface and let UnitOfWorkRepositoryBase projects its own base behavior. By going this direction, our UnitOfWorkRepositoryBase looks like this:

public abstract class UnitOfWorkRepositoryBase<T> where T:class,IEntity
{
private IUnitOfWork unitOfWork;
public UnitOfWorkRepositoryBase(IUnitOfWork _unitOfWork)
{
this.unitOfWork = _unitOfWork ?? throw new Exception("IUnitOfWork instance is require for class initialization");
}
public abstract Task<List<T>> GetEntitiesAsync(GetOption<T> option = null);
public abstract Task<T> GetEntityAsync(string entityId, GetOption<T> option = null);
protected abstract Task<CrudResult<string>> PersistAddEntityAsync(T entity);
protected abstract Task<CrudResult<bool>> PersistUpdateEntityAsync(T entity);
protected abstract Task<CrudResult<bool>> PersistDeleteEntityAsync(T entity);
//
public void QueueForRemove(T entity)
{
this.unitOfWork.RegisterDelete(entity, this as UnitOfWorkRepositoryBase<IEntity>);
}
public void QueueForUpdate(T entity)
{
this.unitOfWork.RegisterUpdate(entity, this as UnitOfWorkRepositoryBase<IEntity>);
}
public void QueueForAdd(T entity)
{
this.unitOfWork.RegisterAdd(entity, this as UnitOfWorkRepositoryBase<IEntity>);
}
}

Alright deal done. Now, code call like the above userRepository.PersistAdd(user1) will throw a compile time error because instance of our UnitOfWorkRepositoryBase doesn’t have access to it.

Now, the next thing worth considering is how unitOfWork handles our unit of transaction. Our architecture has a default implementation of the IUnitOfWork interface. All source code available on github but, here is the snippet.

public class UnitOfWork : IUnitOfWork
{
Dictionary<IEntity, UnitOfWorkRepositoryBase<IEntity>> deletedEntities = new Dictionary<IEntity, UnitOfWorkRepositoryBase<IEntity>>();
Dictionary<IEntity, UnitOfWorkRepositoryBase<IEntity>> updatedEntities = new Dictionary<IEntity, UnitOfWorkRepositoryBase<IEntity>>();
Dictionary<IEntity, UnitOfWorkRepositoryBase<IEntity>> addedEntities = new Dictionary<IEntity, UnitOfWorkRepositoryBase<IEntity>>();
private static UnitOfWork instance = null;
public static UnitOfWork Instance
{
get
{
if (instance == null)
instance = new UnitOfWork();
return instance;
}
}
private UnitOfWork()
{
}
public void RegisterDelete(IEntity entity, UnitOfWorkRepositoryBase<IEntity> entityRepository)
{
this.deletedEntities.Add(entity, entityRepository);
}
public void RegisterUpdate(IEntity entity, UnitOfWorkRepositoryBase<IEntity> entityRepository)
{
this.updatedEntities.Add(entity, entityRepository);
}
public void RegisterAdd(IEntity entity, UnitOfWorkRepositoryBase<IEntity> entityRepository)
{
this.addedEntities.Add(entity, entityRepository);
}
public Task<bool> CommitChangesAsync()
{
.
.
.
}
}

One thing worth observing is that we’ve defined our UnitOfWork implementation using a Singleton design pattern. Singleton pattern restrict access to UnitOfWork to just 1 single instance anywhere, anytime.

The reason behind this is nothing more than our objective of ensuring a single point of commit for all our Repositories. If we’d allowed multiple instances of UnitOfWork to be instantiated and use, a mis-used might arise in this manner:

static void main(){
IUnitOfWork unitOfWork1= new UnitOfWork();
UserRepositoryBase userRepository= new SQLUserRepository(unitOfWork);
IUnitOfWork unitOfWork2=new UnitOfWork();
LogRepositoryBase logRepository= new FileLogRepository(unitOfWork2);
//A mis-used can come like this
User user=new User(){...};
Log log =new Log(){...};
unitOfWork1.RegisterForAdd(user,userRepository);
unitOfWork2.RegisterForAdd(log,logRepository);
//then follow with
unitOfWork1.CommitChanges();
uniOfWork2.CommitChanges();
}

Obviously, the above will work but if unitOfWork1 fails during commit, it does’t force a rollback on unitOfWork2 which cripple our idea of Atomicity.

Next, we define a dictionary object that act like a Queue for keeping entities taking part in a unit of transaction. You’d recall earlier that our UnitOfWorkRepositoryBase exposed methods such as QueueForAdd, QueueForUpdate and QueueForDelete. These methods call the unitOfWork Rgister equivalent. This methods are optional definition on our UnitOfWorkRepositoryBase but gives us the flexibility to pass entities over to the unitOfWork instance either through the unitOfWork instance itself or through the UnitOfWorkRepositoryBase instance.

The commitChangesAsync() method does the actual persistence of the entities.

Case cleared for Question 2. Up next to Question 3 which says

“If a push to the database fails, how does one ensure that the database remain consistent by purging out current update?”

I’m particularly concern about this. With no implementation required from developers, I want every derive classes of UnitOfWorkRepositoryBase to automatically enforce Atomicity. Currently, while the unitOfWork commits changes, if one of the entities fails to add, the operation should aborts and then rollback previous commits. This prevents a partial commit from persisting to our datastore.

To Guarantee that every of our UnitOfWorkRepositoryBase will rollback operation when a unit of operation fails, I needed to turn to a useful interface provided by the .NET Framework, the IEnlistmentNotification interface.

The .net framework provide support for Transaction since .net 2.0. The beauty of this Transaction namespace is to solve the problems that might lead to Transactional Inconsistency, the problem we’re currently facing.

But before any operation could take advantage of this useful transactional paradigm, 2 things are needed.

1. A Resource manager
2. A Transaction manager

Luckily for us, the .net provides us with transaction managers such as MSDTC (Microsoft Distributed Transactions Coordinator) which manages all resource manager taking part in a transaction. Tell them when to commit or abort.

However, the Resource manager needs to register itself with the Transaction Manager and this is exactly what IEnlistmentNotification helps to achieve. For any resource manager that wants to take part in a Transaction, it must implement the IEnlistmentNotification interface.

So, at this point, my thought is to make our UnitOfWorkRepositoryBase a ResourceManager and this implies implementing the IEnlistmentNotification expected by the Transaction Manager.

The IEnlistementNotification expose 3 methods with the following signatures

1. void Commit(Enlistment enlistment)
2. void InDoubt(Enlistment enlistment)
3. void Prepare(PreparingEnlistment preparingEnlistment)
4. void Rollback(Enlistment enlistment)

Doing that, our UnitOfWorkRepositoryBase becomes,

public abstract class UnitOfWorkRepositoryBase<T> :  IEnlistmentNotification where T:class,IEntity
{
Boolean enlisted;
Dictionary<IEntity, EnlistmentOperations> transactionElements = new Dictionary<IEntity, EnlistmentOperations>();
.
.
.
.
void enlistForTransaction()
{
if (this.enlisted == false)
{
Transaction.Current.EnlistVolatile(this, EnlistmentOptions.None);
this.enlisted = true;
}
}
public void Commit(Enlistment enlistment)
{
enlistment.Done();
}
public void InDoubt(Enlistment enlistment)
{
Rollback(enlistment);
}
public void Prepare(PreparingEnlistment preparingEnlistment)
{
preparingEnlistment.Prepared();
}
public void Rollback(Enlistment enlistment)
{
foreach (IEntity entity in transactionElements.Keys)
{
EnlistmentOperations operation = transactionElements[entity];
switch (operation)
{
case EnlistmentOperations.Add: PersistDeleteEntityAsync(entity as T); break;
case EnlistmentOperations.Delete: PersistAddEntityAsync(entity as T); break;
case EnlistmentOperations.Update: PersistUpdateEntityAsync(entity as T); break;
}
}
enlistment.Done();
}
}

Officially, our UnitOfWorkRepositoryBase is now a resource manager and can fully take part in a Transaction.

There’s something else, something very important. The idea is make this design pattern support Atomicity by default. How do we achieve that? We’re also building this so as to be datastore agnostic but we don’t even know which datastore anyone using this Architecture will use. Different datastores provide a means to initiate transactional queries and others do not. Still, we need it to guarantee Atomicity.

A smart way we can think of is to assume that by default, no datastore provides a way to initiate a transactional query. Hence, to mimic a transactional query, we’ll have to truly commit the entity and when a rollback occur, we’ll rollback the commit.

The drawback of this approach is that it’s violate the datastore Transactional property called ‘Read Consistency’. This means another user may read the partial commit before it is rolled-back.

The way standard databases like SQL server handle transactional queries is not by persisting directly into the actual table but using an in-memory storage. When the nod is given i.e no error occur, then the in-memory commit is pushed into the real database table. This approach ensure that Read Consistency is maintained. But as said earlier, not every datastore provides a means of handle transactional query and our default implementation works for them. But, what of datastores like SQL-SERVER that provides a neat approach for transactional queries?

Humm, that’s really a big question. We need to find our way to make architecture flexibly work for both scenario without hampering performance in both cases.

To do this, we need to introduce 3 protected virtual methods

1. TransactionalAdd
2. TransactionaalDelete
3. TransactionalUpdate

did you guess our reason for making them virtual? right.. We want derive classes to override them or use our default implementation. Any guess why they’re also protected? Again, right. we don’t want instance member to have access to this method. Why? you should answer that yourself.

So, to ensure maximum performance when using a datastore like SQL-SERVER that provides an inbuilt support for Transaction, we should override this methods and provide proper SQL-SERVER transaction support by also providing a resources manager that attached itself to the current .Net Framework Transaction scope. I did implement example for MongoDB and it’s right in the repository of this project on github too.

Finally, our UnitOfWorkRepositoryBase becomes.

public abstract class UnitOfWorkRepositoryBase<T> :  IEnlistmentNotification where T:class,IEntity
{
.
.
.
.
protected virtual Task<CrudResult<string>> TransactionalAddAsync(IEntity entity)
{
if (Transaction.Current != null)
{
enlistForTransaction();
transactionElements.Add(entity, EnlistmentOperations.Add);
return this.PersistAddEntityAsync(entity as T);
}
else
return this.PersistAddEntityAsync(entity as T);
}
protected virtual Task<CrudResult<bool>> TransactionalUpdateAsync(IEntity entity)
{
if (Transaction.Current != null)
{
enlistForTransaction();
GetEntityAsync(entity.Id).ContinueWith((resultTask) =>
{
T original = resultTask.Result as T;
transactionElements.Add(original, EnlistmentOperations.Update);
});
return this.PersistUpdateEntityAsync(entity as T);
}
else
return this.PersistUpdateEntityAsync(entity as T);
}
protected virtual Task<CrudResult<bool>> TransactionalDeleteAsync(IEntity entity)
{
if (Transaction.Current != null)
{
enlistForTransaction();
GetEntityAsync(entity.Id).ContinueWith((resultTask) =>
{
T original = resultTask.Result as T;
transactionElements.Add(original, EnlistmentOperations.Delete);
});
return this.PersistDeleteEntityAsync(entity as T);
}
else
return this.PersistDeleteEntityAsync(entity as T);
}
}

At this stage, we’ve finally answered Q3. If any commit fails, at least, we can be sure a reversal will happen whether our datastore supports Transactional commit or not.

Because we’ve built our pattern to be framework agnostic. (currently, we can make it work with entity framework without solely relying on entityframework, we can introduce multiple datastores with rest of mind), We can easily conclude we’ve answered Q4 and Q5.

I hope to write more with examples on how other frameworks can easily be integrated into our design pattern in the future. But, for now, the entire project source code can be found here on Github.