Last week’s article comparing and critiquing Clean Architecture and Java Best Practice is a smash hit! If you haven’t already read it, it should go do that right now so that you can put this week’s article in perspective, which is also about Clean Architecture.
Make sure you check out the comments that resulted from me sharing it on LinkedIn too. Particularly the exchange between Alex Chernyshev and I.
Alex had responded to my promise to provide code examples to another commenter by sharing a link to one of Joel Spolsky’s posts and talking about how my “small little blocks with arrows” broke down in the face of real requirements.
I was kind of honored that someone would use something from Joel to refute my claims. While I don’t always agree with him, he’s clearly a very bright guy.
Alex, if you’re reading, this article is the response that I promised you.
Does Sorting, Filtering, Paging, or Multiple Views on a Complex Entity Break Clean Architecture?
It would be easy for me to turn my nose up, say something like, “Those are just implementation details,” and move on. The developer in me wants to let it go at that, but the teacher and mentor in me won’t allow it.
It’s a pretty cheap out – even if it is the truth. So, this week, we’re going to take a look at these specific questions.
Pagination Is a Cross-Cutting Concern, But Can Be Factored Into Request/Response Model And Into Calls To Gateways
This was my first claim which I responded to Alex with. He seemed astonished I would make such a claim without knowing the DataSource implementation, the amount of data present, and whether caching was involved.
He seemed very concerned with cache expiration.
Your Core Application Should Not Care About Caching
The core of your application should be business logic. Period.
While caching can be vital in some cases, that doesn’t mean that you let it become intrusive. However, if you do let it become intrusive, and it’s in the Gateway implementations, that still has no impact on the business logic or view.
There are many ways you can bring caching in – several of which can be almost entirely transparent to the application – Gateway implementations included.
The cache expiration? That’s something you configure on the cache, and not in your application. It’s off in a configuration file, configured on the cache server, or – as a last and worst option – in the code the brought your cache to life in a particular Gateway implementation.
Pagination IS a Cross-Cutting Concern, and Clean Architecture Supports That!
Let’s think for a moment about how pagination works, and the pieces that are required to support pagination of a list type operation. The last time I implemented paging, I tracked:
- The Current Page Number
- The Desired Page Size
- The Total Number of Pages
Additions to the Request Model
The Clean Architecture suggests using a type called RequestModel as input to the Interactor. The inference is that this becomes a parameter when you call the Interactor.
Uncle Bob says nothing about what this type looks like, except that it should be a flat data structure. That is, it has no getters nor setters – just bare fields.
To support paging, we could go two ways:
- Add pageSize and pageNumber as fields of RequestModel (or a subclass/sub-interface)
- Add another type (let’s call it PageInfo) as a field of RequestModel and have the fields reside on that new type.
Additions to the Response Model
The output type in Clean Architecture is the ResponseModel. Just like the RequestModel, it’s a standardization of what comes out, but there is no prescription about what it looks like other than that it be flat.
This one is supposed to be consumed by the Presenter implementation, which generates the ViewModel.
We have the same options for incorporating paging information here as we did with the RequestModel. The only difference is that we need to be concerned about the total number of pages. The view needs this information to figure out – based on current and total pages – if the user can navigate forward or backward.
I’d probably fold all the fields into PageInfo and hang that on Requests and Responses for list type data. It’s the approach that makes sense.
If you’re real picky about the fact that you don’t need total pages on the request, you could separate each concern into its own type.
Paging is an Implementation Detail Fulfilled by the Gateway Implementations
So, if we have all the information we need from the view to know which page to generate and how big that page is, and we have some type of Template Method pattern in our Gateway implementation, paging is pretty simple.
How paging is performed is entirely up to the Gateway implementation though.
Some platforms support paging directly with DML; some do not. Most contemporary persistence frameworks will help you implement paging in either case – resorting to setting the right properties on the bare JDBC objects if that’s what it takes for your chosen platform.
Sorting and Filtering – A Million Records! Can Clean Architecture Do It?
I tend to implement Sorting and Filtering the same way as Paging. You need information sent on the RequestModel that is passed through to and implemented by the Gateway implementations.
Remember to pass these back in the ResponseModel too.
This is less straight-forward to implement than the paging – I won’t lie to you. That doesn’t mean it can’t be done though.
How can I say that? Well, because – just like the paging – I’ve done it.
If you’re using a nice persistence framework, it’s pretty easy to map Entity fields to database fields and use the inputs to configure a query before execution. There’s definitely work and code involved, but the nice thing is that – once you’ve written that big – you can sort and filter on arbitrary fields.
Oh, and getting to large datasets: who cares? There is no code you’re going to write differently for 100 rows or 1,000,000 rows.
You might restrict your page size. That would probably be wise.
Beyond that, it’s up to the database to handle. If performance is poor, you should have a look at the indices available and whether or not there may be some partitioning scheme that might benefit you.
It’s amazing the tricks you can apply to improve database performance. All the good ones? The ones that make your application’s attempts at improving performance look puny? They’re done at the database level and have nothing to do with your application design.
That assumes your queries are reasonable. Teaching you to write sane queries is beyond the scope of this article. However, what I can say is that Clean Architecture makes no stipulation about how you write your queries – that is an implementation detail.
Another thing worth pointing out is that the product of the row and column count is what will cause bad performance. So, you might be able to pull 100 rows back from one table in the time it takes to pull 25 out of another one.
I/O is expensive. The more you do, the more you pay.
Will Clean Architecture Support Multiple Views of Complex Entities?
This was Alex’s final question. His scenario is a really large table in a CRM – the client table.
He needs to retrieve data from this table as:
- A view that has a subset of fields from the table which can be paged through.
- A view that has a subset of fields from the table which can be filtered – possibly by fields which are not present in the view.
- A list of id/name to be used when assigning a client to various different “things”.
As you can tell from the above description of how I do pagination, sorting, and filtering, it’s not a real special thing. To me, asking for a subset of fields isn’t that special either, but let’s consider the impact to the elements of Uncle Bob’s Clean Architecture anyway.
No Alternation Required to RequestModel and Response Model – Probably
There is no impact to the RequestModel or ResponseModel beyond that which I’ve already stated. That is, assuming you don’t make the set of returned fields configurable – then you would need the RequestModel to tell you which fields to return.
The Secret Rests in The Gateway Implementation … And The Presenters
It doesn’t matter whether you’re allowing the view to tell you the fields that should be emitted or you code them into the query itself – the Gateway implementation owns determining which fields are going to be returned.
Do you need a new Entity? Probably not, but you could come up with a simplified Entity if you really, really wanted.
I’d probably use the same Entity for both of the first views.
The view that really makes a person wonder about needing a new Entity is that tiny list that’s used for assignment. Remember that our Entity has many, many fields.
Even if our first two views which pull a subset of fields may not work ideally. They may and they may not.
The real trick here – rendering-wise – is the Presenter. The presenter for the first two may be the same, but it’s probably different for the last one. In that case, you want a simple list of id/name and more would be overkill.
You’d need different controllers for the first two and the last one. I really have a hard time seeing the first two as being different, so I’d use the same controller for both of them.
The last one? Well, you may be using a different Entity (perhaps one specialized to label/value operations) for this one, so you’re going to have a different Controller, Interactor, and you’re probably going to need a different method to call on your Gateway.
If you’re taking the Interface Segregation Principle serious, you might have an entirely separate Gateway.
That seems like overkill to me.
Wrapping it Up
It’s interesting to me that so many folks look at Uncle Bob’s Clean Architecture and feel it cannot be applied to real applications due to one implementation concern or another.
That’s part of the design challenge, and it’s the finer detail – the design – that needs to be fleshed-out within the context of the architecture.
If you need a certain input or a certain output, you factor it in. … but you factor it into the appropriate spot. Hopefully, this article has helped you see how you might do that.
The trick, and I’m starting to feel like I’ll be saying this a lot, is that the concern is represented (read: defined) in the business logic and then used by other layers.
Paging, sorting, and filtering are first-class citizens when it comes to concerns. They aren’t big enough to deserve a place at the table when drawing the architecture, but they definitely need to be included in the design.
How you fulfill these is up – ultimately – to the Gateway implementations though. Oh, and your ability to translate the messages that come in the request into the appropriate calls on your chosen persistence platform or framework.
I’m really glad Alex brought these concerns to light. As I told him, I imagine there are more people than just him that have them.
Hopefully, I’ve helped dispel Alex’s concerns. If not, I invite both Alex and you to comment at the bottom of this post and help me understand how or why I haven’t.
I love to help! I actually think everyone that writes code should consider themselves as being part teacher.
Not everyone takes up that mantle though. We’re all different, and that’s okay.
I think next week I need to start getting into code.
Well, that’s it for this week! I hope you found this week’s article to be helpful. If you have concerns which I haven’t addressed here – again – I invite you to comment and let your questions be known!
If you enjoyed the article, please remember to like, share, or tweet it to your own audience or network. It helps way more than you could imagine.
Until next week, this is Eddie Bush with Craftsmanship Counts reminding you to keep your tests passing and your code clean!