Last week’s article comparing and critiquing Clean Architecture and Java Best Practice is a smash hit! If you haven’t already read it, it should go do that right now so that you can put this week’s article in perspective, which is also about Clean Architecture.
Make sure you check out the comments that resulted from me sharing it on LinkedIn too. Particularly the exchange between Alex Chernyshev and I.
Alex had responded to my promise to provide code examples to another commenter by sharing a link to one of Joel Spolsky’s posts and talking about how my “small little blocks with arrows” broke down in the face of real requirements.
I was kind of honored that someone would use something from Joel to refute my claims. While I don’t always agree with him, he’s clearly a very bright guy.
Alex, if you’re reading, this article is the response that I promised you.
Does Sorting, Filtering, Paging, or Multiple Views on a Complex Entity Break Clean Architecture?
It would be easy for me to turn my nose up, say something like, “Those are just implementation details,” and move on. The developer in me wants to let it go at that, but the teacher and mentor in me won’t allow it.
It’s a pretty cheap out – even if it is the truth. So, this week, we’re going to take a look at these specific questions.
Pagination Is a Cross-Cutting Concern, But Can Be Factored Into Request/Response Model And Into Calls To Gateways
This was my first claim which I responded to Alex with. He seemed astonished I would make such a claim without knowing the DataSource implementation, the amount of data present, and whether caching was involved.
He seemed very concerned with cache expiration.
Your Core Application Should Not Care About Caching
The core of your application should be business logic. Period.
While caching can be vital in some cases, that doesn’t mean that you let it become intrusive. However, if you do let it become intrusive, and it’s in the Gateway implementations, that still has no impact on the business logic or view.
There are many ways you can bring caching in – several of which can be almost entirely transparent to the application – Gateway implementations included.
The cache expiration? That’s something you configure on the cache, and not in your application. It’s off in a configuration file, configured on the cache server, or – as a last and worst option – in the code the brought your cache to life in a particular Gateway implementation.
Pagination IS a Cross-Cutting Concern, and Clean Architecture Supports That!
Let’s think for a moment about how pagination works, and the pieces that are required to support pagination of a list type operation. The last time I implemented paging, I tracked:
- The Current Page Number
- The Desired Page Size
- The Total Number of Pages
Additions to the Request Model
The Clean Architecture suggests using a type called RequestModel as input to the Interactor. The inference is that this becomes a parameter when you call the Interactor.
Uncle Bob says nothing about what this type looks like, except that it should be a flat data structure. That is, it has no getters nor setters – just bare fields.
To support paging, we could go two ways:
- Add pageSize and pageNumber as fields of RequestModel (or a subclass/sub-interface)
- Add another type (let’s call it PageInfo) as a field of RequestModel and have the fields reside on that new type.
Easy, peasy.
Additions to the Response Model
The output type in Clean Architecture is the ResponseModel. Just like the RequestModel, it’s a standardization of what comes out, but there is no prescription about what it looks like other than that it be flat.
This one is supposed to be consumed by the Presenter implementation, which generates the ViewModel.
We have the same options for incorporating paging information here as we did with the RequestModel. The only difference is that we need to be concerned about the total number of pages. The view needs this information to figure out – based on current and total pages – if the user can navigate forward or backward.
I’d probably fold all the fields into PageInfo and hang that on Requests and Responses for list type data. It’s the approach that makes sense.
If you’re real picky about the fact that you don’t need total pages on the request, you could separate each concern into its own type.
Paging is an Implementation Detail Fulfilled by the Gateway Implementations
So, if we have all the information we need from the view to know which page to generate and how big that page is, and we have some type of Template Method pattern in our Gateway implementation, paging is pretty simple.
How paging is performed is entirely up to the Gateway implementation though.
Some platforms support paging directly with DML; some do not. Most contemporary persistence frameworks will help you implement paging in either case – resorting to setting the right properties on the bare JDBC objects if that’s what it takes for your chosen platform.
Sorting and Filtering – A Million Records! Can Clean Architecture Do It?
I tend to implement Sorting and Filtering the same way as Paging. You need information sent on the RequestModel that is passed through to and implemented by the Gateway implementations.
Remember to pass these back in the ResponseModel too.
This is less straight-forward to implement than the paging – I won’t lie to you. That doesn’t mean it can’t be done though.
How can I say that? Well, because – just like the paging – I’ve done it.
If you’re using a nice persistence framework, it’s pretty easy to map Entity fields to database fields and use the inputs to configure a query before execution. There’s definitely work and code involved, but the nice thing is that – once you’ve written that big – you can sort and filter on arbitrary fields.
Oh, and getting to large datasets: who cares? There is no code you’re going to write differently for 100 rows or 1,000,000 rows.
You might restrict your page size. That would probably be wise.
Beyond that, it’s up to the database to handle. If performance is poor, you should have a look at the indices available and whether or not there may be some partitioning scheme that might benefit you.
It’s amazing the tricks you can apply to improve database performance. All the good ones? The ones that make your application’s attempts at improving performance look puny? They’re done at the database level and have nothing to do with your application design.
That assumes your queries are reasonable. Teaching you to write sane queries is beyond the scope of this article. However, what I can say is that Clean Architecture makes no stipulation about how you write your queries – that is an implementation detail.
Another thing worth pointing out is that the product of the row and column count is what will cause bad performance. So, you might be able to pull 100 rows back from one table in the time it takes to pull 25 out of another one.
I/O is expensive. The more you do, the more you pay.
Will Clean Architecture Support Multiple Views of Complex Entities?
This was Alex’s final question. His scenario is a really large table in a CRM – the client table.
He needs to retrieve data from this table as:
- A view that has a subset of fields from the table which can be paged through.
- A view that has a subset of fields from the table which can be filtered – possibly by fields which are not present in the view.
- A list of id/name to be used when assigning a client to various different “things”.
As you can tell from the above description of how I do pagination, sorting, and filtering, it’s not a real special thing. To me, asking for a subset of fields isn’t that special either, but let’s consider the impact to the elements of Uncle Bob’s Clean Architecture anyway.
No Alternation Required to RequestModel and Response Model – Probably
There is no impact to the RequestModel or ResponseModel beyond that which I’ve already stated. That is, assuming you don’t make the set of returned fields configurable – then you would need the RequestModel to tell you which fields to return.
The Secret Rests in The Gateway Implementation … And The Presenters
It doesn’t matter whether you’re allowing the view to tell you the fields that should be emitted or you code them into the query itself – the Gateway implementation owns determining which fields are going to be returned.
Do you need a new Entity? Probably not, but you could come up with a simplified Entity if you really, really wanted.
I’d probably use the same Entity for both of the first views.
The view that really makes a person wonder about needing a new Entity is that tiny list that’s used for assignment. Remember that our Entity has many, many fields.
Even if our first two views which pull a subset of fields may not work ideally. They may and they may not.
The real trick here – rendering-wise – is the Presenter. The presenter for the first two may be the same, but it’s probably different for the last one. In that case, you want a simple list of id/name and more would be overkill.
You’d need different controllers for the first two and the last one. I really have a hard time seeing the first two as being different, so I’d use the same controller for both of them.
The last one? Well, you may be using a different Entity (perhaps one specialized to label/value operations) for this one, so you’re going to have a different Controller, Interactor, and you’re probably going to need a different method to call on your Gateway.
If you’re taking the Interface Segregation Principle serious, you might have an entirely separate Gateway.
That seems like overkill to me.
Wrapping it Up
It’s interesting to me that so many folks look at Uncle Bob’s Clean Architecture and feel it cannot be applied to real applications due to one implementation concern or another.
That’s part of the design challenge, and it’s the finer detail – the design – that needs to be fleshed-out within the context of the architecture.
If you need a certain input or a certain output, you factor it in. … but you factor it into the appropriate spot. Hopefully, this article has helped you see how you might do that.
The trick, and I’m starting to feel like I’ll be saying this a lot, is that the concern is represented (read: defined) in the business logic and then used by other layers.
Paging, sorting, and filtering are first-class citizens when it comes to concerns. They aren’t big enough to deserve a place at the table when drawing the architecture, but they definitely need to be included in the design.
How you fulfill these is up – ultimately – to the Gateway implementations though. Oh, and your ability to translate the messages that come in the request into the appropriate calls on your chosen persistence platform or framework.
Closing Thoughts
I’m really glad Alex brought these concerns to light. As I told him, I imagine there are more people than just him that have them.
Hopefully, I’ve helped dispel Alex’s concerns. If not, I invite both Alex and you to comment at the bottom of this post and help me understand how or why I haven’t.
I love to help! I actually think everyone that writes code should consider themselves as being part teacher.
Not everyone takes up that mantle though. We’re all different, and that’s okay.
I think next week I need to start getting into code.
Well, that’s it for this week! I hope you found this week’s article to be helpful. If you have concerns which I haven’t addressed here – again – I invite you to comment and let your questions be known!
If you enjoyed the article, please remember to like, share, or tweet it to your own audience or network. It helps way more than you could imagine.
Until next week, this is Eddie Bush with Craftsmanship Counts reminding you to keep your tests passing and your code clean!
Hi. This is a really good post about Clean Architecture. I’m curious about how do you handle cross cutting stuff like logging, dependency injection, getting the current date and time, and so on…
Great questions!
I apply the dependency inversion principle to logging. It’s too pervasive not to, and it’s easy to add (for example) an SLF4J implementation/adapter that simply delegates to SLF4J (or whatever the framework de-jure is). This may also work for “time”, but – really – I tend to use Clock or something similar, now that we have that.
Dependency injection occurs at a higher level. It’s arguably outside the scope of the system’s concern. However, more practically, I find that I use a combination of Factories/Providers and Spring DI. I mean … why not? The factories can be useful for tying things together so that you have fewer individual pieces to wire together. These would target a mechanism layer such as web or database.
ServiceLoader (in Java) can be useful for DI too. For example, to load the Factory/Provider implementation.
Some people tell me that factories have fallen out of vogue. I tell them they give too much to Spring (or whichever DI they use).
Hope it helps!
Where should one store pagination state? I mean pagenumber on the client side?
If you think about it, pagination is a cross-cutting concern. We receive cues about how to paginate on a request and have to pass those cues all the way to whichever data store is being used.
Who delegates to persistence? Probably a service of some sort, delegated to by a story. So, we have to factor this into the story request, the service call, and the persistence call. Most of this is probably just pass-through, unless you have – for example – a requirement to only allow users to see page 1 (a limited trial mode, maybe?). Then, the service might be interested in preventing requests beyond page 1 (or 2, or whatever) from being fulfilled.
The rubber meets the road in your persistence implementation. You need to convert from your core model’s representation to whatever is needed for your persistence implementation. And, you’ve got to collect information to provide the caller with what they need to know where they are at in the overall dataset. So, you’ve got to account for communicating back max records or pages, along with the actual data desired.
So the response from persistence needs to reflect pagination, as do the response from the service and story. And, of course, your presenter.
This means that the core model, even though it likely does next to nothing with pagination information, must define the form that pagination information will be represented. And, that layers which use a different form (most likely persistence, but possibly web) must take responsibility of converting to/from that core representation as-needed.
Hope it helps 🙂
I am trying to use clean architecture in mobile app, is not like a web app with reload/refresh and is not always about the view. The app gathers some client side state that is needed in further use cases. How should I handle this client side state? As some in-memory datasource and use a service/repository to get data from it.
For example (poor example), after a use cases my guest user chosen X, some use cases in the future should take in consideration that use is in X state.
I think the question is related to client side state, witch is not on the server or api but is needed for some use cases that use repositories for API requests, etc.
I haven’t had this problem, but I’ll give you my view anyway. Don’t worry – it’s totally free! 🙂
Sounds like you have an operation that is related strictly to view logic. If you were on the web, and it was acceptable for it to be in effect on a single browser, you might store this in a Cookie. But, if the change of the setting needs to be seen beyond the current browser, you’d basically have to send those settings to the back-end to be stored in … some sort of storage device. Maybe a relational device, but that isn’t significant – it could be document storage or even anything else that would persist across sessions/devices (and restarts on the server-side!)
You’re not implementing a functional requirement – this state is ancillary to that and only impacts how the view is rendered. That doesn’t mean you can’t apply the same technique you’re using for functional requirements though. You just might have to create some things on the ‘client’ to support that approach which you had not anticipated.
Bringing it to mobile … I don’t get to play with mobile that much, but I believe you have a way to store user preferences as they relate to an application. This may be acceptable as a place to store these things. However, if you change the state on your phone and it should also be shown on your tablet, (and maybe web too?) it would need to get sent to the server.
If your mobile app and server-side are using the same language, and you have a very clean separation between the ‘clean framework’ pieces and the code that builds on it, you could put that in a single JAR (or GEM, or DLL) and include that in both places to help minimize duplication.
That’s my 2 cents. I hope it at least gets you thinking in a useful direction!
Thanks for stopping by and asking for input!
Eddie
Thank you for your answer, it helps me go further and understand. Mobile app not being as a web app with reload/refresh confuses me somehow.
1.User authentication token should go on some in-memory storage and used with some repository?. Then for following requests repos should talk between them for the token?
(viewmodel & controller are on the same class in my app)
I was using the guest user example because I think I build some of my logic in viewmodel (in flutter, viewmodel as top inherited widget it lives as long as the application). This logic resides in viewmodel. For example: after calls to usecase the app is in GUEST mode, this data is saved in viewmodel as a state(changing state redraw screen*). I need this to be available on other usecases from other viewmodels. This is the state I am talking about, I think is saved on the wrong place, it should go on some in-memory/object and implemented within a repository.
AppStateRepository (get, set app state). I don’t think that passing to each usecase the app state from appviewmodel is a solution.
*How flutter uses “lift the state up”: Inherited Widget(AppState – viewmodel)->Widgets…->Inherited Widget(UserState – viewmodel)->Profile Widget(view). Changes to Inherited widgets redraw the below widget tree. Widgets are classes.
huh 🙂
I found something that is closer of what I was thinking:
https://medium.com/insiden26/reactive-clean-architecture-with-android-architecture-components-685a6682e0ca
“Data
The responsibility of this layer is to manipulate and coordinate the different sources of data. It has three main components: the repository, the network services and the reactive store .
The main idea is to get the data from the external source through the network services, store this data inside the reactive store, which can be seen as the internal data source, and expose the data through the repository to the upper layer.”
Makes sense!
Glad I could help you along the path!