Edit: writing a more specific question, as per comment, I guess my question boils down to: are dynamic properties designs appropriate for applications with rich UI and complex business logic (as opposed to simpler “data entry” applications)? If so, what are some resources where I could learn more about it?
Original question with context:
I’m working on the design and architecture of a web app for managing a city’s public assets (buildings, street lamps, manholes, etc.).
While most objects and properties will be common for every city, there will be some that will require customization (eg. one city will have a field that others don’t; or one city will have a specific requirement on how a given input should be formatted). These things also change depending on [local] laws, so there is potential for frequent change of requirements. Also, even though currently we will have only simple business logic around this data (eg. simple validations such as non-null, mask (formatting), max size), we already envision future use cases that will involve more complex logic (eg. more complex validation such as non-null if another field is non-null; call an API to populate a dropdown).
Given those requirements, we are currently torn between going with a fixed- or dyanamic-property approach. Each has its own pros and cons and while we do have experience with the fixed-property approach and thus know its pros and cons well, we don’t really have experience with projects that go the full dynamic-property route.
I’ve seen a few questions and articles around how to implement dynamic properties (such as the EAV model), but I couldn’t find any that went into the “when” it is best to do that and things such as gotchas to be aware, etc.
So far I thought of three different approaches:
- Fixed properties: Design everything the “traditional” way. Custom fields would be implemented via inheritance. Eg.:
namespace Buildings { class Building(int ID, ...); class BuildingManager {...} }and then for city “Foo”‘s customizationnamespace Buildings.Foo { class FooBuilding(int FooID, int ID, ...) : Building(ID, ...); class FooBuildingManager : BuildingManager { override Validate(Building building) { ... } } }- Pros: Every pro of using fixed properties (clearer code, easy/natural strong/static typing, etc.) plus naturally extends all the way: we can make this design work all the way from the data store to the front end (eg. via component inheritance and specialization).
- Cons: 1) Every customization will require an update of the entire system; 2) if we underestimate how much customization our customers will need this could get very unwieldy very fast, hurting maintainability and readability; 3) depending on how we design the code, this approach may limit what kind of customization we are able to do and/or may lead to bad code (eg. some base code that wasn’t well designed to be overridden).
- Dynamic properties: Use a pattern such as EAV and go fully dynamic. We could also either have every single customer have their own model (I guess this would be best if we use a nosql data store), or design a solution to have it be possible to have some kind of “inheritance” (eg. a “DefaultAttributes” and a “CustomAttributes” table). The backend would be unaware of the semantics of the data and the frontend would build every form dynamically.
- Depending on the kinds of more complex custom validations that we need in the future, this could get very complex or very ugly very fast (or both). One example of how that could get very complex is by implementing support for some kind of programming language/interface (such as a visual node programming system); One example of how it could get very ugly is by storing code in the database to be executed in runtime by the back and front ends.
- Depending on the level of polish and good UX that we need, the same as above could happen (eg.: implement a visual designer and renderer and store things such as style, position, etc in the database in a structured way; or store raw front end code in the database)
- So, pros: 1) very customizable; 2) code duplication low to none; 3) the “possibility” to gracefully evolve the system as requirements change/evolve.
- Cons: 1) Could [d]evolve into either a much more complex system or an unmaintainable mess; 2) could suffer from performance issues on customers with big databases; 3) could end up in a situation where we don’t want extremely unmaintainable code (eg. store code in the model) but don’t have the resources to create a sophisticated customization system either, leading to severely limited options on UI/UX and business logic decisions.
- Mixed: Have fixed properties, “hardcoded” logic and a fixed layout for common/shared data and logic, and dynamic properties for custom fields on top of that.
- Pros: Much of the best of both worlds (maintainability, readability, performance, etc.)
- Cons: Some of the worst of both worlds: 1) UI would have to be split between “base” and “custom” fields, which is not ideal (ideally they would integrate seamlessly), potentially limiting our ability to create good UI/UX; 2) could end up in a situation where one customer wants one of the base fields to behave differently, leading into weird situations such as having to make that originally base field now custom for everyone else, or have a new custom field only for that customer and have they ignore the base one; 3) possibly hard to balance how far to go fixed vs dynamic.
What approach would make sense for our case? Are there any gotchas that we should be aware? Am I overthinking this?
2
IMHO you already laid out all the pros and cons correctly, so there is not much left here we can do for you here.
I would recommend to go with “fixed properties” as far as you can go, and use dynamic ones only as a last resort, in areas where customers themselves want to configure the availability of certain properties, and where you don’t expect any business logic more complex than displaying those properties to the user, or printing them on some report. Otherwise you will run into the issues you identified already.
Of course, when you know your customers manage a certain list of similar assets which all have a name, a vendor, a price, a coordinate or shape on a map, but no more specific business requirements, it may be feasible to make the list of asset types extendable by the customer. Still I would be very careful with this approach, because extra assets for which the customer today has no specific requirement can turn into ones with special business requirements tomorrow.
Note even with the “fixed properties” approach, your system can be made configurable in many ways, for example
-
configurability of certain labels for fixed properties, or
-
the visibility of properties which belong to a common topic (together with certain features which belong to this topic)
These kind of configurations are typically on a much coarser granularity than the configurability on the level of individual attributes (“EAV” model). In the systems I have built in the past, configurability without “dynamic properties” was almost everywhere sufficient and manageable.
Don’t forget, when you add an EAV model to a system, you need some power users or experts for maintaining the complex configurations, and if you don’t have an organization around this, don’t expect your customer to have it. With EAV models, you risk to run sooner or later into the well-known Inner-platform effect. The situation may be different of you are one of the big ERP vendors, having specific staff only responsible for configuring their systems individually (and selling this as an extra service, often much more expensive than the core platform itself).
As for resources: questions for 3rd party resources are off-topic here, but FWIW, I would recommend Fowler’s book “Analysis Patterns”, which presents a lot of ideas how to model systems with the right amount of flexibility. You find many excerpt’s from the book on Fowler’s website here.