RetrofittingADomainModel

Thu Oct 2, 2003 5:11 am

What kind of experience do people have with retrofitting a domain
model to an application that didn't get built on top of one in the
first place?

My client's current architecture has a Web interface using the
Jakarta Struts framework, which talks to stateless session EJBs.
These EJBs fetch data from a relational database through stored
procedures using XML-format structured input and output parameters.

The problem is that there is no domain model. The raw data that the
domain model would organise and structure is merely wrapped in the
return value objects from the EJB's business methods, which the EJBs
construct from the stored procedures' returned XML. The value objects
are then used in the Struts handlers to present the data to the user,
and to pass modification and update requests back through the EJBs to
the database.

We have a host of other problems with the application, but doing
anything meaningful to improve the application is made horribly
difficult because of the lack of a proper domain layer and a badly
named bunch of unrelated value objects that contain all the domain
data.

Between components, coupling is really high and cohesion is really
low.

Not least of the problems that this causes is that it is horribly
difficult (in many situations effectively impossible) to write unit
tests, because it's basically impossible to isolate any one layer or
section of the application to call, without invoking a branching
cascade of method invokations that end up all over the place in the
database.

There's no real fundamental difficulty here. I've drawn up the domain
model as it exists conceptually as a collection of POJOs, and I'll be
designing a clean interface to them, probably as a small number of
stateless session beans on top of either BMP entity beans (not
decided yet), as per the pattern discussed elsewhere in this group.

And then it's just a matter of RefactoringMercilessly to transition
to the new architecture.

Minor detail that something like 50% of the application (c. 600
classes) will change ...

Is this a lost cause?

I'm unlikely to get any explicit funding for this, on the basis
that "hey, it works, the business owners couldn't care less about
massaging the code" -- so it has to be done under the radar along
with other development work that's explicitly in the project's scope
and budget. Though of course as soon as the business owners start
wanting signifiant new functionality or 3rd-party external business
system integrations and such, the architecture really needs to be in
good shape.

AllanHalme

---

Wow. That's an unfortunate and (I suspect) all too common situation, Allan. I
think that's a prime example of what Eric meant when he wrote in another thread
"Now we have all the technical hoo-hah, and we seem to have lost the original
vision". Yours is an example of architects likely focusing on technologies
rather than domain abstractions and responsibility allocations (one question:
where is the logic? In the Struts actions? In stored procedures?)

More and more I believe that the most powerful way (maybe the only way?) to get
the attention of the business people is to frame concerns and proposals in
financial terms (see also
http://www.amazon.com/exec/obidos/tg/detail/-/0201728877). I know these are
hard to quantify, but there are two measures of an architecture (or of competing
alternative architectures) that should matter to a business person: its
performance in production, and the productivity it affords its development team.
If performance is poor, then customer referenceability (and the vendor's
reputation) will suffer. If productivity is poor, then the vendor will miss
market opportunities. Both of those prospects should strike fear into the heart
of a business person.

Without compelling evidence that domain-driven design would improve performance
and productivity in your specific context, you're left with only your beliefs
and sense of aesthetics. Hopefully they are shared by your client's team, and I
wish you the courage to refactor towards domain-driven design. I'm trying to
compile the evidence in my context, but meanwhile I'm not overlooking the
positive morale effects of a more aesthetically pleasing design and
implementation.

RandyStafford

---

Allan,
It is a common situation. Unfortunately, changing 600 classes "under the radar" doesn't sound practical to me. Maybe, if you are fairly autonomous and can include time for it in estimates of other tasks. But I've gradually come around to the opinion that there has to be some minimum of management support for major design changes.

It would be great if we could quantify the cost of a bad design, but I don't know how. (I'll try to take a look at the books mentioned in this discussion.) When I'm trying to pursuade managers of the need to invest now, I usually talk with them about he functionality they hope to have in the future and about the amount of unpredictability in that. I try to get across to them that some ground work has to be done and that it could be too late if we wait. I use metaphores and examples. (The introduction to the "Supple Design" chapter has some of my thoughts on this, though pitched to the technical person.)

My arguments change their minds (enough to lead to action) less than half the time.

Still, it is the best option. Without that agreement, we risk falling into a sort of heroic self-dellusion that we can single-handedly save the project in spite of the others. This is very stressful. It occasionally works in the short run, but then noone really understands why the project succeeded, so it isn't any easier the second time. Maybe others have had better experiences. But I think it is better to focus on getting buy-in. Maybe you could refactor one small part to the point that you could show to the business people how quickly you can change things on that part?

EricEvans

---

EE> It would be great if we could quantify the cost of a bad design,
EE> but I don't know how.

The best tool I've found for that is comparisons with examples from
other industries that are easy for people to grok (e.g. the Edsel).

Henry Petroski's books are full of great stories and examples about
the importance of design in engineering.

GreggIrwin

---

> Allan,
> It is a common situation. Unfortunately, changing 600 classes "under
> the radar" doesn't sound practical to me.

You're right. I'm not very happy myself ...

> Maybe, if you are fairly
> autonomous and can include time for it in estimates of other tasks.

... and this is what I'm hoping to be able to do.

I'll actually be able to get rid of about 100 classes, make minor
tweaks to another 100 or so, and then just do the real logic
refactoring in just some dozens of other classes to get this done.

> But I've gradually come around to the opinion that there has to be
> some minimum of management support for major design changes.

I agree with your wisdom. Fortunately, my client project manager
is a good listener and I'm planning on trying to explicitly get this
domain refactoring into the next release's formal scope, which will
give us a real schedule and management visibility.

Regards,
AllanHalme

---

> What kind of experience do people have with retrofitting a domain
> model to an application that didn't get built on top of one in the
> first place?

I have experience with this, more than once. Tying this into the
economics of domain-driven design, I'll tell this specific relatively
short story...

I was working on a team building video conferencing systems. The
original product had been pretty literally slammed together for 16-bit
windows systems. (Yes, this was some time ago.) I joined the group after
the first release had been shipped.

After working on some not very well designed code for a few months, one
of my bosses (had a couple of them) asked if I wanted to work on a new
team re-implementing for 32-bit windows. Knowing there was no way much
of the existing code could be ported directly (too many "globals" for
communicating directly between DLLs), I saw this as a great opportunity.

I worked on the conference establishment code first. I took bits and
pieces of the exiting code or at least logic and embedded it within a
domain-specific vocabulary. E.g. seeing in the code several ways to
establish a call, and a lot of IF/ELSE and SWITCH to keep those paths
separate, I created a Call object. The Call was used to establish a
connection, with subclasses for TCP/IP, ISDN, etc. I also separated out
the code for where the call comes from (i.e. where do you get the
information to make the call?)

Not long after the benefits of this refactoring paid off. Some marketing
guy wanted a demo, and he wanted some new features, in particular to
establish a call from an address book, make the connection via TAPI,
etc. The 16-bit team just gave up, they did not have the time or people
to devote to untangling the code for these new features. In the 32-bit
case, two of us did the work in about a day elapsed time.

A guy new to the project, but familiar with TAPI, was able to create a
couple new subclasses with the expected interface. I did some very minor
tweaking of the existing interfaces (which then made the subsequent
development for the system even more reusable).

So in this case refactoring with the domain in mind took a project's
desired new features from impossible to fit into the schedule to "piece
of cake", which got a fair bit of notice in the organization.

PatrickLogan

---

Yesterday Martin Fowler's Patterns of Enterprise Application Architecture
arrived in the mail, and he says some things in there that got me thinking.

His pattern Transaction Script [1] is more or less what our current
architecture does. (He also has a Domain Model pattern -- Eric, what's your
view on that? I'm planning on buying your book but haven't yet got it or read
it.)

Chapter 2, "Organizing Domain Logic", appears to me to be a really good
succinct description of the issues I'm facing when I'm designing and
considering retro-implementing a domain model. What strikes me most is where
he says "[...] how do you choose between the three patterns? It's not an easy
choice, and it very much depends on how complex your domain logic is."

Well. Considering that my domain logic is actually pretty simple (complexity
value is roughly 2.71) and that the current implementation has value in that
it exists and works, I'm inclined to just steadily refactor toward a Domain
Model and a Service Layer [2], if for no other reason then for the sake of
elegance.

My point being that in my case it looks like it may not be worth trying to get
a major domain refactoring onto the next release's formal scope and budget.

AllanHalme

---

That section in Martin's book was actually the first reference to my book! (See page 120, [Evans]. It refers to it as "Domain Driven" because that was my working title, later expanded to "Domain-Driven Design".)

Martin does a good job there of explaining when a domain model really pays off and when another approach may work just as well. The Transaction Script, for example, is easier for a lot of people when dealing with problems of moderate complexity. (I say "easier for some people" because, the way my brain is wired, I often find even quite simple problems most natural to address with models... simple models, of course.)

EricEvans

---

Sun Jan 25, 2004 1:47 pm

[I'm in a long-winded mood, get your fresh cup of coffee first.]

I posted last year regarding retrofitting a domain model to my
current project -- here's where I stand, and I've got some new
questions. [I'm buying the DDD book tomorrow and haven't read
it yet, so I'm not up to speed on Evans' terminology re
Controllers and Repositories, beyond what I've been able to
glean from between the lines in this group.]

We're now just about to start the development phase for our new
release, and we've got seven weeks till mandatory code freeze.

The current architecture has two layers, a UI layer modeled on
Struts, and a pretty messy data-access layer that accidentally
mostly follows Fowler's Transaction Scripts pattern, with ad
hoc value objects as request parameters and response objects.

All new and modified code, however, will follow the following
architecture. (TDD + unit testing + refactoring + continuous
integration + UML + removal-of-cubicle-walls-between-developers
rulz.)

- UI
- Domain
- Persistence

The domain layer is a collection of POJOs and we constantly
maintain a UML diagram thereof that we use to talk amongst
ourselves and, so some extent, with the project's internal
business customers.

We have (at the moment) two of what I suppose are here referred
to as Repositories -- entry points into the graph of the domain
objects; a ProjectRegistry that has a single static method
getProjectbyId(String id), and a UserRegistry that likewise has
a single static method, getUserByUsername(String username).
If you want to find what Locations belong to a project, you
first get a reference to the relevant Project by giving the id
to the registry, then you use getters on the Project object and
on futher domain objects returned thereby, i.e. you navigate
the object graph to where you want to go. At no point is there
any need for the client to know anything of the persistence
mechanism.

The persistence layer has two internal layers. The upper layer
provides the persistence interface to the domain layer and has
classes like ProjectDA, UserDA, DocumentDA, ProgressDA, etc --
these don't map 1-to-1 to domain objects, but each manage the
persistence for a coherent subset of domain objects; e.g.
the ProgressDA handles a collection of domain objects that
relate to tracking a projec's progress; DocumentDA handles
the myriad of documentation-related domain objects, etc.
The lower internal layer consists of one (1) DA class per
type of persistent storage -- we've got JDBC, LDAP, a
commercial documentation system with a proprietary Java client
interface, and some other stuff hidden behind a HTTP interface.
The upper layer calls on the lower layer, which does the
actual dirty work toward the physical data stores.

All new code and all code that we modify or touch is developed
and refactored to this new architecture. This also means that,
as we're still at an early stage, we're still also making
some basic decisions about how our domain persistence will work,
which brings us to the first question.

The current thinking is that the domain objects' getters know
whether the requested attribute has been loaded and if not,
fetch it from the persistence layer. Setters always write
the set attribute straight to persistent storage. This is the
"simplest possible solution that could work", as per XP
philosophy.

There are, however, some obvious problems that this raises.

(i) Ripple-loading. Our thinking is that we should be able to
use knowledge of the business domain and of users' UI workflow
to optimize loading in such a way as to effectively avoid ripple
loading, at least in most cases (critical path situations).
That is, we could put this logic in the persistence layer and
have it, in certain specific situations, load more than actually
requested and cache that within itself. Though it seems to me
that this could be a bit naive, and if it doesn't turn out to
work, then how do we manage ripple loading when we hit it? I
think Fowler recommends something like "deal with it with a
specific localized optimization when and if it actually happens".

(ii) Writing changes to persistent storage. The simplistic
approach envisioned is probably way too simple and will just
end up in a monstrous number of database &c. updates. I think
that we should take Fowler