DDD is back in Paris with a brand new Meetup group!

The first DDD Open Forum of the brand new Paris DDD meetup was last night, hosted by Arolla, and it was good to meet again after a long time with twenty-some Paris DDD aficionados!

@tjaskula, the organizer of this new group, opened the evening with a welcome introduction. He also gave many suggestions of areas for discussion and debate.

A quick survey revealed that one third of the participants were new to Domain-Driven Design, while another third was on the other hand rather comfortable with it. This correlated with a rather senior audience, with only one attendee with less than 5 years experience and many 10+ years developers, including 22 years and 30 years experience developers, and still coding! If you work in Paris, I guess you know them already…

It was an open space session, so we first proposed a lot of topics for discussion with post-its on the wall: how to sell or convince about DDD, introduction on concepts, synchronizing between contexts…

We all decided to start with a walk through of the fundamentals of DDD: Bounded Contexts, Ubiquitous Language, Code as Model… It was great to have this two-way knowledge transfer between seniors and juniors, in an interactive fashion and with lot of questions, including some rather challenging and skeptical ones! There was also some UML bashing of course.

We concluded by eating Galettes des Rois, together with cider and beer, and a lot of fun. Thanks everyone for your questions and contributions, and see you soon on next meetup!

The many proposals for discussion

Read More

Principles for using annotations

Deciding where and how to place the annotations is not innocent. The last thing we want is to create extra maintenance effort because of the annotations. In other words, we want annotations that are stable, or that change for the same reasons and at the same time than the elements they annotate. This article suggests some good practices on how to design annotations.

Annotations are location-based

annotations
A special kind of wall annotation

Language annotations or even good-old xDoclet tags enable to augment program elements with additional semantics, which can be used to configure tools, frameworks or containers.

Configuration is now increasingly done through annotations spread all over the project elements. The key advantage is that the location of the annotation directly references the program element (interface, class etc.), as opposed to configuration files that must reference program elements using awkward and error-prone qualified names: “com.mycompany.somepackage.MyClass”, that are also fragile to refactoring.

For example, we can annotate an entity to declare how it must be persisted, we can annotate a class to declare how it must be instantiated by a Dependency Injection framework, and we can annotate test methods to declare their purpose.

If not placed and thought carefully, annotations can make your code harder to maintain. This happens when annotations are placed at the “wrong” place, or when they introduce undesirable coupling, as we will see.

Dependencies still matter

The question of coupling between elements of the code base is also relevant for annotations. That the coupling is done via an annotation rather than plain code does not make it more acceptable.

We want to group together things that change together. As a consequence, put your annotations on the elements that change with the annotations.

In particular, when the annotation is used to declare a dependency:

Only annotate the dependent element, not the element depended on

If you use Dependency Injection and you want the class MyServiceImpl to be injected everywhere the interface MyService is used, then Guice offers the annotation @ImplementedBy:

@ImplementedBy(MyServiceImpl.class)
interface MyService {... }

This annotation is a direct violation of the advice above, since it makes a pure abstraction (an interface) aware of an implementation, whereas the regular dependency should be the other way round: only the implementation must depend on the interface.

I must however acknowledge that the annotation @ImplementedBy is quite convenient for unit tests anyway, to declare a default implementation for the interface. And it was done just for that, as described in the Guice documentation along with a warning:

Use @ImplementedBy carefully; it adds a compile-time dependency from the interface to its implementation.

Favor intrinsic annotations

annotation2
Another annotation on the wall in Paris

If you want to declare that a service is stateless, you cannot get it wrong: just put the annotation @Stateless on its interface. This is straightforward because being stateless is a truly intrinsic property. It also makes perfect sense to annotate a method argument with the @Nullable annotation, as the capability to expect null or not is really intrinsic to the method.

On the other hand, a service interface does not really care about how it is called. It can be called by another object (local call) or remotely, through some remote proxy. The object is not intrinsically local or remote in itself.

The point is that the decision to consume the service locally or remotely does not belong to the service, in itself, but depends on each client. It actually belongs to each use-case considered.

Said another way, specifying @Remotable or @Local directly on the service would require the developer of the service to guess how it will be used!

Intrinsic properties really belong to the element and therefore are stable, as opposed to use-case-specific properties that vary with the particular case of use. Hence, if we want stable annotations:

Only annotate an element about its intrinsic properties, and avoid annotating about use-case-specific properties.

Annotations as pointcuts

Let’s consider an example of  an accounting service in a bank. Only selected categories of staff can access this service. We can use annotations to declare its security configuration:

@RolesAllowed({"auditor", "bankmanager", "admin"})

The problem with that approach is that it couples the service directly to the user roles defined elsewhere; as a consequence, if we want to change the user roles (we now need to add the user role “externalauditor”), we will have to review every security annotation and change them. On the other hand, if we want to change the access policy (which happen every time a new senior management comes into place), we will also have to change annotations all over the code. How can we improve that?

We can improve the situation by going back to the business analysis on the topic and separate what’s intrinsic and what’s not. In other words, we want to find out how did a BA came up with the security roles for the service.

Rather than specifying the need for security in terms of allowed user roles, we can instead declare the facts: the service is “sensitive” and is about “accounting”:

@Domain(Accounting)
@Confidentiality(Sensitive)
And now a beautiful car annotation
And now a beautiful car annotation

Then we can define expressions that use the declared annotations (which are now stable because they are intrinsic) to select elements (here services) and associate them to allowed user roles. These rules should be defined outside of the elements they apply to, typically in configuration files.

Thanks to the annotations that already define half of the security knowledge, expressing the rules becomes much simpler that doing it method by method. So next time the senior management changes and decides that from now on, “every service that is both Confidentiality(Sensitive) and Domain(Accounting) is only allowed to corporate-officer roles”, you just have to update a few rules expressed in terms of domain and confidentiality level, rather than by listing many method.

The mindset is very similar to AOP where we first define pointcuts, and then attach advices to them. Here we use annotations as an alternative way to declare the pointcuts.

Conclusion

Annotations are very efficient to declare properties about program elements directly on the elements. They are robust versus refactoring and are easier to use than specifying long qualified names in XML files.

To get the best of annotations, we still need to consider the coupling they can introduce, in particular with respect to dependencies. If a class should not know about another, its annotations should not either.

Annotations are much more stable (less likely to change) when they only relate to intrinsic properties of the elements they are located on. When we need to configure cross-cutting concerns (security, transactions etc.) annotations can be used to declare the half of the knowledge that is really intrinsic to the elements, in the same spirit than pointcuts in AOP.

All that leads to the acknowledgement that even though annotations can be of huge value, in practice there is still a case for configuration files to complement them. In this approach, annotations on elements declare what belongs to the elements, while each use-case-specific configuration file makes use of the annotations and as a result is much simpler.

Read More

How I became enthusiastic about patterns

In my very first job, I  was doing R&D, working on a map-matching algorithm. The goal of this algorithm was to pinpoint a moving car on a vector map, based on the data from various sensors, including a GPS, an electronic compass and the car odometer. Such algorithm was essential for the business of the company, and there was very little literature on the subject.

The R&D challenge

At school I had been taught some C programming, so I started doing the algorithm in plain C code. One special case after another, the code began to grow until it became quite complicated. I had a specially equipped car with a computer and all the gear in it to do real testing on the roads around the office, from the highway to forest road, city streets or even car parks, and this was fun! But situation after situation, I had to make the code more and more complicated. At some point, it became obvious to me that the mode of implementation (plain C code) had become the main obstacle for improving the algorithm. It was becoming increasingly difficult to grow the sophistication in a mess of structured code.

My early mentors

Yes savoir-faire can be found in books (but not only)
Yes savoir-faire can be found in books (but not only)

At the same time I was willing to progress, so I was getting closer to the few very experienced colleagues. Our company was a startup in 2000, and there were many more junior developers than senior ones. At first, I thought UML could help me (it did not indeed) so I started asking questions about UML. When I became more comfortable with UML, a senior colleague told me I should now have a look at design patterns, starting with the Composite. So I took the GoF book on my desk and began to look at it as a reference to get design ideas during the day. I also borrowed the pattern pattern book from Mark Grand and read it in the train.

And then it has been “Wow patterns are a great way of transfering knowledge!”. I remember reading the pattern “Cache” in the book. It was not in itself a very innovative design idea, but I understood that the pattern format was ideal to document just any idea. I hate long explanations in long books, and the pattern format, which tends to be short and structured, is perfect for quick scanning whenever you’re looking for ideas. Even when I didn’t find a pattern for my case, I found it stimulating to read other people ideas.

Enthusiasm and success as a result

I started to apply the State and the Strategy patterns into the map-matching algorithm and this made it much, much simpler. It actually made it so much simpler that we were now able (the team was growing at that time) to go an order of magnitude further in sophistication, while being perfectly in control of the code. The introduction of two simple design patterns had suddenly given a really big advantage to a piece of code essential to the company! This is how I became enthusiastic about patterns.

The reality

I am also enthusiastic about deserts.
I am also enthusiastic about deserts.

What actually happened is that reading and starting to play with patterns just taught me object-oriented programming. Patterns acted like examples of good design, until the underlying principles became natural. Later I discovered the SOLID principles of Robert C. Martin, and recognized the principles behind many design patterns. In my next job experiences I took the habit to look for patterns for whatever problem I was encountering, and to my surprise, I found out that most common problems were already being taken care of in the form of analysis patterns or other kinds of patterns! To give the most obvious example, Martin Fowler “Things that change with time” is really a must-read, which you can apply easily to solve your problem.

Conclusion

This is how I became enthusiastic about pattern, not just design patterns but every kind of patterns, from analysis patterns to domain driven design patterns, enterprise integration patterns, PLoP patterns and many patterns from various authors. I know my enthusiasm is a bit exaggerated, a bit like the souvenir of a first love that cannot be much objective. Fortunately I quickly learnt when not to use patterns, to keep things as possible as they can be, and to do unit testing. By the way the benefits of unit tests also struck me when I started with them, but not as much as patterns, there can be only one first time, and my first first time was with patterns, not unit tests!

Read More

Systematic combination of subpatterns generates design patterns

Composite patterns, such a the Bureaucracy pattern, are patterns built by the composition of other “smaller” patterns. However even usual design patterns can be considered composite patterns made of smaller subpatterns.

The goal is therefore to find out which are the main subpatterns that enable to reconstruct as many design patterns as possible.

The subpatterns

From an early analysis I believe that four subpatterns form the foundation of many GoF design patterns:

  1. Type Hierarchy and its variants (interface, abstract class, non final class…)
  2. Type Set, a set of types with no particular relationship between them
  3. Delegation and its variants (one-to-one, one-to-many)
  4. Allocation (creation of instances) and its variants

The experiment

To validate this proposal, I have coded a small experiment to generate every combination of two subpatterns out of (1) and (2) plus one relationship between them out of (3) and (4), or every combination of one subpattern out of (1) and (2) plus a relationship of (3) and (4) to either itself, or in the case of (1), between a super type and a subtype or the other way round.

For each generated combination I have automatically generated its corresponding class diagram, and grouped them together into one SVG document, converted into PDF using Inkscape. The resulting picture is available here, and you can see a partial preview below:

combinations_thumb

Results

The interesting finding is that this simple combinatorial method re-discovers 11 well-known design patterns (in the reading order of the picture):

  • Template Method (delegation to self, expecting a subtype to provide the service)
  • Composite (Considering a one-to-many delegation) or Decorator / Proxy (one-to-one delegation)
  • Prototype (creates instance of this type)
  • Bridge (one Abstraction hierarchy delegates to another Implementor hierarchy)
  • Abstract Factory (one Factory hierarchy creates instances of another (or several others) Products hierarchy)
  • … skip 4 next …
  • Strategy or State or Builder (some client delegates to some hierarchy)

Note that there are several cases where we could see the Factory Method, Adapter or Facade patterns. This would lead to some 13 design patterns out of the 23 in the GoF book.

Discussion

It is obvious anyway that the four subpatterns used here are not enough to rebuild the full set of the 23 design patterns. The Command pattern is unique in its use of a separate invoker, and the Chain of Responsibility is clearly about how handlers self-assign a given task without any centralized management. In these two examples however, the unique characteristics is not in their structure but rather in their dynamics. Sequence diagrams would be much more relevant to convey that kind of information.

The full test runs in one go in a fraction of second. The source code is available here: patternitygraphic_src, the code of the experiment is in CombinationTest.

EDIT: the plain SVG file that was generated is also available here for download:

Read More

New: Java API for UML diagrams

As part of the Patternity effort, I spent some time creating a simple Java API to generate UML diagrams programmatically from Java, in SVG format.

This small API called for now Patternity Graphic is working and available here: patternitygraphic_src as a source Zip (alpha release of course).

It can render small class diagrams with hierarchic, flow and radial layouts, and arbitrary sequence diagrams with unlimited nesting of method calls and embedded comments. My focus was to support the subset of diagram elements and capabilities required to display patterns occurrences.

Here is a sample sequence diagram with a nice call stack:

sequence1

And here is a class diagram for a simple dummy hierarchy:

hierarchy

Boxes and links have various styles, defined in a template.svg template file, here is a random display of the boxes styles:

boxstyles

Apart from unit tests there is no documentation. If you are interested to reuse that please contact me for help. The Zip was exported in Eclipse with the project Export… function, and the project requires Commons Collections. Svg diagrams can be converted into images using Batik.

Other projects you might consider to generate UML diagrams: UMLGraph, modsl, umlspeed, jsigner, MetaUML, and of course GraphViz.

Any feedback appreciated!

Read More

Documenting the design of software using patterns?

I have experimented an approach that considers every design pattern as the recursive composition of smaller patterns. This led to a prototype tool to illustrate its benefits by generating design-level documentation of annotated source code.

Eat your own dog food

The source code of this tool itself was used as the code base to apply the tool on, in order to generate its own design documentation. The Java Doclet API was used to retrieve the Javadoc tags in the class Javadoc comments, e.g. :

@pattern KnowledgeLevel OperationLevel=MyOtherClass some free text comment…

The pattern catalogue

A subset of design patterns and some other patterns such as the Knowledge Level (Fowler) has been defined in small definitions files then loaded at tool startup. We call this set of patterns definitions the pattern catalogue.

Here is an example of the definition file for the Builder pattern, notice the declaration of the member roles at the bottom, that also declares the expected pattern kind:

Encapsulates the creation of a complex object from a source object.
@extends DesignPattern
@category DesignPattern
@category Creational
@author GoF
@book [GoF 95]  E. Gamma, R. Helm. R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley 1995.
@style ObjectOrientedProgramming
@reference [http://en.wikipedia.org/wiki/Builder_pattern ]
@defaultrole Builder
@role Director=TypeHierarchy
@role Builder=TypeHierarchy
@role Product=TypeSet
@role Factory=AbstractFactory

Parsing the annotations to build the pattern occurrences graph

Every time a Javadoc tag “@pattern Pattern” is met in the Javadoc part of a class, an occurrence of the corresponding pattern is created in memory, linking to the corresponding class in the Java programming language metamodel.

Whenever such pattern uses a Type Hierarchy (this is declared in the definition of the pattern), then the tool will look for every subtype of the declared (super) type in order to complete the pattern occurrence as well as possible.

Even patterns that are usually considered as atomic patterns can be explained as the composition of smaller “elemental” patterns. For instance, A Builder [GoF] usually defines an AbstractBuilder role and a ConcreteBuilder role that implements it. We can express this part of the Builder pattern as a TypeHierarchy elemental pattern in the role Builder; the supertype of the TypeHierarchy therefore represents the AbstractBuilder role, while every other concrete type of this hierarchy represents ConcreteBuilder roles.

By analogy with the members of a class, we call members patterns and member occurrences the patterns and occurrences (respectively) a pattern or occurrence is composed of. The tool totally ignore fields and methods, it only considers types (interfaces and classes).

The benefits of this approach are to reuse a bigger part of the pattern definitions through composition and to help address the variant problem.

Once the pattern occurrences graph is completely built, it is rendered into an UML class diagram using Graphviz dot. Here is an example of overview for the full project:

coremodulediagram_final
Example of overview diagram for the Core module of the prototype source code

In the above diagram we can see how the patterns naturally sit on top of the several type hierarchies while connecting them together. Also the relationships between patterns become very obvious, such as the Interpreter that references the Builder and the Composite sub-patterns. This suggests that such diagrams are very valuable to document the design of a code base.

Topological sort

The approach of representing every pattern as the composition of other patterns yields to directed graphes, at the pattern level (the “meta” level) and at the pattern occurrences level. One consequence is that we can apply a topological ordering on such graphs. In the current prototype we need to do a topological sort on the patterns definitions so that we can know in advance in what order to process pattern occurrences declarations: the patterns used must be built before the patterns using them.

The pattern occurrences graph can also be navigated using the usual Depth-First search or Breadth-First search. DFS is convenient to generate the Overview diagrams such as the diagram shown above, whereas BFS is convenient to drill down the design from the top to the bottom, one level at a time. This can therefore generate the following table of contents:

Patterns graph navigation index
Patterns graph navigation index

Natural language generation alternative

As an alternative to visual diagrams, a tool can also generate natural language text to communicate the exact same information. I have tried this approach using simple text templates in each pattern definitions. Each pattern therefore defines its own design description, but using variable names to be filled later with the data from the pattern occurrences graph.

The biggest work in this approach is to deal with the plural, the enumeration of collections (comma between each item but the last etc.), truncating collections that are too long, and the complete omission of a section if a pattern member is totally missing.

Here is a sample of text generation for the prototype tool itself. Notice how the structure is rigid for each pattern occurrence: Pattern name, short description, then template text evaluated with the actual types names, followed by the comment that was put after the @pattern declaration:

Design of the module Core
This module is essentially made of 9 pattern occurences: Strategy, Visitor, KnowledgeLevel, Interpreter, Builder and AbstractFactory.

Visitor
The visitor design pattern is a way of separating an algorithm from an object structure. A practical result of this separation is the ability to add new operations to existing object structures without modifying those structures
This module defines a Visitor OccurrenceVisitor. comment

Strategy
A Strategy object encapsulates an algorithms that can be selected on-the-fly at runtime.
This module defines the interface of a Strategy: Plugin. It enables plugins to manipulate the Patternity metamodel.

AbstractFactory
Provides one or more method(s) that encapsulate(s) the creation of object.
The type OccurrenceFactory defines the interface of an AbstractFactory that creates instances of Occurrence and TypeOccurrence.

Interpreter
Define a representation for a language along with an interpreter that uses the representation to interpret sentences in the language.
The type(s) Occurrence and TypeOccurrence define(s) an object representation of the considered language. The considered language describes a graph of patterns occurrences. It uses the Visitor Visitor to add new operations to the object representation without modifying this structure.

Strategy
A Strategy object encapsulates an algorithms that can be selected on-the-fly at runtime.
This module defines the interface of a Strategy: Loader. Encapsulates how to load every pattern definition into the pattern.repository.

[...]

KnowledgeLevel
A Knowledge Level is a group of objects that describes how another group of objects should behave.
This module defines a KnowledgeLevel (also known as metamodel). It is made of two levels of objects: objects in the knowledge level, i.e. the instances of Pattern define how objects in the operational level can behave, i.e. instances of Occurrence. Typically each type in the operation level maintains a reference to its corresponding type in the knowledge level in order to know how to behave.

Conclusion

Later, at work, when asked to produce a design documentation targeted at other developers, I have experimented doing a similar effort manually, using nothing but patterns. I am still impatient to receive some feedback from the people that will read the resulting documentation in order to validate or infirm this approach.

Read More

UML is a too low level language

I recently taught design pattern to 20 of my co-workers in a kind of training session, here are some thoughts I had while preparing this presentation…

In OO you usually end up with many classes, especially when you correctly use design patterns; each class is also generally very simple and rather short. Thus classes are now in OO the equivalent of the “if-then-else-for” instructions in structured programming: low-level language elements.

Unfortunatly, UML is essentially about class-level static description (class diagram), dynamic objects interactions (sequence, object diagram). Even more OO-independent part of UML such as state diagram is still at the class level, as typical implementation using the State pattern (GoF) implies using a class for each state.

What is the higher level above the class-level ? In my opinion, it is the (design) pattern level, for at least three reasons:
– granularity: a design pattern usually deals with more than one class, so it is a natural way to talk about several classes as only one entity;
– patterns language: patterns may be solutions to solve recurrent problems, sure, but they first of all define a common language to talk about the entire solutions in a standard, shared way;
– patterns intent: patterns are always motivated by an explicit intent; this intent is the real high-level need of the developer, whereas the solution that uses classes, methods overriding, methods delegation, methods callbacks… is nothing but the best known way to meet the intent using OO language.

UML support for patterns is quite minimal (dash lined and named bubbles with links to every participant in the pattern), and adds nearly no value compared to simple UML comments. If you know the patterns, and you do if you’re reading this to this line, I’d better tell you “this package contains adapters from the third party financial instruments tree to our custom financial instruments tree” than show you a diagram with 35 classes in a package and 35 classes in another package, 35 interfaces and 35 bubbles showing the pattern Adapter is applied 35 times respectively for each class, class and interface…

In the business design (some say analysis), the same is true. It’s shorter and more accurate to say the model follows the Contract pattern (Fowler), than to show the UML diagram. But on the other side, there is no formal language, visual or textual, other than plain English to describe patterns at their full level. At this level, packages would would be first-class entities, not just boxes that contains classes. Code generation could also go one step further, since if the machine knows about our intent, it could also provide the solution to our intent, maybe by implementing the correct pattern automatically, using the best implementation tradeoff automatically, but now I’m dreaming…

Initially published on Jroller on Tuesday March 15, 2005

Read More