Patterns in general, not just design patterns

Over time, patterns have appeared on many different topics, not all related to programming. Here is a list of patterns and pointers to other lists of patterns, to illustrate two things:

  1. Knowledge and experience in general can be packaged into patterns, often using the pattern form. Patterns are convenient for reuse, in any domain.
  2. There are already plenty of patterns, and they cover a really wide range of situations. Given the number of patterns available today, whatever your problem, you will likely find helpful patterns for it.

This should encourage you to search for existing patterns whenever you need additional insight, or just an already documented reference for what you’ve just done (documenting the design of software using patterns).

The list in this post will grow, but without the ambition of listing every possible pattern.

Software Design

Here are two essential lists of books on patterns:

Other books and/or websites focus on patterns for software development, each with a specialized perspective:

Please notice that while many patterns are to be applied directly to software design or source code, there are also many patterns about the process of building software.

This list could not be complete without referring to the big Pattern Almanach by Linda Rising, available on her website here in PDF or in a web version here.

The Hillside.net Patterns catalog and the many papers published after each PLoP also provide many patterns that may well cover your problem.

User interfaces patterns

Testing patterns

Misc

Probably any domain can benefit from using patterns to represent chunks of knowledge and experience. Here are some examples quite foreign to software developement:

Conclusion

Though this listing is far from extensive (there are many independently published articles on patterns everywhere), it shows that the pattern community has been quite active to mine as many patterns it could, and this knowledge has been carefully documented in the pattern form, ready for reuse in your next projects.

Of course you don’t need to know them all, not even read them all in advance. However it does help to be aware that for most problems, there probably already exists a few patterns that can prove helpful, and they are only an Internet search away from your needs!

Read More

Toward smarter dependency constraints (patterns to the rescue)

Low coupling between objects is a key principle to help you win the battle against software entropy. Making sure your dependencies are under control matters. Several tools can enforce dependencies restrictions, such as JDepend. However in a real project with many classes, packages and modules, the real issue is how to decide and configure the allowed and forbidden dependencies. Per class? Per package? Per Module? Based on gut feeling? Is there a theory for that?

Of course, in a layered architecture, the layers specify the dependencies. This is not bad, but I am sure we can do better.

Smarter dependencies

To go further, I suggest expanding our vocabulary of concepts. In OO languages such as Java, everything is a class (or an interface), grouped into packages. Such classification is not really helpful. Fortunately, several books provide ready to use vocabularies in the form of patterns languages (not only design patterns, but patterns in general). Some of these patterns are foundations on which rules to manage dependencies can be proposed.

Disclaimer: the dependencies rules suggested below are hypothesises to be debated and verified against a corpus of actual projects, I would be happy to be given counter-examples and counter arguments.

The child really depend upon the mother
The child really depend upon the mother

Domain Driven Design

The book Domain Driven Design by Eric Evans defines a rich vocabulary of concepts used in every application, and we can leverage that vocabulary to propose some dependencies principles between them:

  • ValueObject never depends upon Entity nor Services
  • Entities should not depend upon Services (maybe not a hard rule)
  • Generic SubDomain should not depend upon Core Domain
  • Core Domain should not depend upon Cohesive Mechanism (the “What” should not depend upon the “How”)
  • Domain Layer should not depend on any infrastructure code
  • Abstract Core module never depends on its specialized or implementation modules

Analysis Patterns

The book Analysis Patterns by Martin Fowler also provides patterns as a richer vocabulary, from which we could propose:

  • Elements from a Knowledge Level should not depend upon elements from the corresponding Operation Level

I did not find that rule written in the book but every example appears to support it. Considering that classes and subclasses in usual OOP are a special case of Knowledge Level built-into the language, this would lead to:

  • Abstraction never depends upon their Implementations

which is similar to the second part of the Dependency inversion principle by Robert C. Martin:

Abstractions should not depend upon details. Details should depend upon abstractions.

Since many analysis patterns in the Analysis Patterns book involve the Knowledge Level pattern, this single dependency rule already applies to many analysis patterns: Party Type Generalizations, Measurement, Observation, Protocol, Associated Observation, Measurement Protocol etc. The pattern Quantity can be seen as a specialized ValueObject (see Domain Driven Design above) hence should also not depend on any Entity nor Service.

Design Patterns

The book Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma et al. presents the classic design patterns. These patterns define participants which are named.  In the pattern participant ignorance principle I discussed the concepts of ignorant Vs. dedicated participants within a pattern, and their consequences for dependencies:

  • Ignorant pattern participants should never depend on dedicated participants
  • Pattern participant never depend on the “Client” participant
  • For each ConcreteX participant, the corresponding abstract X never depends on it (Abstraction never depends upon their Implementations)

In practice, this means:

  • In the Adapter pattern, the Adaptee should not depend upon the Adapter, and the Target should depend upon nothing
  • In the Facade pattern, the sub systems should not depend upon the Facade
  • In the Iterator pattern, the Aggregate should not depend upon the Iterator; however every Java collection is a counter example as they contain their own ConcreteIterator.
  • In creational patterns (Abstract Factory, Prototype, Builder etc.), the Product and ConcreteProduct should not depend on the dedicated participant that does the allocation (the Factory etc.)
  • And so on for other patterns, some of which being already discussed in the pattern participant ignorance principle.

In short, if we look at the design patterns as a set of types with inheritance/implementation, invocation/delegation and creation relationships between them, the dependencies should not flow in the reverse direction of the relationships; in other words, using UML arrows, the dependencies should only be allowed in the direction of the arrows.

Addiction to sugar is a kind of dependency
Addiction to sugar is a kind of dependency

Patterns of Enterprise Architecture

In the book Patterns of Enterprise Application Architecture by Martin Fowler, the Separated Interface Pattern proposes a way to manage dependencies by defining the interface between packages in a separate package from its implementation. This is similar to the Dependency inversion principle, as discussed here, which states:

A. High-level modules should not depend upon low-level modules. Both should depend upon abstractions.

By the way this is also very similar to the recommendation in Domain Driven Design:

Abstract Core module never depends on its specialized or implementation modules.

Finally, in the spirit of UML stereotypes that we sometimes put on packages to express their intent:

  • Utils never depends on anything but other Utils

What for?

If we manage to make every use of the above pattern explicit in the source code, for instance using Java annotations or simply Javadoc tags, then it becomes possible for a tool to deduce dependencies constraints and automatically enforce them.

Imagine, just add @pattern ValueObject in your Javadoc comment, and voila! A tool is now able to deduce that if you happen to import anything but basic java.* you must be warned.

Of course the fine tuning of the default behavior can take some time (do we accept that ValueObjects may depend upon low level utils like StringUtils? Probably yes), but the result will at least be stable regardless of the refactorings.

Given the existing variety of patterns over there, I am confident that just any class or interface within a project can be declared as being a participant in at least one pattern, and have therefore its dependency constraints deduced at the same time.

Read More

Essential patterns books for today object-oriented developers

The more patterns developers know, the most efficient they become within a team: it only takes one or two words (the pattern name) to communicate a design decision or proposal, instead of 10 mn of explanations. Communication also gets much more accurate and to-the-point (or less fuzzy).

Because patterns often form a pattern language, not only they offer a standard vocabulary but they also help structuring the mind by their relationships: patterns can be related as exclusive alternatives, or rather often-go-together, which is useful.

Patterns are not always supposed to be “tricks” to learn, or extra complication to introduce to a design; indeed even obvious and uninteresting pattern are worth reading in books, just for the social advantage of being able to reference them. Going further, sometime you can just go to the book to find out which is the pattern that you just did without knowing its name…

Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley Professional Computing Series)
1.  Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley Professional Computing Series) by Erich Gamma et al.
The most essential book, a definitive must-have. Every experienced software developer must know these 23 design patterns.

Analysis Patterns: Reusable Object Models (Addison-Wesley Object Technology Series)
2.  Analysis Patterns: Reusable Object Models (Addison-Wesley Object Technology Series) by Martin Fowler
The essential complement to the GoF, to extend the benefits of patterns from design to analysis (closer to the problem to solve in the domain). Each pattern will not necessarily teach you a new solution, but will always give you have a standard name for each solution, which is already worth the book.

Domain-Driven Design: Tackling Complexity in the Heart of Software
3.  Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans
A book that can change the way you deal with software. Considering the domain as the main driver of the design is so powerful an approach! Works best together with the analysis patterns.

Patterns of Enterprise Application Architecture (Addison-Wesley Signature Series)
4.  Patterns of Enterprise Application Architecture (Addison-Wesley Signature Series) by Martin Fowler
Other analysis patterns to solve many technical problems, useful complement to Analysis Patterns.

Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions (Addison-Wesley Signature Series)
5.  Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions (Addison-Wesley Signature Series) by Gregor Hohpe
Very complete pattern language (both test and visual vocabulary) about messaging, will improve the communication within a team a lot, both in efficiency and in accuracy.

Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects
6.  Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects by Douglas Schmidt
Important book, more specialized toward optimized low-level problems (web servers…). often quoted. The other volumes are less relevant for today software development on modern languages or virtual machines.

Patterns in Java: A Catalog of Reusable Design Patterns Illustrated with UML, 2nd Edition, Volume 1
7.  Patterns in Java: A Catalog of Reusable Design Patterns Illustrated with UML, 2nd Edition, Volume 1 by Mark Grand
The book I discovered design patterns in, quite affordable, but some colleagues do not like it that much.

Read More

Sharing a common architecture or vision within a team

What do you think about when you hear the word “architecture” about software? Fowler defines it: whoNeedsArchitect “In most successful software projects, the expert developers working on that project have a shared understanding of the design system design. This shared understanding is called ‘architecture.'”.

However for most people the word “architecture” comes full of middleware connotations, hence I consider this word now inappropriate. Let us consider here the concept of a shared “vision” of the design.

First it has to be a shared vision of the domain, for instance the interest rate swaps trading business. Words must have only one meaning, known by everyone, from IT, marketing or support. A shared glossary is key for that.

Then the IT team must share a consistent set of practices: consistent between developers, and consistent between practices themselves. This also means that the set of practices de-facto excludes other practice that conflict with it, and this has to be clearly acknowledged.

Even though any set of practices can be considered arbitrary, everybody in a team must adhere to it, preferably wholeheartedly. Of course the more accepted and documented the practices, the easier it is for everybody to adhere to them.

For example, a team can have a set of practices that includes Agile programming (be prepared for change, no big design upfront), automated testing (unit tests, stress tests), domain-driven analysis as devised by Eric Evans, the use of proven solutions when they exist (hence design and analysis patterns from GoF, POSA, Fowler, Doug Lea…), a bit of functional-programming style (favour immutability and side-effect-free functions, pipe and filters…), and Eclipse default coding conventions.

Some practices that also belong to the shared set of practices might not be directly documented, though they derived from articles, papers, blogs and some personal opinions. Examples of such custom practices that I like a lot include “Align the code with the business” and “think options not solutions”.

“Align the code with the business”, focuses at the smallest level, i.e. at the class and method level. Just every bit of the application must make sense with the domain considered. A class must reflect some relevant concept, concrete or more abstract, of the domain, not of the requirement. The rationale is that while requirements change all the time, the domain concepts are very stable; they are not true or false, they just exist!

“Think options not solutions” refers to the very basics of Agile and XP: “be prepared for change”. It also acknowledges the fact that most of the code in a well done software project provides standalone, encapsulated and reusable building blocks that are then assembled together using a smaller amount of glue code, that can even be reduced by using a dependency-injection framework such as Spring. Typically, a concept from the domain, such as an interpolation algorithm in the finance domain, translates into an Interpolation interface along with several alternate implementations. Most of the time, only one implementation out of the ten possible is really used at a time. However the option to use the other nine alternatives is there. This also means that we can assemble a release of the software using some subset of its building blocks, and assemble another release using another subset, at the same time. One codebase, several possible applications: this is the goal.

In addition to every explicit practices, the existing code base implicitly shows practices that can either be reused just for the sake of consistency, or replaced as widely as possible when they conflict. Prior art, when of good-enough quality, can form a skeleton for a shared vision of how to create software. Again this is arbitrary, but consistent, and consistent design is valuable in itself, and often more than the design decisions themselves.

Of course there is a point where more shared practices become counter-productive, for instance when there are too many of them (more efforts to learn by newcomers, feeling of loosing freedom, lack of flexibility). However when they are standard enough, or becoming standard enough, developer should consider that acquiring these practices a long term, beneficial, investment.

Initially published on Jroller on Monday January 21, 2008

Read More

My definition of abstraction

One thing human are particularly pride of is our ability to use abstraction to solve problems. We in software development all know that “another level of abstraction can solve my problem”. But what do you call abstraction ?

Abstract is the opposite of concrete. Concrete things are simply things, the kind of things a dog can see, bite and eat: me, the bone on the floor, the bee it’s trying to catch on the fly… In my job, concrete things are the formula to compute the price of a fixed income bond, the interest rate risk in a portfolio or the report the manager wants the tool to generate.

When I first work on a thing, this thing is concrete to me. The first bond I am computing the price for is just a simple concrete bond. But the second, different bond is a revelation: while it is different to the first bond, it also has lots in common ! wow ! I can notice it, and especially I’d like to save work from this, as I am a lazy developer. So I gonna use abstraction…

Concretely, I would define abstraction as the mental action of finding the commonalities and the differences between two or more things. The common part of every thing is what I will call the abstract thing, and I will then call each thing with its differences a concrete thing.

So my (very practical) definition of abstraction has to be explicit to be of value to me, so that I can reuse the common part of things. The most obvious example of abstraction is the very notion of class against instances of a class, aka objects: the class represents the common part (the structure) of a group of objects, while each object has its own values that make it different to its mates. What is common between every bond I will be given is that they all share a group of parameters and calculation conventions, thus I will create my pricing software for this abstract bond in order to be able to price every bond by just describing its differences against the common bond.

Young children early develop their abstract skills by playing with things; They soon see the difference between one thing and two things as the notion of numbers, the difference between a red shape and a green same shape as the notion of colour.

As said before, abstraction is very much about classification: given the red square and the green square, we can see they both have a square shape, so we can classify them in the shape = square category. Given another green circle, we can also classify the green square and the green circle in the colour = green category. This way, we have identified two abstract notions, shape and colour, from very concrete things.

As I am -more and more- a lazy developer, my next question is: can my computer abstract things for me ? After all abstraction helps in handling what’s common in things in order to handle this common part only once, and I have identified the very notion of abstraction itself as an abstract notion, maybe in order to reuse it ? (by the way what I am doing at this point in this writing also follows the abstract notion of recursion, talk about abstraction, then talk about the abstraction of abstraction). Some clues: in common spreadsheets, when we enter a number sequence, say 1, 2, 3, the spreadsheet recognizes the arithmetic sequence so that it can guess its next values. I’m tempted to claim it is an abstraction ability already… Not sure in fact. Obviously, it is in AI field (artificial intelligence) that neuronal networks have natural classification abilities, so the answer is yes, software can abstract concepts.

Initially published on Jroller on Monday April 11, 2005

Read More