Patterns express intents

Patterns represent a couple (intent, solution); sometime they refer to a solution, more often they essentially represents an intent, independently of its solution.

Sometimes the solution part of patterns includes a trick or a workaround to overcome the limits of a language, but patterns cannot be reduced to that trick. Indeed, a very important role of patterns (not only design patterns but patterns in general ) is that they represent stereotypes of intents.

A matter of intent

Therefore, it does not really matter if the Strategy pattern can be expressed using a Java interface, a C++ functor or a first-class function: it remains a Strategy because this is just what we want: “Strategy lets the algorithm vary independently from clients that use it“.

Is the intent of this book holder clear?
Is the intent of this book holder clear?

Another similar pattern is the Command pattern, which intent is:” Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.” Here the intent talks about ‘object’ because it was written for an object-oriented context, however it can easily be made generic if you think ‘handle on function’ (or closure etc.) instead of object. Again, even if first class functions such as delegates in C# can achieve this goal, they do not replace the need to declare the precise intent: “you want to parameterize clients with different requests, queue or log requests, and support undoable operations.”  So in some sense, just using a functor without declaring that the intent is to do a Strategy or a Command is like using untyped variables: you are supposed to know what you are doing, but it is implicit.*

Yet another example with the Visitor pattern and its intent: “Represent an operation to be performed on the elements of an object structure. Visitor lets you define a new operation without changing the classes of the elements on which it operates.” This is typically achieved through double-dispatch in languages lacking multimethods, but regardless of how it is implemented the intent remains, and this is what matters most.

Generic Vs. specialized intents

For example, the intent of the generic Proxy pattern defined in a paper from James Noble:

The Proxy pattern is used to “Provide a surrogate or placeholder for another object the Subject to control access to it”. A Proxy object provides the same interface as the original Subject object, but intercepts any messages directed to the Subject. A Proxy object can therefore be used in place of the Subject by a client which is designed to access the Subject, without the client being aware the Subject has been replaced by a Proxy.

This intent can then be specialized for various purposes, leading to several specialized patterns:

What's your intent if you buy that? (yes this is brand new furniture for sale)
What's your intent if you buy that? (yes this is brand new furniture for sale)
  • Remote Proxy: provides a local representative to an object that is only available on a remote machine
  • Protection Proxy: checks the access rights before directing to the original object
  • Virtual Proxy: creates expensive object only on demand so that they are created only when necessary
  • Cache Proxy: an object representative that remembers the result of calling the methods of an object to avoid directing subsequent call again to this object
  • Counter Proxy (smart pointer management), etc.

We say that these patterns are specializations of the Proxy pattern. The main Proxy pattern introduces the common solution—providing a placeholder for an object. Every specialized proxy pattern is a special kind of Proxy: A “Protection Proxy” is a special kind of “Proxy”. The specialization here only deals with the Intent part of the patterns.

Conclusion

Now that functional languages are getting more attention, it becomes fashionable to question the usefulness of patterns: “Scala does that without the need for patterns”. I agree Scala is great, I disagree this argument. Patterns are first of all signs to denote intents, even if they can do more.

References

Patterns as Signs, James Noble and Robert Biddle, Victoria University of Wellington, New Zealand

Classifying Relationships Between Object-Oriented Design Patterns, James Noble, Microsoft Research Institute, Macquarie University

* By the way, how to achieve “undoable operations” by using first class functions in an elegant way? this would require passing two functions always together: do() and undo()

Read More

The Self-descriptiveness pattern

The Self-Descriptiveness pattern can save your life many times in the course of any software project. The power behind it is to harness the computer, rather than yourself.

What is Self-Descriptiveness?

Self-Descriptiveness is a property of any system that is able to describe itself with no external help. Just ask it “describe yourself” and it will. In other words, it is a system where its documentation is embedded into itself.

This pattern is so essential, however it is not expressed often. I imagine it is too obvious for those that appreciate it, whereas other do not imagine its benefits and focus on how memory or bandwidth they save instead.

You can find this pattern in many areas:

Database

Database tables in many databases are self-descriptive, thanks to definitions or system tables. This enables to query the database for its own schema, without any other documentation or explanation. This also enables tools to browse a database just by connecting to it. Databases try to be user-friendly (SQL), so here Self-Descriptiveness also means affordability for users.

XML

XML is one of the most famous example of self-descriptive format. The use of named tags around the values to describe them is supposed to enable humans or even tools to read and extract their meaning even with no prior knowledge of the exact format. The ordering could vary, elements may be missing, it would remain readable. In this case Self-Descriptiveness means robustness against variations, perhaps to be more future-proof.

This key holder is self-descriptive
This key holder is self-descriptive

Spreadsheets, associative arrays

Spreadsheets with header columns, Map and associative arrays in general are other low-tech forms of self-descriptiveness we use all the time. You don’t have to remember what each columns represents (or each slot in a Map), the headers (or keys) tell you that directly. Here Self-Descriptiveness means convenience for the developers.

Reflection

Reflection built into languages such as Java enables the software to query its own structure. It becomes then possible to introspect each class to find out which fields and methods it has, and what class or interface its extends and implements. The metamodel of Java makes class definitions self-descriptive. This feature is essential to load at runtime code that was not known at compile time. Here Self-Descriptiveness means extensibility.

Analysis patterns

This leads to the Knowledge Level pattern (also known as Metamodel), which uses objects to describe the behavior of other objects. The Operation Level becomes self-descriptive thanks to its Knowledge Level. This is typically how databases and virtual machines implement their Self-Descriptiveness. The Knowledge Level pattern is key to achieve Self-Descriptiveness for systems.

The Quantity patterns is a simple yet very powerful form of Self-Descriptiveness for your ordinary project, especially in the Domain Layer. The idea is to keep the unit attached with the value. In the case of the Money pattern, the unit is the currency. If your application deals with percents, by all means introduce from the beginning a class Percentage that follows the Quantity pattern. This simple class has the potential to completely eliminate every percent-related bug! In the finance domain, interest rates, usually expressed in percents but that are actually in percent per year, deserve their own Quantity class as well. This class should be distinct from the usual percentage. Again this simple decision wil ensure that no one will ever confuse one for another. You could even go further and type each kind of interest rate by its destination, either a coupon rate, or a yield, and therefore have the computer verify they must not be confused.

Be explicit!

Self-Descriptiveness could be generalized into one principle: “Be explicit!”. This means that everything implicit must be identified and made explicit. Your application only deals with one currency? Make the currency explicit anyway. Your time periods are all an integer number of years? Make the time unit (years) explicit anyway. All your data come from Reuters? Add a field “Source” to track the source of the data anyway. Even if YAGNI, the consistency of your domain matters. And perhaps you will gonna need it!

This chair is self-descriptive, and explicit!
This chair is self-descriptive, and explicit!

Self-Descriptiveness is obviously very precious for debugging and maintenance. First you reduce the possibility of bugs: you can try to sum 0.05 +2% + 35bp (basis points are one-hundreds of percent) and still have a valid result (7.35%) if you use Quantity objects for them. Then debugging becomes a breeze, everything tells its story, especially if you have written judicious toString() methods.

Optimization on the other hand appears to dislike Self-Descriptiveness. XML is very verbose, a Quantity object consumes more memory than just the primitive value inside, etc. In most cases, it is not worth compromising Self-Descriptiveness for the sake of optimization. And even when you need real performance, there are solutions to retain the benefits of Self-Descriptiveness without its cost. XML compression works very well, associative maps can sometimes be implemented as random access in an array, etc. And extreme needs can be satisfied with extremely sophisticated solutions*.

To be fully honest, Self Descriptiveness can hardly be achieved in practice as explained by Mark Baker in his blog, but your project deserves as much Self Descriptiveness as you can possibly afford!

*For instance the FIX protocol, used for financial data feeds, is a very old key-value format. It is not really self-descriptive as the keys are numbers, not labels. However just like self-descriptive formats, the keys impede efficiency: they consume bandwidth for almost nothing. So instead of changing the format in favour of efficiency, the protocol FAST has been invented. This optimization done by the computer, not by developers, extracts in real time the values using a message template, compress them and send them in fixed order, in other words in a fully implicit way. On reception the message is automatically rebuilt against the template. More details about FIX and FAST can be found here and here. This is a fairly sophisticated solution.

Pictures taken at the Milano Furniture Fair 2009.

Read More

Patterns as another kind of types

Patterns represent a couple (intent, solution), where the intent matters most. Based on that intents, that can be generic or specialized, I propose to consider patterns like types in languages with strong typing, for the compiler to enforce their constraints.

Declaring patterns: what for?

Consider the very simple Quantity pattern from Analysis Patterns (Fowler):

Represent dimensioned values with both their amount and their unit

By simply declaring this pattern, we immediately express many things:

  1. We wish to represent a quantity with its unit
  2. Quantity is a special kind of ValueObject (as in Fowler or DDD), hence:
    1. It is Immutable, therefore it has no setter, only a valued constructor
    2. Identity is based solely on its values, two value objects are equal if they share the same values
  3. In Java, the hashcode() and equals() methods must be implemented solely on the values of the immutable fields, and the hashcode can be cached internally; the toString() result can be cached as well
  4. In the spirit of Domain Driven Design, it also probably follows Closure of Operation and Side -Effect-Free Functions
  5. It must not depend on any heavyweight dependencies such as middleware, database or gui classes
  6. It must not depend on Entities and Services
Strangely typed furniture
Strangely typed furniture

If we had an explicit way to mark the use of the pattern in the code, and if the compiler was able to know about this pattern, then it could enforce all the above. Moreover, it could also generate corresponding unit tests to assert the dynamic and runtime aspects constrained by the pattern.

This approach can be considered similar to the typing used in programming languages, where the compiler can enforce many constraints according to the type declarations given by the developer.

In practice

In current languages there is no explicit way to declare the use of patterns in the source code. The reader of the source must pay attention to various hints and compare to his/her prior knowledge to decide whether or not a pattern is used. Typical hints include naming conventions, where the class or interface names ends with the name of the pattern participant: an interface TradeFactory is probably a Factory that creates Trade instances.

Since many years I have been using a custom Javadoc tag @pattern to explicitly mark the patterns I use in my source code, hoping that a tool could use it in the future. Over time I have noticed several benefits from doing that, the biggest one being that I am forced to be 100% clear about what I want. Many times I noticed that asking myself to explicitly declare in patterns what I was doing required me to think deeper. Though I know what I am doing, when trying to express that intent using patterns from the books I am forced to clarify my intent to decide which exact pattern I am applying.  This is fully in the spirit of the Pragmatic Programmer‘s “Programming by Coincidence“, and also in the spirit of Test Driven Development where the unit test is forcing you to clarify what you really want.

In other words, we could call that Pattern Driven Development. PDD, yet another acronym!

Yet another strangely typed object (the disco vaccum cleaner)
Yet another strangely typed object (the disco vaccum cleaner)

Discussion

It may not sound intuitive that there exists a documented pattern for each and every possible design decision, but I can confirm that the panel of existing patterns is very wide and already covers a lot of common design decisions. Such patterns are far from sophisticated solutions or tricks, they usually just name intents and known practices. But this is exactly what we need. Dirk Riehle, in a recent paper entitled Design Pattern Density Defined, found that in the case of open-source frameworks, and considering only classic design patterns, the density of such patterns was between 40 and 70%, where density is defined by “The design pattern density of an object-oriented framework is the percentage of its collaborations that are design pattern instances.

In addition, it becomes increasingly common for software teams to use custom patterns as a convenient way of documenting their common design practices. Actually this is how the pattern story began when Kent Beck and Ralph Johnson decided to document the design of HotDraw in the now classic paper: Patterns generate architectures. To illustrate that, the Drupal team is using custom patterns in addition to standard patterns to document their design in an efficient way, as described in the program of the DrupalCon Paris 2009 conference:

This session will cover the how and why of design patterns, and review the most common patterns used in Drupal as well as a number of common patterns we could be using. Some are Drupal-specific patterns and some more more general software design patterns.

Another major example of patterns dedicated for a particular project is the the Eclipse IDE. Erich Gamma, the main Eclipse designer -one of the Gang of Four- wrote the book Contributing to Eclipse: principles, patterns, and plug-ins (together with Kent Beck) where Eclipse-specific and standard patterns are used together as the primary mean of documenting the design.

Conclusion

Patterns are signs to denote intents, and I propose to promote the use of patterns (patterns already existing or to be defined in the context of a particular project) to trace these intents explicitly, and to enable tools to enforce pattern-specific constraints.

Pictures taken at la Biennale Internationale Design, Saint Etienne 2008

Read More

Toward smarter dependency constraints (patterns to the rescue)

Low coupling between objects is a key principle to help you win the battle against software entropy. Making sure your dependencies are under control matters. Several tools can enforce dependencies restrictions, such as JDepend. However in a real project with many classes, packages and modules, the real issue is how to decide and configure the allowed and forbidden dependencies. Per class? Per package? Per Module? Based on gut feeling? Is there a theory for that?

Of course, in a layered architecture, the layers specify the dependencies. This is not bad, but I am sure we can do better.

Smarter dependencies

To go further, I suggest expanding our vocabulary of concepts. In OO languages such as Java, everything is a class (or an interface), grouped into packages. Such classification is not really helpful. Fortunately, several books provide ready to use vocabularies in the form of patterns languages (not only design patterns, but patterns in general). Some of these patterns are foundations on which rules to manage dependencies can be proposed.

Disclaimer: the dependencies rules suggested below are hypothesises to be debated and verified against a corpus of actual projects, I would be happy to be given counter-examples and counter arguments.

The child really depend upon the mother
The child really depend upon the mother

Domain Driven Design

The book Domain Driven Design by Eric Evans defines a rich vocabulary of concepts used in every application, and we can leverage that vocabulary to propose some dependencies principles between them:

  • ValueObject never depends upon Entity nor Services
  • Entities should not depend upon Services (maybe not a hard rule)
  • Generic SubDomain should not depend upon Core Domain
  • Core Domain should not depend upon Cohesive Mechanism (the “What” should not depend upon the “How”)
  • Domain Layer should not depend on any infrastructure code
  • Abstract Core module never depends on its specialized or implementation modules

Analysis Patterns

The book Analysis Patterns by Martin Fowler also provides patterns as a richer vocabulary, from which we could propose:

  • Elements from a Knowledge Level should not depend upon elements from the corresponding Operation Level

I did not find that rule written in the book but every example appears to support it. Considering that classes and subclasses in usual OOP are a special case of Knowledge Level built-into the language, this would lead to:

  • Abstraction never depends upon their Implementations

which is similar to the second part of the Dependency inversion principle by Robert C. Martin:

Abstractions should not depend upon details. Details should depend upon abstractions.

Since many analysis patterns in the Analysis Patterns book involve the Knowledge Level pattern, this single dependency rule already applies to many analysis patterns: Party Type Generalizations, Measurement, Observation, Protocol, Associated Observation, Measurement Protocol etc. The pattern Quantity can be seen as a specialized ValueObject (see Domain Driven Design above) hence should also not depend on any Entity nor Service.

Design Patterns

The book Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma et al. presents the classic design patterns. These patterns define participants which are named.  In the pattern participant ignorance principle I discussed the concepts of ignorant Vs. dedicated participants within a pattern, and their consequences for dependencies:

  • Ignorant pattern participants should never depend on dedicated participants
  • Pattern participant never depend on the “Client” participant
  • For each ConcreteX participant, the corresponding abstract X never depends on it (Abstraction never depends upon their Implementations)

In practice, this means:

  • In the Adapter pattern, the Adaptee should not depend upon the Adapter, and the Target should depend upon nothing
  • In the Facade pattern, the sub systems should not depend upon the Facade
  • In the Iterator pattern, the Aggregate should not depend upon the Iterator; however every Java collection is a counter example as they contain their own ConcreteIterator.
  • In creational patterns (Abstract Factory, Prototype, Builder etc.), the Product and ConcreteProduct should not depend on the dedicated participant that does the allocation (the Factory etc.)
  • And so on for other patterns, some of which being already discussed in the pattern participant ignorance principle.

In short, if we look at the design patterns as a set of types with inheritance/implementation, invocation/delegation and creation relationships between them, the dependencies should not flow in the reverse direction of the relationships; in other words, using UML arrows, the dependencies should only be allowed in the direction of the arrows.

Addiction to sugar is a kind of dependency
Addiction to sugar is a kind of dependency

Patterns of Enterprise Architecture

In the book Patterns of Enterprise Application Architecture by Martin Fowler, the Separated Interface Pattern proposes a way to manage dependencies by defining the interface between packages in a separate package from its implementation. This is similar to the Dependency inversion principle, as discussed here, which states:

A. High-level modules should not depend upon low-level modules. Both should depend upon abstractions.

By the way this is also very similar to the recommendation in Domain Driven Design:

Abstract Core module never depends on its specialized or implementation modules.

Finally, in the spirit of UML stereotypes that we sometimes put on packages to express their intent:

  • Utils never depends on anything but other Utils

What for?

If we manage to make every use of the above pattern explicit in the source code, for instance using Java annotations or simply Javadoc tags, then it becomes possible for a tool to deduce dependencies constraints and automatically enforce them.

Imagine, just add @pattern ValueObject in your Javadoc comment, and voila! A tool is now able to deduce that if you happen to import anything but basic java.* you must be warned.

Of course the fine tuning of the default behavior can take some time (do we accept that ValueObjects may depend upon low level utils like StringUtils? Probably yes), but the result will at least be stable regardless of the refactorings.

Given the existing variety of patterns over there, I am confident that just any class or interface within a project can be declared as being a participant in at least one pattern, and have therefore its dependency constraints deduced at the same time.

Read More

After Technical Debt, technical Options

In finance, options are powerful tools for traders, and many design practices including design patterns can be seen as options. Options can -perhaps- yield a great benefit for a certain and immediate cost. If this cost is cheap enough it can be quite attractive.

An option to buy a stock is a right to buy the stock at a predefined price (let’s say EUR25) before a given date in the future. Until this date, you can freely exercise the option (i.e. buy the stock at EUR25), or not (do nothing). Of course if it happens that the stock price is greater than EUR25 (let’s say the stock is now EUR35), if you exercise the option you earn money (EUR10), and for that reason the option is worth something (EUR10 here, or even more if we expect the price to go even higher in the future).

In finance, the price of the option is directly linked to the expected benefit it can bring, even though this benefit is not certain. To be more precise, the more the stock price moves up and down, the bigger the option value: this instability of the price is called volatility, and you can think of it like a standard deviation in statistics. Options traders actually trade volatility: the higher the volatility (i.e. the less stable the prices), the more valuable the option.

Many design practices can be seen as options, and some are indeed cheap, almost free options. A free option is something that a reasonable investor cannot refuse to get, since it costs nothing while it can perhaps yields a benefit: this is actually an uncertain version of a “free lunch”. A design practice that would cost nothing is hard to resist, since it yields potential return in exchange for nothing.

In the spirit of the Technical Debt, we will focus on the cost of maintaining the code over time. Any change to the code must be weighted in accordance to the burden it will bring compared to its benefit.

Plenty of options here
Plenty of options here

Technical options in the code

In practice, everything that improves the flexibility of the software is de facto like an option: it creates additional opportunities that we can use or not in the future. Typically, it can enable the designer to change his mind at a later stage at no cost, or to defer a design decision.

The design principle to code against abstractions rather than implementation comes immediately to mind: coding against interfaces is probably the number one technical option. Therefore the question is: given this piece of concrete code, should I introduce an interface and reference the code through the interface, or should I leave it like that? Just like in finance, the answer is: it depends on the upfront cost of the option (adding the extra interface) compared to the anticipated stability of the feature the code is doing (is it likely to change in the future?).

In this case, the upfront cost depends on the tools (with modern IDEs with wizards and refactoring capabilities it is easier to add or even extract an interface automatically), and perhaps on what you like to do or not (if you hate having too many classes then adding an interface will cost you more, subjectively). Some commercial source control systems that are not convenient with widespread refactoring also make the cost of future changes much higher, and as such they encourage building flexibility upfront.

Going further, many design patterns can be considered as options to improve flexibility for a minimal cost. Out of the GoF patterns, the patterns Abstract Factory, Strategy, State, Visitor, Iterator, Bridge and Builder directly plan for easy change of behaviour or extension in the future (typically through additional interfaces as well), and thus are simple yet efficient ways to buy technical options.

Various elements combine together for bigger value added
Various elements combine together for bigger value added

Technical options Vs. YAGNI

Knowing the future is hard, and YAGNI is there to remind us that. If there are practices that cost a little to introduce and that can bring benefits later, and if we are so poor at anticipating change, why not just introduce them later? This is right, in most cases YAGNI applies, unless we can anticipate a need. The funny thing is that while the introduction of interfaces (directly or through the patterns mentioned before) creates opportunities to vary the implementations, it also creates opportunities for other opportunities. The new interface enables the addition of non-intrusive structural patterns that revolve around an existing interface, such as Decorator, Composite, Proxy, Adapter, Null Object, and every combination of them as suggested by the Interpreter pattern. (This introduction of such patterns, by the way, is precisely what most IoC containers, application servers and AOP frameworks do to add functionalities). How these patterns combine together under the common “umbrella” interface is kind of fascinating to me, and it always reminds me about the Network Effect: the more we use an interface, with implementations or opportunistic patterns, the more useful it becomes. I believe that a net of patterns like that can become so useful that it becomes irresistible. You will gonna need it.

Of course there is nothing really new in all that, nothing more than a new metaphor to help explain to managers. Even if they do not know what an option is, I am sure they can understand the concept very quickly.

I have been pushing this approach that I called “Think options, not solutions” in my former team to emphasize that we can build the system even if some of its business behaviours are not completely known yet, as long as we can define their contours and start with an arbitrary solution. The goal is that we are confident we can change it later with no pain. This was justified from a business point of view because we did not know what the market was expecting, for the product (an interest rate swaps multilateral trading facility) was too novel. In that environment, change was definitely not something we feared, rather the opposite: it was stimulating.

Just like traders speculate on volatility using options, developers can speculate on the stability of the business features using (or not) design practices that increase flexibility. The little extra effort to introduce these practices must be weighted by the expectation that it will be useful. And just for the fun, anyone interested in computing the value of a technical option using the Black-Scholes pricing model?

Read More

The pattern participant ignorance principle

Not all participants in a pattern are equal, some are aware there is a pattern, while other participants do not, cannot and must not. Let us have a look at that principle and its consequences for coupling and dependencies management.

Some patterns are intrusive, which means that adding them to an existing code base requires changing the code base. Fortunately many patterns are not intrusive, and given an existing code base, introducing such patterns only requires the addition of specific code, more precisely the addition of the pattern participants that are dedicated to the pattern.

However in the patterns books, each pattern defines many participants, many of which being actually already in the existing code, and in the case of non-intrusive patterns they do not have to be changed to play their role in the patterns: these participants are therefore ignorant participants, they participate without knowing. Participants that are not ignorant are called dedicated participants.

ignorance

If we were now to remove the pattern occurrence, we would just delete the dedicated participants and leave the ignorant participants intact. This is what I call the Pattern Participants Ignorance principle (PPIP):

In a pattern occurrence, every participant that would remain after we completely removed the pattern occurrence from the code must NOT be aware of the pattern, and is called an ignorant participant; adding or removing the pattern to the code must not change a single line of the source code of these participants. Participants that are not ignorant are called dedicated participants

If the source code is in source control, adding or removing a pattern should never involve any commit to the source of any ignorant participant. If the source code is not available, we could still introduce the patterns. In simpler words, ignorant participants are passive participants, whereas dedicated participants only exist to support the pattern.

By definition, every ignorant participant should not be aware of the pattern and this implies in particular that they should not be aware of any dedicated participant.

This works well for many patterns, where every participant is ignorant but a few:

  • Decorator: the participants Decorator and Concrete Decorator are the only dedicated participants
  • Proxy: the participant Proxy is the only dedicated participant (more code can actually be also dedicated, for instance to realize a remote proxy)
  • Adapter: the participant Adapter is the only dedicated participant
  • Composite: the participant Composite is the only dedicated participant
  • Null Object: the Null Object itself is the only dedicated participant (of course)

There are patterns that do require modifications to the existing source code, they are intrusive. Intrusive patterns violate Participant Ignorance since ignorant participants must know a little about the pattern because they are modified to support it.

Some patterns are almost non-intrusive except for the calling code (Client participant) that need to be changed a bit:

  • Abstract Factory: the Abstract Factory and Concrete Factory are the only dedicated participants, but the calling code must call the new factory; however the Product and Concrete Product participant are ignorant
  • Builder: the Builder and Concrete Builder are the only dedicated participants, but the calling code must call the new builder; however the Product and Concrete Product participant are ignorant
forbidden
Forbidden to go (really)

Other patterns are much more intrusive. For example, the Visitor pattern, when applied with double-dispatch, requires the addition of accept(Visitor) methods in each element to be visited, which modifies participants that are indeed ignorant and that should not be touched in an ideal world. Visitors using a switch-case approach or a reflection-based approach, or coded in a language that support multi-methods are not intrusive, and “purer”.

The Observer pattern also requires the Subject participant to know about its Observers, with the addition of a collection of Observers, attach() and detach() methods, and the notification logic, whereas the Subject would remain in the absence of the Observer pattern: it is an ignorant participant, and the typical Observer pattern violates  Participant Ignorance.

To be fair, there are patterns for which our perspective is less relevant, where almost every participant appears to be deeply involved (very dedicated) to the pattern, and if we removed the pattern we would change everything. In the State pattern, each State and ConcreteState usually deals with the Context, which knows them as well.

In the Mediator pattern every colleague knows the Mediator, and each Concrete Mediator deals with each Colleague or Concrete Colleague. Introducing a Strategy is also really intrusive, ironically for the purpose of being able to add new behaviours in a non-intrusive way during software evolution…

Participant Ignorance helps to think clearly about the forces in action in a pattern, which usually discourage coupling (the more ignorant participants the better). If like I do you document the use of patterns in your code by using Javadoc tags (or Java annotations), then the principle states that these tags or annotations must only be located in the code of the dedicated participants to leave ignorant participant untouched when refactoring to or from patterns. More interestingly, the PPIP enable to derive allowed and forbidden dependencies between the participants of a pattern occurrence: ignorant participants must never depend on dedicated participants.

Read More

Functional style in Java: implicit learning

Bill Venners from Artima recently published “How Scala Changed My Programming Style“. This echoes the advise that you should “Learn a new language every year (Pragmatic Programmers)”, but this time, this was about Scala (of course):

How did Scala change how I think about programming? In short: I learned to appreciate the functional style. The functional style of programming emphasizes immutable objects, variables that can be initialized but not reassigned (final variables in Java), transformation of data structures, and methods and control constructs that result in a value but have no side effects.

In my programming experience I also changed my Java programming style toward a more functional style thanks to many other influences than learning a new language: patterns and APIs primarily, and even the default suggestions of tools.

Patterns

Some patterns books implicitly document patterns about how to adopt some of the benefits of the functional style in any language, in particular the pattern Immutable (Grand), or Side-effect free functions (Evans), Pipes and Filter (Hohpe), etc.

Classic design patterns such as Iterator, when combined with Decorator, easily yield a functional style of chaining elements together; when also combined with Strategy, for instance for the criterion to filter an Iterator with (in other words a predicate), you can get an equivalent of the higher-order map and filter methods Collect and Select on a collection, and so on.

API

The Apache Commons Collections API provides “standard” interfaces and implementations for this approach, with the interfaces Predicate, Closure and Transformer, the powerful FilterIterator and many other convenient ready-to use classes. This API really suggests to think differently than the usual imperative style.

Recent approaches such as Linq or Quaere “add a querying syntax reminiscent of SQL” to .Net and Java applications, and allow “you to filter, enumerate and create projections over a number of collections and other queryable resources using a common, expressive syntax”. Again, these APIs do suggest to manipulate objects and data in a typical functional style.

Tools

Even an IDE such as Eclipse, when using its default code completion templates often suggest the use of final. Tools such as Checkstyle or PMD also do so. Lots of small hints that together can eventually lead you to have a deeper look at what is functional programming.

Final word

I have to admit however that the authors that I read most were themselves often strongly influenced by Smalltalk, which has a strong functional potential… Also it is rather easy to regard patterns or even APIs as other languages: patterns can form a pattern language, and an API can be considered a kind of DSL

Since we adopted this functional style in my team our code got really smaller, clearer, less buggy (thanks to immutable and side-effect free methods) and much easier to extend, maybe at the cost of more (extremely small) Java classes: but who cares?

Read More

Essential patterns books for today object-oriented developers

The more patterns developers know, the most efficient they become within a team: it only takes one or two words (the pattern name) to communicate a design decision or proposal, instead of 10 mn of explanations. Communication also gets much more accurate and to-the-point (or less fuzzy).

Because patterns often form a pattern language, not only they offer a standard vocabulary but they also help structuring the mind by their relationships: patterns can be related as exclusive alternatives, or rather often-go-together, which is useful.

Patterns are not always supposed to be “tricks” to learn, or extra complication to introduce to a design; indeed even obvious and uninteresting pattern are worth reading in books, just for the social advantage of being able to reference them. Going further, sometime you can just go to the book to find out which is the pattern that you just did without knowing its name…

Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley Professional Computing Series)
1.  Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley Professional Computing Series) by Erich Gamma et al.
The most essential book, a definitive must-have. Every experienced software developer must know these 23 design patterns.

Analysis Patterns: Reusable Object Models (Addison-Wesley Object Technology Series)
2.  Analysis Patterns: Reusable Object Models (Addison-Wesley Object Technology Series) by Martin Fowler
The essential complement to the GoF, to extend the benefits of patterns from design to analysis (closer to the problem to solve in the domain). Each pattern will not necessarily teach you a new solution, but will always give you have a standard name for each solution, which is already worth the book.

Domain-Driven Design: Tackling Complexity in the Heart of Software
3.  Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans
A book that can change the way you deal with software. Considering the domain as the main driver of the design is so powerful an approach! Works best together with the analysis patterns.

Patterns of Enterprise Application Architecture (Addison-Wesley Signature Series)
4.  Patterns of Enterprise Application Architecture (Addison-Wesley Signature Series) by Martin Fowler
Other analysis patterns to solve many technical problems, useful complement to Analysis Patterns.

Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions (Addison-Wesley Signature Series)
5.  Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions (Addison-Wesley Signature Series) by Gregor Hohpe
Very complete pattern language (both test and visual vocabulary) about messaging, will improve the communication within a team a lot, both in efficiency and in accuracy.

Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects
6.  Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects by Douglas Schmidt
Important book, more specialized toward optimized low-level problems (web servers…). often quoted. The other volumes are less relevant for today software development on modern languages or virtual machines.

Patterns in Java: A Catalog of Reusable Design Patterns Illustrated with UML, 2nd Edition, Volume 1
7.  Patterns in Java: A Catalog of Reusable Design Patterns Illustrated with UML, 2nd Edition, Volume 1 by Mark Grand
The book I discovered design patterns in, quite affordable, but some colleagues do not like it that much.

Read More

Systematic combination of subpatterns generates design patterns

Composite patterns, such a the Bureaucracy pattern, are patterns built by the composition of other “smaller” patterns. However even usual design patterns can be considered composite patterns made of smaller subpatterns.

The goal is therefore to find out which are the main subpatterns that enable to reconstruct as many design patterns as possible.

The subpatterns

From an early analysis I believe that four subpatterns form the foundation of many GoF design patterns:

  1. Type Hierarchy and its variants (interface, abstract class, non final class…)
  2. Type Set, a set of types with no particular relationship between them
  3. Delegation and its variants (one-to-one, one-to-many)
  4. Allocation (creation of instances) and its variants

The experiment

To validate this proposal, I have coded a small experiment to generate every combination of two subpatterns out of (1) and (2) plus one relationship between them out of (3) and (4), or every combination of one subpattern out of (1) and (2) plus a relationship of (3) and (4) to either itself, or in the case of (1), between a super type and a subtype or the other way round.

For each generated combination I have automatically generated its corresponding class diagram, and grouped them together into one SVG document, converted into PDF using Inkscape. The resulting picture is available here, and you can see a partial preview below:

combinations_thumb

Results

The interesting finding is that this simple combinatorial method re-discovers 11 well-known design patterns (in the reading order of the picture):

  • Template Method (delegation to self, expecting a subtype to provide the service)
  • Composite (Considering a one-to-many delegation) or Decorator / Proxy (one-to-one delegation)
  • Prototype (creates instance of this type)
  • Bridge (one Abstraction hierarchy delegates to another Implementor hierarchy)
  • Abstract Factory (one Factory hierarchy creates instances of another (or several others) Products hierarchy)
  • … skip 4 next …
  • Strategy or State or Builder (some client delegates to some hierarchy)

Note that there are several cases where we could see the Factory Method, Adapter or Facade patterns. This would lead to some 13 design patterns out of the 23 in the GoF book.

Discussion

It is obvious anyway that the four subpatterns used here are not enough to rebuild the full set of the 23 design patterns. The Command pattern is unique in its use of a separate invoker, and the Chain of Responsibility is clearly about how handlers self-assign a given task without any centralized management. In these two examples however, the unique characteristics is not in their structure but rather in their dynamics. Sequence diagrams would be much more relevant to convey that kind of information.

The full test runs in one go in a fraction of second. The source code is available here: patternitygraphic_src, the code of the experiment is in CombinationTest.

EDIT: the plain SVG file that was generated is also available here for download:

Read More

What’s wrong with Design Patterns ?

I’m a big fan of design patterns, they do make my work easier, more explicit, more maintenable and also more enjoyable; however I’m surprised that not everyone knows them and actually uses them. What’s wrong with design patterns ?

Some facts:

  • they are difficult to learn and to teach, it is nearly impossible to teach design patterns at school unless students already have significant hands-on practice of software development, unless they already encountered real-world problems
  • they are difficult to implement correctly when you start with them, you often misuse them at first tries
  • they do not match the way many people think about software, many people still resist to them, they think using design patterns is counter-intuitive (this is especially true if you started your carrer with a procedural point of view, how many times people tell me: “using patterns or not, it is the same at the end of the day in the software !”)
  • once you know them, you are impressed at the power and expressiveness they give you, so you tend to overuse them

Since I’m still convinced design patterns MUST be known by any professional developer today, what are the best ways to learn or teach them ?

  • There is no magic answer, however these are some clues:
  • mentoring is undoubtedly the best way to learn design patterns; so the best way to learn them is to find coworkers that know them more than you do; it is even better to work on the same part of the software to really get the point in using, -or not using- patterns (pair programming ?)
  • If patterns have to be taught in a classroom environment to people that already have work experience (as in company internal training sessions), it is worthless to describe and explain the catalog of patterns, you’d better explain what you found difficult or confusing when you were learning them, this is much more valuable for your audience. For instance, explain that the intent of a pattern comes first, that yes it is normal that the UML diagrams may be the same for two or three patterns, because their intent is different, that yes they are already using some patterns without even being aware of it… I think presenting patterns through refactoring examples should also be very relevant since the value-added of patterns is more obvious.
  • Maybe design patterns are too abstract to be learnt directly (1); as usual when abstraction is a problem, we should then focus on more concrete examples (or instances). In the case of design patterns, more concrete examples may be J2EE patterns (it is very surprising to see that many people know much more about J2EE patterns than fundamental GoF patterns !), or Analysis Patterns that derive from design patterns and that solve a particular domain problem. Once the apprentice knows enough such concrete patterns, it becomes easier to describe the abstract pattern that sums them all.

(1) Actually one strength of design patterns is that they are abstract enough to be reusable to a great extent.

Initially published on Jroller on Monday April 25, 2005

Read More