Patternity

Pattern grammar for the variant problem

cyrille

For tools to be aware of patterns, the patterns must be formalized, at least partially. At this point I must quote Gregor Hohpe to clarify my thoughts, as I strongly agree with his skipticism:

Typically, when people ask me about “codifying” or “toolifying” patterns my first reaction is one of skepticism. Patterns are meant to be a human-to-human communication mechanism, not a human-to-machine mechanism. After all, I have pointed many people to the fact that a pattern is not just the piece of code in the example section. It’s the context-problem-forces-solution combination that makes patterns so useful.

Patterns link together a problem part to a solution part. This is expressed within the limits a stated context outside of which it is no more applicable. Patterns also emphasize the forces involved, that you must consider to decide how and whether or not to apply the pattern.

Patterns litterature usually describes examples of application of patterns. In your project, you will have to do more work to adapt the pattern solution to your exact need. A pattern solution may be stretched a lot, but the pattern remains as long as humans still recognize its presence. Every different way of applying a pattern is called a pattern variant.

Addressing the variant problem

Formal descriptions of anything human is too restrictive, and this is especially true for patterns which are the product of human analysis, in that they resist simple formalization. However if we focus on sub-parts of patterns, it becomes easier to formalize them, at least for their solution part.

For example a design pattern that uses (in its solution part) some form of inheritance admits several variants. At first, the pattern solution seems to resist against its formalization. However if we now focus on the inheritance part only, we can enumerate every possible alternative for it. For example we can use:

  • interface and classes that implement it
  • abstract class and classes that extend it
  • concrete class provided it is not final (assume we’re in Java), so that it can be extended

Notice that each alternative is a solution to the same problem “How to realize some

Tree structure in volume (Milano International Fair 2009)
Tree structure in volume (Milano International Fair 2009)

form of inheritance”. We can say that each alternative is indeed a pattern, and by the way Mark Grand already described them in Patterns in Java Volume 1. These patterns are easy to formalize, as they can be precisely described in terms of programming language elements.

How can we split a pattern into parts? The idea is to identify the areas that are fragile with respect to the variant problem in the solution part of a pattern, and to consider them as lower-level problems embedded inside the bigger pattern.

In the example before, the problem was to achieve “some form of inheritance”, and we listed three patterns that address this problem.

Provided it can be split into sub-parts (hence into smaller problems), any pattern solution can be formalized by recursively formalizing its sub-parts. If a sub-part cannot be easily made formal, then it can be split again into sub-parts, and so on until each sub-part can.

Given a pattern solution that we want to formalize:

  • If it can be described formally directly, then we are done (terminal)
  • If it cannot be fully described formally, then extract the problematic sub-parts into sub-problems, then find every pattern that addresses each sub-problem, and formalize their solution part

We can then represent a pattern as a tree of smaller patterns, where the solution part of each patter is connected to the problem part of another pattern.

From pattern language to pattern grammar

In the car, some parts can be replaced by other alternate parts that play the same contract (e.g. the wheels)
In the car, some parts can be replaced by other alternate parts that play the same contract (e.g. the wheels)

When the pattern is applied into the code, at each node in the tree there is actually a selection of which variant of the sub pattern to use. As such, each selected pattern represents an atom of decision in the design process.

It then appears that we have a form of grammar for patterns, where there are terminal patterns solutions T (easier to formalize in term of programming language elements), non-terminal parts of patterns solutions N (that cannot be easily formalized but that can formally ask for help to solve their sub-problems), and where the production rules P are nothing but the patterns themselves:

Patterns are production rules that link:
elements of N (the problems) to elements of (N Union T)

Conclusion

I have suggested quickly a way to formalize patterns solutions in spite of their fragility with respect to their variants. This approach identifies patterns as production rules in some grammar over the set of patterns considered.

This perspective is well-suited for tools to work on patterns in real-world projects, where the patterns are indeed applied in many variant forms. The problem of this approach is that every known pattern that is variant-fragile must be reconsidered and have its solution split into a formal part and one or several sub problems to be addressed by specific, lower-level patterns, which themselves must be formalised in the same way.

It is essential that for every problem (“intent”) we can enumerate every pattern that addresses it. Intents can be also classified as a taxonomy, where some intents are specialized versions of more generic intents.

This approach does not claim to formalize the full potential of patterns, it only aims at enabling tools to understand patterns that are already there so that to assist the developers for various tasks.

Read More

Patterns in general, not just design patterns

cyrille 1 Comment

Over time, patterns have appeared on many different topics, not all related to programming. Here is a list of patterns and pointers to other lists of patterns, to illustrate two things:

  1. Knowledge and experience in general can be packaged into patterns, often using the pattern form. Patterns are convenient for reuse, in any domain.
  2. There are already plenty of patterns, and they cover a really wide range of situations. Given the number of patterns available today, whatever your problem, you will likely find helpful patterns for it.

This should encourage you to search for existing patterns whenever you need additional insight, or just an already documented reference for what you’ve just done (documenting the design of software using patterns).

The list in this post will grow, but without the ambition of listing every possible pattern.

Software Design

Here are two essential lists of books on patterns:

Other books and/or websites focus on patterns for software development, each with a specialized perspective:

Please notice that while many patterns are to be applied directly to software design or source code, there are also many patterns about the process of building software.

This list could not be complete without referring to the big Pattern Almanach by Linda Rising, available on her website here in PDF or in a web version here.

The Hillside.net Patterns catalog and the many papers published after each PLoP also provide many patterns that may well cover your problem.

User interfaces patterns

Testing patterns

Misc

Probably any domain can benefit from using patterns to represent chunks of knowledge and experience. Here are some examples quite foreign to software developement:

Conclusion

Though this listing is far from extensive (there are many independently published articles on patterns everywhere), it shows that the pattern community has been quite active to mine as many patterns it could, and this knowledge has been carefully documented in the pattern form, ready for reuse in your next projects.

Of course you don’t need to know them all, not even read them all in advance. However it does help to be aware that for most problems, there probably already exists a few patterns that can prove helpful, and they are only an Internet search away from your needs!

Read More

Patterns as stored arrangements: toward more efficient tools

cyrille 1 Comment

In nature, out of every possible arrangement of several elements, only a few arrangements are stable. This is illustrated with atoms combined together, or smaller particles arranged together into atoms, where not every combination is sustainable.

Unstable arrangements tend to move toward stable ones over time. Whenever you observe the elements, you mostly see stable arrangements of them. Because there is only a relatively small number of stable arrangements, a brain can be trained to recognize them, and they can even be named and incorporated into the language.

Better with a brain

The capability to recognize common arrangements of elements is beneficial because it saves a lot of time and thinking. Rather than describing in the details each arrangement each time, it is therefore very economical -cheaper- to describe each stable arrangement once, and then to declare that such arrangement happens in this or that case. Saying: “this is an equilateral triangle” is times more efficient than explaining what it is: “there are 3 lines of the same length, each connected to the two others such as they form a closed path, etc.” It also enables thinking at a higher level.

In software development

Small parts arranged together into bigger (and higher-level functionality) parts
Small parts arranged together into bigger (and higher-level functionality) parts

In software development, the usual elements are classes, interfaces, methods, fields, associations (implementation, delegation, instantiation) and various constraints between them. Given a few of these elements we can form many possible arrangements, however only a relatively small number of them is useful and of interest. This happens because the useless arrangements tend to be quickly replaced by the skill-full developer into other that are more useful. For example, any arrangement of two distinct classes that depend to each other, forming a cycle, is usually not desirable. Patterns authors have been working for almost two decades to inventory as many useful arrangements as they could find, resulting into many books and publications in every domain.

Common and stable arrangements of two to three classes together form the basis for design patterns as in the GoF book, an idea I have experimented in a previous post: systematic combination of subpatterns generates design patterns.

Common stable arrangements of methods, fields and how they relate with each other within one class are simply stereotypes of classes, which we tend to call patterns anyway like the ValueObject pattern in the Domain Driven Design book.

Note that in this discussion we are focusing on arrangements of programming elements in the solution space, not in the problem space, but pattern express intents too.

Harnessing stable arrangements of things: toward more efficient tools

I believe that making explicit the use of predefined common stable arrangements of programming elements, in the coding process, can boost the efficiency of many tools. I also believe that such common and stable arrangements of programming elements have already been identified and are already well-documented in the existing pattern literature.

Rather than configuring tools at the programming element level (class, field, method etc.), if the code is augmented with explicit declarations of the patterns used, the tools can then be configured at the pattern level. For each tool, the idea is to prepare in advance how to configure it for each supported pattern. This preparation must be automated, so that given an occurrence of a known pattern in the code base, the configuration of the tool can be automatically derived from the particular details of the pattern occurrence.

In other words:

tool configuration=
  auto-configuration(pattern, tool) + pattern occurrence

A simple case study

To start with an example, let us consider the pattern Abstract Factory, that defines an Abstract Factory interface and one or more Product interface(s). Then assume that in our code base we have an occurrence of this pattern, where the interface WidgetFactory participates as the Abstract Factory, and the interfaces Window and Button participate as Product. Concrete classes form two families, one Linux family (LinuxWidgetFactory, LinuxWindow, LinuxButton) and one Mac family (MacWidgetFactory, MacWindow, MacButton), where each concrete class participates as ConcreteFactory, ConcreteProduct and ConcreteProduct respectively.

Dependencies restrictions (à la Jdepend + Junit)

The auto-configuration(AbstractFactory pattern, dependency checker tool) could be programmed like the following:

//Given a pattern occurrence from the actual base: occ

//Factory interface knows about the Product interface, not the other way round
For each Product in occ, add constraint (Product, Must not depend upon, AbstractFactory)

//ConcreteProduct must not know about the AbstractFactory
For each ConcreteProduct participant in occ, add constraint (ConcreteProduct, Must not depend upon, AbstractFactory)

//ConcreteProduct must not know about the ConcreteFactory
For each ConcreteProduct participant in occ, add constraint (ConcreteProduct, Must not depend upon, ConcreteFactory)

//Interfaces must not depend upon their implementor
For each abstract participant in occ, add constraint (participant, Must not depend upon, implementor of participant)

I have expanded the auto-configuration script to highlight how we can do more sophisticated configuration as soon as it is supposed to be reused many times, something we would never afford for one-shot configuration. Also in the above script, it is quite obvious that we can extract more powerful primitives to simplify the declaration of the script itself.

I have already presented this idea: toward smarter dependency constraints (patterns to the rescue).

Dependency injection (à la Spring)

The auto-configuration(AbstractFactory pattern, IoC container) could be programmed like the following:

//Given a pattern occurrence from the actual base: occ
//Given the selected family (Mac or Linux) we want to inject: F

//Bind the ConcreteFactory from F to the AbstractFactory interface
For the AbstractFactory participant in occ, and for the ConcreteFactory in occ that belongs to F, bind (ConcreteFactory, AbstractFactory)

//Bind each ConcreteProduct from F to the Product interface
For each ConcreteProduct in occ that belongs to F, bind (ConcreteProduct, corresponding Product)

Again we can see that we can automate the binding of each implementation to its corresponding interface from the single explicit selection of the family F we want to inject. This is a great way to factor out dependency injection to make it more manageable. In particular, the configuration is closer to the way we actually think.

Other tools

The same idea can be applied for many other tools, in particular:

Conclusion

In this post I described how it makes sense to consider overall arrangements of programming elements as higher-level constructs, which I identify with patterns (the solution part of patterns in fact). I emphasize the fact that the useful arrangements are not that numerous, and that many of them are already documented in the pattern literature. Finally I present how such arrangements or patterns, if declared explicitly in the code, can be leveraged to automate tools configuration in a powerful way.

As usual, any feedback is highly welcome.

Read More

Patterns express intents

cyrille

Patterns represent a couple (intent, solution); sometime they refer to a solution, more often they essentially represents an intent, independently of its solution.

Sometimes the solution part of patterns includes a trick or a workaround to overcome the limits of a language, but patterns cannot be reduced to that trick. Indeed, a very important role of patterns (not only design patterns but patterns in general ) is that they represent stereotypes of intents.

A matter of intent

Therefore, it does not really matter if the Strategy pattern can be expressed using a Java interface, a C++ functor or a first-class function: it remains a Strategy because this is just what we want: “Strategy lets the algorithm vary independently from clients that use it“.

Is the intent of this book holder clear?
Is the intent of this book holder clear?

Another similar pattern is the Command pattern, which intent is:” Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.” Here the intent talks about ‘object’ because it was written for an object-oriented context, however it can easily be made generic if you think ‘handle on function’ (or closure etc.) instead of object. Again, even if first class functions such as delegates in C# can achieve this goal, they do not replace the need to declare the precise intent: “you want to parameterize clients with different requests, queue or log requests, and support undoable operations.”  So in some sense, just using a functor without declaring that the intent is to do a Strategy or a Command is like using untyped variables: you are supposed to know what you are doing, but it is implicit.*

Yet another example with the Visitor pattern and its intent: “Represent an operation to be performed on the elements of an object structure. Visitor lets you define a new operation without changing the classes of the elements on which it operates.” This is typically achieved through double-dispatch in languages lacking multimethods, but regardless of how it is implemented the intent remains, and this is what matters most.

Generic Vs. specialized intents

For example, the intent of the generic Proxy pattern defined in a paper from James Noble:

The Proxy pattern is used to “Provide a surrogate or placeholder for another object the Subject to control access to it”. A Proxy object provides the same interface as the original Subject object, but intercepts any messages directed to the Subject. A Proxy object can therefore be used in place of the Subject by a client which is designed to access the Subject, without the client being aware the Subject has been replaced by a Proxy.

This intent can then be specialized for various purposes, leading to several specialized patterns:

What's your intent if you buy that? (yes this is brand new furniture for sale)
What's your intent if you buy that? (yes this is brand new furniture for sale)
  • Remote Proxy: provides a local representative to an object that is only available on a remote machine
  • Protection Proxy: checks the access rights before directing to the original object
  • Virtual Proxy: creates expensive object only on demand so that they are created only when necessary
  • Cache Proxy: an object representative that remembers the result of calling the methods of an object to avoid directing subsequent call again to this object
  • Counter Proxy (smart pointer management), etc.

We say that these patterns are specializations of the Proxy pattern. The main Proxy pattern introduces the common solution—providing a placeholder for an object. Every specialized proxy pattern is a special kind of Proxy: A “Protection Proxy” is a special kind of “Proxy”. The specialization here only deals with the Intent part of the patterns.

Conclusion

Now that functional languages are getting more attention, it becomes fashionable to question the usefulness of patterns: “Scala does that without the need for patterns”. I agree Scala is great, I disagree this argument. Patterns are first of all signs to denote intents, even if they can do more.

References

Patterns as Signs, James Noble and Robert Biddle, Victoria University of Wellington, New Zealand

Classifying Relationships Between Object-Oriented Design Patterns, James Noble, Microsoft Research Institute, Macquarie University

* By the way, how to achieve “undoable operations” by using first class functions in an elegant way? this would require passing two functions always together: do() and undo()

Read More

Patterns as another kind of types

cyrille 1 Comment

Patterns represent a couple (intent, solution), where the intent matters most. Based on that intents, that can be generic or specialized, I propose to consider patterns like types in languages with strong typing, for the compiler to enforce their constraints.

Declaring patterns: what for?

Consider the very simple Quantity pattern from Analysis Patterns (Fowler):

Represent dimensioned values with both their amount and their unit

By simply declaring this pattern, we immediately express many things:

  1. We wish to represent a quantity with its unit
  2. Quantity is a special kind of ValueObject (as in Fowler or DDD), hence:
    1. It is Immutable, therefore it has no setter, only a valued constructor
    2. Identity is based solely on its values, two value objects are equal if they share the same values
  3. In Java, the hashcode() and equals() methods must be implemented solely on the values of the immutable fields, and the hashcode can be cached internally; the toString() result can be cached as well
  4. In the spirit of Domain Driven Design, it also probably follows Closure of Operation and Side -Effect-Free Functions
  5. It must not depend on any heavyweight dependencies such as middleware, database or gui classes
  6. It must not depend on Entities and Services
Strangely typed furniture
Strangely typed furniture

If we had an explicit way to mark the use of the pattern in the code, and if the compiler was able to know about this pattern, then it could enforce all the above. Moreover, it could also generate corresponding unit tests to assert the dynamic and runtime aspects constrained by the pattern.

This approach can be considered similar to the typing used in programming languages, where the compiler can enforce many constraints according to the type declarations given by the developer.

In practice

In current languages there is no explicit way to declare the use of patterns in the source code. The reader of the source must pay attention to various hints and compare to his/her prior knowledge to decide whether or not a pattern is used. Typical hints include naming conventions, where the class or interface names ends with the name of the pattern participant: an interface TradeFactory is probably a Factory that creates Trade instances.

Since many years I have been using a custom Javadoc tag @pattern to explicitly mark the patterns I use in my source code, hoping that a tool could use it in the future. Over time I have noticed several benefits from doing that, the biggest one being that I am forced to be 100% clear about what I want. Many times I noticed that asking myself to explicitly declare in patterns what I was doing required me to think deeper. Though I know what I am doing, when trying to express that intent using patterns from the books I am forced to clarify my intent to decide which exact pattern I am applying.  This is fully in the spirit of the Pragmatic Programmer‘s “Programming by Coincidence“, and also in the spirit of Test Driven Development where the unit test is forcing you to clarify what you really want.

In other words, we could call that Pattern Driven Development. PDD, yet another acronym!

Yet another strangely typed object (the disco vaccum cleaner)
Yet another strangely typed object (the disco vaccum cleaner)

Discussion

It may not sound intuitive that there exists a documented pattern for each and every possible design decision, but I can confirm that the panel of existing patterns is very wide and already covers a lot of common design decisions. Such patterns are far from sophisticated solutions or tricks, they usually just name intents and known practices. But this is exactly what we need. Dirk Riehle, in a recent paper entitled Design Pattern Density Defined, found that in the case of open-source frameworks, and considering only classic design patterns, the density of such patterns was between 40 and 70%, where density is defined by “The design pattern density of an object-oriented framework is the percentage of its collaborations that are design pattern instances.”

In addition, it becomes increasingly common for software teams to use custom patterns as a convenient way of documenting their common design practices. Actually this is how the pattern story began when Kent Beck and Ralph Johnson decided to document the design of HotDraw in the now classic paper: Patterns generate architectures. To illustrate that, the Drupal team is using custom patterns in addition to standard patterns to document their design in an efficient way, as described in the program of the DrupalCon Paris 2009 conference:

This session will cover the how and why of design patterns, and review the most common patterns used in Drupal as well as a number of common patterns we could be using. Some are Drupal-specific patterns and some more more general software design patterns.

Another major example of patterns dedicated for a particular project is the the Eclipse IDE. Erich Gamma, the main Eclipse designer -one of the Gang of Four- wrote the book Contributing to Eclipse: principles, patterns, and plug-ins (together with Kent Beck) where Eclipse-specific and standard patterns are used together as the primary mean of documenting the design.

Conclusion

Patterns are signs to denote intents, and I propose to promote the use of patterns (patterns already existing or to be defined in the context of a particular project) to trace these intents explicitly, and to enable tools to enforce pattern-specific constraints.

Pictures taken at la Biennale Internationale Design, Saint Etienne 2008

Read More

Toward smarter dependency constraints (patterns to the rescue)

cyrille

Low coupling between objects is a key principle to help you win the battle against software entropy. Making sure your dependencies are under control matters. Several tools can enforce dependencies restrictions, such as JDepend. However in a real project with many classes, packages and modules, the real issue is how to decide and configure the allowed and forbidden dependencies. Per class? Per package? Per Module? Based on gut feeling? Is there a theory for that?

Of course, in a layered architecture, the layers specify the dependencies. This is not bad, but I am sure we can do better.

Smarter dependencies

To go further, I suggest expanding our vocabulary of concepts. In OO languages such as Java, everything is a class (or an interface), grouped into packages. Such classification is not really helpful. Fortunately, several books provide ready to use vocabularies in the form of patterns languages (not only design patterns, but patterns in general). Some of these patterns are foundations on which rules to manage dependencies can be proposed.

Disclaimer: the dependencies rules suggested below are hypothesises to be debated and verified against a corpus of actual projects, I would be happy to be given counter-examples and counter arguments.

The child really depend upon the mother
The child really depend upon the mother

Domain Driven Design

The book Domain Driven Design by Eric Evans defines a rich vocabulary of concepts used in every application, and we can leverage that vocabulary to propose some dependencies principles between them:

  • ValueObject never depends upon Entity nor Services
  • Entities should not depend upon Services (maybe not a hard rule)
  • Generic SubDomain should not depend upon Core Domain
  • Core Domain should not depend upon Cohesive Mechanism (the “What” should not depend upon the “How”)
  • Domain Layer should not depend on any infrastructure code
  • Abstract Core module never depends on its specialized or implementation modules

Analysis Patterns

The book Analysis Patterns by Martin Fowler also provides patterns as a richer vocabulary, from which we could propose:

  • Elements from a Knowledge Level should not depend upon elements from the corresponding Operation Level

I did not find that rule written in the book but every example appears to support it. Considering that classes and subclasses in usual OOP are a special case of Knowledge Level built-into the language, this would lead to:

  • Abstraction never depends upon their Implementations

which is similar to the second part of the Dependency inversion principle by Robert C. Martin:

Abstractions should not depend upon details. Details should depend upon abstractions.

Since many analysis patterns in the Analysis Patterns book involve the Knowledge Level pattern, this single dependency rule already applies to many analysis patterns: Party Type Generalizations, Measurement, Observation, Protocol, Associated Observation, Measurement Protocol etc. The pattern Quantity can be seen as a specialized ValueObject (see Domain Driven Design above) hence should also not depend on any Entity nor Service.

Design Patterns

The book Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma et al. presents the classic design patterns. These patterns define participants which are named.  In the pattern participant ignorance principle I discussed the concepts of ignorant Vs. dedicated participants within a pattern, and their consequences for dependencies:

  • Ignorant pattern participants should never depend on dedicated participants
  • Pattern participant never depend on the “Client” participant
  • For each ConcreteX participant, the corresponding abstract X never depends on it (Abstraction never depends upon their Implementations)

In practice, this means:

  • In the Adapter pattern, the Adaptee should not depend upon the Adapter, and the Target should depend upon nothing
  • In the Facade pattern, the sub systems should not depend upon the Facade
  • In the Iterator pattern, the Aggregate should not depend upon the Iterator; however every Java collection is a counter example as they contain their own ConcreteIterator.
  • In creational patterns (Abstract Factory, Prototype, Builder etc.), the Product and ConcreteProduct should not depend on the dedicated participant that does the allocation (the Factory etc.)
  • And so on for other patterns, some of which being already discussed in the pattern participant ignorance principle.

In short, if we look at the design patterns as a set of types with inheritance/implementation, invocation/delegation and creation relationships between them, the dependencies should not flow in the reverse direction of the relationships; in other words, using UML arrows, the dependencies should only be allowed in the direction of the arrows.

Addiction to sugar is a kind of dependency
Addiction to sugar is a kind of dependency

Patterns of Enterprise Architecture

In the book Patterns of Enterprise Application Architecture by Martin Fowler, the Separated Interface Pattern proposes a way to manage dependencies by defining the interface between packages in a separate package from its implementation. This is similar to the Dependency inversion principle, as discussed here, which states:

A. High-level modules should not depend upon low-level modules. Both should depend upon abstractions.

By the way this is also very similar to the recommendation in Domain Driven Design:

Abstract Core module never depends on its specialized or implementation modules.

Finally, in the spirit of UML stereotypes that we sometimes put on packages to express their intent:

  • Utils never depends on anything but other Utils

What for?

If we manage to make every use of the above pattern explicit in the source code, for instance using Java annotations or simply Javadoc tags, then it becomes possible for a tool to deduce dependencies constraints and automatically enforce them.

Imagine, just add @pattern ValueObject in your Javadoc comment, and voila! A tool is now able to deduce that if you happen to import anything but basic java.* you must be warned.

Of course the fine tuning of the default behavior can take some time (do we accept that ValueObjects may depend upon low level utils like StringUtils? Probably yes), but the result will at least be stable regardless of the refactorings.

Given the existing variety of patterns over there, I am confident that just any class or interface within a project can be declared as being a participant in at least one pattern, and have therefore its dependency constraints deduced at the same time.

Read More

After Technical Debt, technical Options

cyrille

In finance, options are powerful tools for traders, and many design practices including design patterns can be seen as options. Options can -perhaps- yield a great benefit for a certain and immediate cost. If this cost is cheap enough it can be quite attractive.

An option to buy a stock is a right to buy the stock at a predefined price (let’s say EUR25) before a given date in the future. Until this date, you can freely exercise the option (i.e. buy the stock at EUR25), or not (do nothing). Of course if it happens that the stock price is greater than EUR25 (let’s say the stock is now EUR35), if you exercise the option you earn money (EUR10), and for that reason the option is worth something (EUR10 here, or even more if we expect the price to go even higher in the future).

In finance, the price of the option is directly linked to the expected benefit it can bring, even though this benefit is not certain. To be more precise, the more the stock price moves up and down, the bigger the option value: this instability of the price is called volatility, and you can think of it like a standard deviation in statistics. Options traders actually trade volatility: the higher the volatility (i.e. the less stable the prices), the more valuable the option.

Many design practices can be seen as options, and some are indeed cheap, almost free options. A free option is something that a reasonable investor cannot refuse to get, since it costs nothing while it can perhaps yields a benefit: this is actually an uncertain version of a “free lunch”. A design practice that would cost nothing is hard to resist, since it yields potential return in exchange for nothing.

In the spirit of the Technical Debt, we will focus on the cost of maintaining the code over time. Any change to the code must be weighted in accordance to the burden it will bring compared to its benefit.

Plenty of options here
Plenty of options here

Technical options in the code

In practice, everything that improves the flexibility of the software is de facto like an option: it creates additional opportunities that we can use or not in the future. Typically, it can enable the designer to change his mind at a later stage at no cost, or to defer a design decision.

The design principle to code against abstractions rather than implementation comes immediately to mind: coding against interfaces is probably the number one technical option. Therefore the question is: given this piece of concrete code, should I introduce an interface and reference the code through the interface, or should I leave it like that? Just like in finance, the answer is: it depends on the upfront cost of the option (adding the extra interface) compared to the anticipated stability of the feature the code is doing (is it likely to change in the future?).

In this case, the upfront cost depends on the tools (with modern IDEs with wizards and refactoring capabilities it is easier to add or even extract an interface automatically), and perhaps on what you like to do or not (if you hate having too many classes then adding an interface will cost you more, subjectively). Some commercial source control systems that are not convenient with widespread refactoring also make the cost of future changes much higher, and as such they encourage building flexibility upfront.

Going further, many design patterns can be considered as options to improve flexibility for a minimal cost. Out of the GoF patterns, the patterns Abstract Factory, Strategy, State, Visitor, Iterator, Bridge and Builder directly plan for easy change of behaviour or extension in the future (typically through additional interfaces as well), and thus are simple yet efficient ways to buy technical options.

Various elements combine together for bigger value added
Various elements combine together for bigger value added

Technical options Vs. YAGNI

Knowing the future is hard, and YAGNI is there to remind us that. If there are practices that cost a little to introduce and that can bring benefits later, and if we are so poor at anticipating change, why not just introduce them later? This is right, in most cases YAGNI applies, unless we can anticipate a need. The funny thing is that while the introduction of interfaces (directly or through the patterns mentioned before) creates opportunities to vary the implementations, it also creates opportunities for other opportunities. The new interface enables the addition of non-intrusive structural patterns that revolve around an existing interface, such as Decorator, Composite, Proxy, Adapter, Null Object, and every combination of them as suggested by the Interpreter pattern. (This introduction of such patterns, by the way, is precisely what most IoC containers, application servers and AOP frameworks do to add functionalities). How these patterns combine together under the common “umbrella” interface is kind of fascinating to me, and it always reminds me about the Network Effect: the more we use an interface, with implementations or opportunistic patterns, the more useful it becomes. I believe that a net of patterns like that can become so useful that it becomes irresistible. You will gonna need it.

Of course there is nothing really new in all that, nothing more than a new metaphor to help explain to managers. Even if they do not know what an option is, I am sure they can understand the concept very quickly.

I have been pushing this approach that I called “Think options, not solutions” in my former team to emphasize that we can build the system even if some of its business behaviours are not completely known yet, as long as we can define their contours and start with an arbitrary solution. The goal is that we are confident we can change it later with no pain. This was justified from a business point of view because we did not know what the market was expecting, for the product (an interest rate swaps multilateral trading facility) was too novel. In that environment, change was definitely not something we feared, rather the opposite: it was stimulating.

Just like traders speculate on volatility using options, developers can speculate on the stability of the business features using (or not) design practices that increase flexibility. The little extra effort to introduce these practices must be weighted by the expectation that it will be useful. And just for the fun, anyone interested in computing the value of a technical option using the Black-Scholes pricing model?

Read More

The pattern participant ignorance principle

cyrille

Not all participants in a pattern are equal, some are aware there is a pattern, while other participants do not, cannot and must not. Let us have a look at that principle and its consequences for coupling and dependencies management.

Some patterns are intrusive, which means that adding them to an existing code base requires changing the code base. Fortunately many patterns are not intrusive, and given an existing code base, introducing such patterns only requires the addition of specific code, more precisely the addition of the pattern participants that are dedicated to the pattern.

However in the patterns books, each pattern defines many participants, many of which being actually already in the existing code, and in the case of non-intrusive patterns they do not have to be changed to play their role in the patterns: these participants are therefore ignorant participants, they participate without knowing. Participants that are not ignorant are called dedicated participants.

ignorance

If we were now to remove the pattern occurrence, we would just delete the dedicated participants and leave the ignorant participants intact. This is what I call the Pattern Participants Ignorance principle (PPIP):

In a pattern occurrence, every participant that would remain after we completely removed the pattern occurrence from the code must NOT be aware of the pattern, and is called an ignorant participant; adding or removing the pattern to the code must not change a single line of the source code of these participants. Participants that are not ignorant are called dedicated participants

If the source code is in source control, adding or removing a pattern should never involve any commit to the source of any ignorant participant. If the source code is not available, we could still introduce the patterns. In simpler words, ignorant participants are passive participants, whereas dedicated participants only exist to support the pattern.

By definition, every ignorant participant should not be aware of the pattern and this implies in particular that they should not be aware of any dedicated participant.

This works well for many patterns, where every participant is ignorant but a few:

  • Decorator: the participants Decorator and Concrete Decorator are the only dedicated participants
  • Proxy: the participant Proxy is the only dedicated participant (more code can actually be also dedicated, for instance to realize a remote proxy)
  • Adapter: the participant Adapter is the only dedicated participant
  • Composite: the participant Composite is the only dedicated participant
  • Null Object: the Null Object itself is the only dedicated participant (of course)

There are patterns that do require modifications to the existing source code, they are intrusive. Intrusive patterns violate Participant Ignorance since ignorant participants must know a little about the pattern because they are modified to support it.

Some patterns are almost non-intrusive except for the calling code (Client participant) that need to be changed a bit:

  • Abstract Factory: the Abstract Factory and Concrete Factory are the only dedicated participants, but the calling code must call the new factory; however the Product and Concrete Product participant are ignorant
  • Builder: the Builder and Concrete Builder are the only dedicated participants, but the calling code must call the new builder; however the Product and Concrete Product participant are ignorant
forbidden
Forbidden to go (really)

Other patterns are much more intrusive. For example, the Visitor pattern, when applied with double-dispatch, requires the addition of accept(Visitor) methods in each element to be visited, which modifies participants that are indeed ignorant and that should not be touched in an ideal world. Visitors using a switch-case approach or a reflection-based approach, or coded in a language that support multi-methods are not intrusive, and “purer”.

The Observer pattern also requires the Subject participant to know about its Observers, with the addition of a collection of Observers, attach() and detach() methods, and the notification logic, whereas the Subject would remain in the absence of the Observer pattern: it is an ignorant participant, and the typical Observer pattern violates  Participant Ignorance.

To be fair, there are patterns for which our perspective is less relevant, where almost every participant appears to be deeply involved (very dedicated) to the pattern, and if we removed the pattern we would change everything. In the State pattern, each State and ConcreteState usually deals with the Context, which knows them as well.

In the Mediator pattern every colleague knows the Mediator, and each Concrete Mediator deals with each Colleague or Concrete Colleague. Introducing a Strategy is also really intrusive, ironically for the purpose of being able to add new behaviours in a non-intrusive way during software evolution…

Participant Ignorance helps to think clearly about the forces in action in a pattern, which usually discourage coupling (the more ignorant participants the better). If like I do you document the use of patterns in your code by using Javadoc tags (or Java annotations), then the principle states that these tags or annotations must only be located in the code of the dedicated participants to leave ignorant participant untouched when refactoring to or from patterns. More interestingly, the PPIP enable to derive allowed and forbidden dependencies between the participants of a pattern occurrence: ignorant participants must never depend on dedicated participants.

Read More

Systematic combination of subpatterns generates design patterns

cyrille 4 Comments

Composite patterns, such a the Bureaucracy pattern, are patterns built by the composition of other “smaller” patterns. However even usual design patterns can be considered composite patterns made of smaller subpatterns.

The goal is therefore to find out which are the main subpatterns that enable to reconstruct as many design patterns as possible.

The subpatterns

From an early analysis I believe that four subpatterns form the foundation of many GoF design patterns:

  1. Type Hierarchy and its variants (interface, abstract class, non final class…)
  2. Type Set, a set of types with no particular relationship between them
  3. Delegation and its variants (one-to-one, one-to-many)
  4. Allocation (creation of instances) and its variants

The experiment

To validate this proposal, I have coded a small experiment to generate every combination of two subpatterns out of (1) and (2) plus one relationship between them out of (3) and (4), or every combination of one subpattern out of (1) and (2) plus a relationship of (3) and (4) to either itself, or in the case of (1), between a super type and a subtype or the other way round.

For each generated combination I have automatically generated its corresponding class diagram, and grouped them together into one SVG document, converted into PDF using Inkscape. The resulting picture is available here, and you can see a partial preview below:

combinations_thumb

Results

The interesting finding is that this simple combinatorial method re-discovers 11 well-known design patterns (in the reading order of the picture):

  • Template Method (delegation to self, expecting a subtype to provide the service)
  • Composite (Considering a one-to-many delegation) or Decorator / Proxy (one-to-one delegation)
  • –
  • Prototype (creates instance of this type)
  • –
  • –
  • Bridge (one Abstraction hierarchy delegates to another Implementor hierarchy)
  • Abstract Factory (one Factory hierarchy creates instances of another (or several others) Products hierarchy)
  • … skip 4 next …
  • Strategy or State or Builder (some client delegates to some hierarchy)
  • …

Note that there are several cases where we could see the Factory Method, Adapter or Facade patterns. This would lead to some 13 design patterns out of the 23 in the GoF book.

Discussion

It is obvious anyway that the four subpatterns used here are not enough to rebuild the full set of the 23 design patterns. The Command pattern is unique in its use of a separate invoker, and the Chain of Responsibility is clearly about how handlers self-assign a given task without any centralized management. In these two examples however, the unique characteristics is not in their structure but rather in their dynamics. Sequence diagrams would be much more relevant to convey that kind of information.

The full test runs in one go in a fraction of second. The source code is available here: patternitygraphic_src, the code of the experiment is in CombinationTest.

EDIT: the plain SVG file that was generated is also available here for download:

Read More

New: Java API for UML diagrams

cyrille 3 Comments

As part of the Patternity effort, I spent some time creating a simple Java API to generate UML diagrams programmatically from Java, in SVG format.

This small API called for now Patternity Graphic is working and available here: patternitygraphic_src as a source Zip (alpha release of course).

It can render small class diagrams with hierarchic, flow and radial layouts, and arbitrary sequence diagrams with unlimited nesting of method calls and embedded comments. My focus was to support the subset of diagram elements and capabilities required to display patterns occurrences.

Here is a sample sequence diagram with a nice call stack:

sequence1

And here is a class diagram for a simple dummy hierarchy:

hierarchy

Boxes and links have various styles, defined in a template.svg template file, here is a random display of the boxes styles:

boxstyles

Apart from unit tests there is no documentation. If you are interested to reuse that please contact me for help. The Zip was exported in Eclipse with the project Export… function, and the project requires Commons Collections. Svg diagrams can be converted into images using Batik.

Other projects you might consider to generate UML diagrams: UMLGraph, modsl, umlspeed, jsigner, MetaUML, and of course GraphViz.

Any feedback appreciated!

Read More