Thursday, 1 February 2018

Dynamic configuration management in microservice architecture with Spring Cloud

In the scary world of monolithic applications we succeeded in working out numerous design patterns and good practices. When we move towards distributed systems, we should apply them carefully, bearing in mind that in microservice infrastructure we need to deploy, scale, and reconfigure our systems much quicker. In this article I will focus on configuration issues, emphasizing why keeping properties/yml files with your codebase is good, and when it might not suffice. I will also give you an overview of how you can improve your configuration management with Spring Cloud, and how easy it is to reconfigure your application live without rebuilding or rebooting it.

Configuration files


It seems to be obvious now to keep your configuration outside of the code, so that anyone can update it without the struggle to find a proper line to be modified. However I find it not so evident for people to build applications independently from the environments they are running on. Anyway, making your build environment-dependent is not a good idea, as you won’t know what environment is a particular package supposed to be used for (unless you provide some crazy naming strategy for your artifacts, but don’t tell me about them, I’m frightened enough while thinking of it). It should be a natural thing to create profiles related to your environments and split the config among profile-dependent .properties files (or separate them within the .yml file), building one profile-agnostic deployment package, passing a proper profile/config file to your application while running it. If you do so, you will see a light in the tunnel. But the light is dimmed as there is still a couple of things you should be aware of.

Please note that environment is not the only thing that may define the profile. As an example: in projects that I was working on, we had to deploy the same application for three different countries, where each country had its own configuration. Then we had 9 combinations of profiles: dev, test, prod for each of the three countries.

First of all, having config files bounded with the deployment package, you need to rebuild and redeploy it each time some property changes. It is a horror when your configuration changes more frequently than the code. Although you can always keep some of your configuration outside of the jar file, it will usually require rebooting your app after updating your properties there.

Moreover, having a distributed system, where scalability is one of its key features you need to be able to react to configuration changes quickly, and apply them to all instances that are currently running. Redeploying them one by one can be a tough task if it is not properly automated.

It might also be a case that developers have restricted access to credentials to production databases, external services, etc. In one of monolithic applications that I was working on, the production deployment required client’s administrator to set particular properties in an external config file before starting the application. No one had any trace about what was changed there, when it happened and by whom. I hope you don’t need any more arguments to see that this is a very bad idea to do so.

You can see now how complicated and confusing configuration issues might be. Microservices are by definition small independent applications realising some business logic within a bounded context. Each microservice, before being deployed, must be a bundled unit that can be run on any desired environment: dev, test, prod, etc. without the need of rebuilding it for each of them. We should be able to scale and reconfigure them at will, and keep all sensitive settings secured if needed. This is the moment when Spring Cloud Config comes to the rescue. But before that, lets have a look at our simple infrastructure example, that you can find on my github account.

Example Overview


We have two simple microservices: Customer Manager and Product Manager, that realise some business logic regarding Customers and Products respectively. The access to both of our applications is available from the WWW only through the Gateway, which is an implementation of API Gateway pattern with the help of Zuul Router. Each application registers itself (we are using client-side service discovery) in Eureka Discovery Server, which maintains service registry. In our case all microservices communicate with each other via HTTP protocol, and all of them are of course SpringBoot applications.


When we set up our environment we can take a look at Discovery Server dashboard, where we can see that all three microservices successfully registered themselves in Eureka. This in turn means that they will be able to communicate with each other without preconfiguring their exact URLs, but using application names to find them out from Eureka Server instead.


Spring Cloud Config


Having our example in place, and bearing in mind all configuration issues that we've discussed, let's get straight to the solution. Spring Cloud Config is a tool that provides both server and client-side support for managing configuration in distributed environment in a centralized manner.

EnvironmentRepository


Config server can be treated as a remote property source. It can be backed by numerous storage providers, like Git (which is the default setting), SVN, or distributed file system. You can even have more than one storage at the same time. With Git we can easily manage configuration files for any application and any environment. Moreover, we gain access control, which means we can decide who can modify particular files (for example restricting access to production properties). We also have full traceability - we know who changed what, and when, and we can easily revert those changes.

The way we want to store our data is defined by EnvironmentRepository interface:
public interface EnvironmentRepository {
  Environment findOne(String application, String profile, String label);
}

It tells us a lot about how things are organized on both config server and backing service. You can see that in order to get a particular environment, we need to locate it with following variables:
  • application - application name
  • profile - application profile
  • label - additional parameter used by the backing service (branch name, tag, etc. in case of Git repository)
In order to get proper configuration clients can use following endpoints:

  • /{application}-{profiles}.[properties|yml|yaml|json]
  • /{label}/{application}-{profiles}.[properties|yml|yaml|json]
  • /{application}/{profile}[/{label}]

Embedded Config Server


Benefiting from spring's convention over configuration approach, we can set up our Config Server in just a few steps. First of all, you need to declare following dependency:

  org.springframework.cloud
  spring-cloud-config-server


and mark config class with @EnableConfigServer annotation:
@SpringBootApplication
@EnableConfigServer
public class ConfigServerApplication {

  public static void main(String[] args) {
    SpringApplication.run(ConfigServerApplication.class, args);
  }
}

We will also need to define some properties inside our application.yml file, like below:
spring:
  application:
    name: config-server
  cloud:
    config:
      server:
        git:
          uri: https://github.com/bslota/config-repository.git
          clone-on-start: true
          search-paths: 'config/{application}'

server:
  port: 8888

You can see here that we set port number to 8888, which is a conventional and recommended setting. We also told our application to look for config files in a github repository, and clone it right after the startup. Cloning might take a while increasing the booting time, but as soon as the server is up it will be able to serve configuration quickly from the very first request. There is also a search-paths property specified, which tells that config files for each application will be placed in proper folders under the config/ directory. While configuring Git as backing service, you can use {application}, {profile} and {label} placeholders, so that it fits your needs. Here we have a very basic configuration, where properties for all applications are being held in one common repository, but you could easily apply one repo per profile or one repo per application strategy by using {profile} and {application} placeholders in spring.cloud.config.server.git.uri property respectively. More details you will find in documentation.

In order to keep  the Config Server consistent with our existing infrastructure, we will register it in Eureka Discovery Server. Now our system looks like this:




And here is the shot from Eureka Server dashboard:


Config Server is now the central part of our infrastructure that is supposed to both store and serve configuration for all other microservices. Thus, you should always scale it properly, so that your system stays resilient.

Spring Cloud Config Client


Okay, we have our Git repository in place, and we have our Config Server up and running. It's high time we start using it. Before I show you how to connect our applications with the server, I need to tell you a few words about so-called bootstrap context. In a big short, a bootstrap context in Spring Cloud applications is a parent for main application context and it is used for the purpose of loading and decrypting (if needed) properties from external sources. It is defined in bootstrap.yml file unless otherwise specified. What you should put in this file is information about how to locate config server, and properties that are necessary to properly identify environment there. Do you remember application, profile, and label placeholders? This is exactly what we need to pass to the server, and we can do it with following properties:
  • spring.cloud.config.name - by default it is equal to spring.application.name property value
  • spring.cloud.config.profile - by default equal to comma separated list of currently active profiles
  • spring.cloud.config.label - its default value is valuated on the server side and is equal to master if Git is used as backing service
Having such well assumed defaults, all we need to set in bootstrap.yml is spring.application.name property.

In order to connect to the config server, we need to declare following dependency:

  org.springframework.cloud
  spring-cloud-starter-config

As soon as we have it, we need to choose one of two available bootstrap strategies. First one (the default one) is config first bootstrap. It assumes that you define config server URI  explicitly in bootstrap.yml file with spring.cloud.config.uri property. In this setup, bootstrap.yml file will have following content:
spring:
  application:
    name: # microservice name
  cloud:
    config:
      uri: http://localhost:8888

The drawback of this strategy is that we cannot benefit from Config Server being registered in service discovery as we are defining its URI explicitly (this problem won't apply if we are using server-side service discovery). We can get rid of this problem by using discovery first bootstrap strategy. In this approach, our application will resolve config server's URI by its name with the help of service discovery. Of course this option has its costs as well. In order to get the configuration we need an extra network roundtrip as we need to communicate with discovery server. Now our bootstrap.yml file looks like this:
spring:
  application:
    name: # microservice name
  cloud:
    config:
      discovery:
        service-id: config-server

eureka:
  client:
    service-url:
      default-zone: http://localhost:8761/eureka/

Now we can easily move the configuration of our services to our github repository.

Manual refresh

Thanks to the actuator endpoints, we can easily trigger numerous actions on each of our applications. Sending POST request to the /refresh endpoint we can tell the service to reload the bootstrap context (updating the environment both from remote and local property sources), rebinding @ConfigurationProperties and log levels, and refreshing all beans annotated with @RefreshScope annotation.

To give some concrete examples, let's take a look at Customer Manager service, that has the following property set in its application.yml file in remote property source:
"premium-email-suffix": "yahoo.com"

and a service class annotated with @RefreshScope annotation, that depends on this property's value.
@Service
@RefreshScope
class CustomerService {

  private final CustomerRepository customerRepository;
  private final String premiumEmailSuffix;

  CustomerService(CustomerRepository customerRepository,
                  @Value("${premium-email-suffix}") String premiumEmailSuffix) {
    this.customerRepository = customerRepository;
    this.premiumEmailSuffix = premiumEmailSuffix;
  }

  List<Customer> findAll() {
     return customerRepository.findAll();
  }

  List<Customer> findAllPremium() {
    return findAll()
             .stream()
             .filter(it -> it.getEmail().endsWith(premiumEmailSuffix)).collect(toList());
  }

  // ... other methods   
}

The service is used by CustomerController, so we can easily check if our change, that we will apply in a minute, works as expected.
@RestController
@RequestMapping("/customers")
public class CustomerController {

  private final CustomerService customerService;

  public CustomerController(CustomerService customerService) {
    this.customerService = customerService;
  }

  @GetMapping
  public List<Customer> customers() {
     return customerService.findAll();
  }

  @GetMapping(params = "premium")
  public List<Customer> premiumCustomers() {
     return customerService.findAllPremium();
  }

  // ... other methods
}

Now if we send the following request:
curl http://localhost:8085/customers?premium

we get this response:
[
  {
    "id": 2,
    "name": "Steve Harris",
    "email": "steve@yahoo.com"
  }
]

Let's make an update to premium-email-suffix property so that its value is now equal to gmail.com, commit it, and push into remote repository. As we had mentioned before, in order to make it visible by our application, we need to send the following request:
curl -X POST http://localhost:8085/refresh

And we can see the updated properties straight away in the response body
["config.client.version","premium-email-suffix"]

To be 100% sure that the manual update worked, let's try to get all premium customers again hoping that this time their emails will end with gmail.com
[
  {
    "id": 1,
    "name": "Bruce Dickinson",
    "email": "bruce.dickinson@gmail.com"
  }
]

Note that similar behaviour would be observed with @ConfigurationProperties components.

You see, it is easy, but if you have several services scaled up to a couple of instances it can make the manual refreshment a terrible experience. You need to find a proper service url and make sure that you performed the refresh to all instances. We can surely agree that it is not something that we would eagerly do. Fortunately, we can automate this process. Read on.


Dynamic changes propagation

The desired scenario would be that all services, whose properties get updated within a push into a remote repository, get notified and refreshed automatically. Thus, first of all, we need to find a way to propagate configuration changes to proper services. To solve this problem we will use Spring Cloud Bus, which was built to propagate management instructions. It makes use of both actuator (by adding new management endpoints) and Spring Cloud Stream (by enabling communitaction with AMQP message brokers). In our case, we will use Apache Kafka as a message broker, and now the desired architecture would look like this:



In order to enable Spring Cloud Bus and connect to Kafka broker, Gateway, Customer Manager, and Product Manager services need to be altered with following dependency (don't worry, this one has transitive dependency on spring-cloud-bus):

  org.springframework.cloud
  spring-cloud-starter-bus-kafka


and configure Zookeeper nodes and Kafka binders (unless you are fine with defaults)
spring:
  cloud:
    stream:
      kafka:
        binder:
          zkNodes: "localhost:2181"
          brokers: "localhost:9092"

Right after the startup, our services will connect to springCloudBus topic.

Now our applications are ready to listen and react to refresh requests coming from message broker and determine if a particular event is dedicated to them or not.

In order enable Config Server to publish this kind of events (RefreshRemoteApplicationEvent to be specific), we need to declare two dependencies:

  org.springframework.cloud
  spring-cloud-config-monitor


  org.springframework.cloud
  spring-cloud-starter-bus-kafka


and configure Zookeeper nodes and Kafka binders in the same way as we did in the other services:
spring:
  cloud:
    stream:
      kafka:
        binder:
          zkNodes: "localhost:2181"
          brokers: "localhost:9092"

You already know what spring-cloud-starter-bus-kafka is used for. The second dependency, though, spring-cloud-config-monitor adds something more - the /monitor endpoint that accepts POST requests with information about what service needs to be refreshed. If this request gets to Config Server it publishes RefreshRemoteApplicationEvent to a message broker. Then the target application consumes it and performs refreshment. Yep, you guessed - we can create webhooks that call this endpoint whenever anything changes in Git! Now here we have a few options. The first one is a simple form based request:

curl -X POST localhost:8888/monitor -d 'path=customer-manager'

The remaining options are Git provider specific. In Github for exapmle, the request might be of following form:
curl -X POST localhost:8888/monitor 
  | -H "X-Github-Event: push"
  | -H "Content-Type: application/json"
  | -d '{"commits": [{"modified": ["customer-manager.yml"] }]}'

If you are interested in details about how to build Github Push Webhooks, please visit this page.


Summary


In this article you could get an overview of how complex configuration issues may be. I gave you an explanation of why keeping configuration in external files is good, and when in might not be enough. You could see an evolving microservice system which we improved step by step by adding advanced configuration management with the use of spring-cloud-config and Git as a backing service. You could also see, that a simple setup with manual refresh ability is fairly easy to provide, but in a much bigger architecture where your services are scaled up to a couple of nodes you simply won't be able to control it. Then, a good solution would be to use a message broker like Apache Kafka and spring-cloud-bus with a proper broker abstraction (spring-cloud-stream) on the client side, and spring-cloud-config-monitor on the server side. If you add a proper Webhook to your Git platform, everything will work automatically as a commit to a proper repository/branch gets pushed.

Further considerations


When you read articles like this, you should always keep in mind that software development is about trade-off-s before you apply it in your infrastructure. You need to know that keeping configuration in a central place is helpful when it comes to maintenance, traceability, and all the stuff that we discussed at the very beginning, but it complicates the CI and CD processes as you have second place where you keep things connected with deployed application. Another thing is that you need to plan how to organize your Git repository and decide if you want to have a repository per application, or maybe one common repository with application-dependent folders and branches that will apply to profiles - there are lots of strategies and it is up to you to pick the one that will suit you best. If you need some more information about spring-cloud-config I recommend visiting this page. Cheers!

Monday, 25 September 2017

Convenience Factory Methods for Collections in Java 9

Java 9 doesn't bring as dramatic changes to the way of coding as its predecessor did but surely we will have some fancy features, one of which I would like to present to you in this article. When I started coding in Groovy about 5 years ago, I was amazed by the collections support there. Especially when it came to initialize an immutable list, I could do it in one simple line:

def band = ["Bruce", "Steve", "Adrian", "Dave", "Janic", "Nicko"].asImmutable()

This year I also started coding in scala, and here we have some fancy tricks like:
val band = "Bruce" :: "Steve" :: "Adrian" :: "Dave" :: "Janick" :: "Nicko" :: Nil

or simply

val band = List("Bruce", "Steve", "Adrian", "Dave", "Janick", "Nicko")

At the same time Java seemed to be torturing me with an add statement list:

List<String> band = new ArrayList<>();
band.add("Bruce");
band.add("Steve");
band.add("Adrian");
band.add("Janick");
band.add("Nicko");
band = Collections.unmodifiableList(band);

or so called "double-brace" initialization (which in fact isn't any special Java feature, but rather a workaround benefiting from anonymous classes and initialization blocks):

List<String> band = Collections.unmodifiableList(new ArrayList<>() {{
  add("Bruce");
  add("Steve");
  add("Adrian");
  add("Janick");
  add("Nicko");
}});

or using arrays API to convert array to an ArrayList

List<String> band = Collections
  .unmodifiableList(Arrays.asList("Bruce","Steve","Adrian", 
                                  "Dave", "Janick","Nicko"));

Lately I could also benefit from Stream API:

List<String> band = Collections
  .unmodifiableList(Stream.of("Bruce","Steve","Adrian", "Dave", "Janick","Nicko")
    .collect(toList()));


Two last options are the cutest ones, but why do I need to first create an array or a stream in order to create a list, and why can't I just use Collections API instead of Arrays API or Stream API? Well, I don't even want to recall the way we can create Sets or Maps in Java - thinking of it makes me wake up in sweat in the middle of the night.

So far, the most concise and elegant way of creating immutable collections was provided by Guava library and classes like ImmutableList, ImmutableMap, ImmutableSet, etc. Below is an example of what was desired to be available in Java out of the box:

List<String> band = ImmutableList.of("Bruce", "Steve", "Dave",
                                     "Adrian", "Janick", "Nicko");

Fortunately, authors of Java 9 implemented JEP 269: Convenience Factory Methods for Collections, which simply provides a set of static factory methods supporting creation of immutable collection instances. Saviours! Read on to get the overview of this feature.

Immutable collections


Before we dig into the topic you should know what immutable collections really are. So, they are collections that cannot be modified once they are created. What I mean by modified is that their state (references they hold, order in which those references are being kept, and the number of elements) will stay untouched. Please note that if an immutable collection holds mutable objects, their state won't be protected anyhow. Although immutable collections still implement List, Set or Map interfaces, methods modifying their contents throw UnsupportedOperationException. You will find sets of such methods in subsequent sections.

Implementation


Implementation can be devided into two main parts. Firstly, java.util package was enriched with package-private ImmutableCollections class, that contains classes providing the immutability feature. Secondly, instances of those classes are being created with the help of static factory methods in already existing interfaces, i.e. List, Set, and Map. In following sections you will find description of both sets of functionalities per each collection interface.

List


An immutable list has an abstract base class AbstractImmutableList<E> and four implementations:

  • List0<E>
  • List1<E>
  • List2<E>
  • ListN<E>

Each of these types correspond to the number of elements that is used to their creation. In java.util.List interface we have 12 static factory methods that use the above implementations to create immutable objects:

// creates empty immutable list
static <E> List<E> of()

// creates one-element immutable list
static <E> List<E> of(E e1)

// creates two-element immutable list
static <E> List<E> of(E e1, E e2)

...

// creates ten-element immutable list
static <E> List<E> of(E e1, E e2, E e3, E e4, E e5, E e6, E e7, E e8, E e9, E e10)

// creates N-element immutable list
static <E> List<E> of(E... elements)

Methods that throw UnsupportedOperationException:

boolean add(E e);
boolean addAll(Collection<? extends E> c);
boolean addAll(int index, Collection<? extends E> c);
void    clear();
boolean remove(Object o);
boolean removeAll(Collection<?> c);
boolean removeIf(Predicate<? super E> filter);
void    replaceAll(UnaryOperator<E> operator);
boolean retainAll(Collection<?> c);
void    sort(Comparator<? super E> c);

Apart from protecting the content of our list, we also get a validation, that prevents us from initiating a list with a null value. Trying to run the following piece of code, we will end up with NullPointerException:

// throws NullPointerException
List<String> band = List.of("Bruce","Steve","Adrian", "Dave", "Janick", null);

Now here is an example of how to properly create immutable list:

List<String> band = List.of("Bruce","Steve","Adrian", "Dave", "Janick","Nicko");

Set


An immutable set is implemented similarly to what we have seen with the List interface. It has an abstract base class AbstractImmutableSet<E> and four implementations:

  • Set0<E>
  • Set1<E>
  • Set2<E>
  • SetN<E>

that again correspond to the number of elements that is used to their creation. In java.util.Set interface we have 12 static factory methods:

// creates empty immutable set
static <E> Set<E> of()

// creates one-element immutable set
static <E> Set<E> of(E e1)

// creates two-element immutable set
static <E> Set<E> of(E e1, E e2)

...

// creates ten-element immutable set
static <E> Set<E> of(E e1, E e2, E e3, E e4, E e5, E e6, E e7, E e8, E e9, E e10)

// creates N-element immutable set
static <E> Set<E> of(E... elements)


Methods that throw UnsupportedOperationException:

boolean add(E e)
boolean addAll(Collection<? extends E> c)
void    clear()
boolean remove(Object o)
boolean removeAll(Collection<?> c)
boolean removeIf(Predicate<? super E> filter)
boolean retainAll(Collection<?> c)


Like with immutable lists, we cannot instantiate a Set with a null value:

// throws NullPointerException
Set<String> band = Set.of("Bruce","Steve","Adrian", "Dave", "Janick", null);


You should also know, that sets differ from lists in a way, that they cannot have duplicate values. With newly provided factory methods, we won't be able to initialize an immutable Set passing more than one object of the same value - we will get IllegalArgumentException:

// throws IllegalArgumentException
Set<String> guitarists = Set.of("Adrian", "Dave", "Janick", "Janick");

Now here is an example of how to properly create immutable set:

Set<String> band = Set.of("Bruce","Steve","Adrian", "Dave", "Janick","Nicko");

Map


Before we describe immutable map technical details we should start with the concept of an entry (java.util.Map.Entry interface) - an aggregate of key - value pair. From Java 9 we have yet another Entry's package private implementation - java.util.KeyValueHolder -  an immutable container, that prevents from instantiating an entry with key or value equal to null (throwing NullPointerException if done so).

In order to create an immutable entry, we can use following static factory method from java.util.Map interface:

static <K, V> Entry<K, V> entry(K k, V v)

Immutable map has an abstract base class AbstractImmutableMap<K, V> with three implementations:

  • Map0<K, V>
  • Map1<K, V>
  • MapN<K, V>

Again we have the following set of factory methods inside java.util.Map interface:

// creates an empty map
static <K, V> Map<K, V> of()

// creates one-element map
static <K, V> Map<K, V> of(K k1, V v1)

// creates two-element map
static <K, V> Map<K, V> of(K k1, V v1, K k2, V v2)

...

// creates ten-element map
static <K, V> Map<K, V> of(K k1, V v1, K k2, V v2, K k3, V v3, K k4, V v4,
                           K k5, V v5, K k6, V v6, K k7, V v7, K k8, V v8, 
                           K k9, V v9, K k10, V v10)

// creates N-element map
static <K, V> Map<K, V> ofEntries(Entry<? extends K, ? extends V>... entries)

You can see that it differs from List or Set factory methods from previous chapters. Using one of the of methods we can create immutable maps that contain up to 10 elements. If we want to have a bigger one, we need to use ofEntries method, accepting varargs of Entry. It shouldn't be any surprise, as we can use varargs for only one argument in a method, so we have no way of passing keys and values of different types this way.

Like with lists and sets, we have some methods that throw UnsupportedOperationException:

void clear()
V compute(K key, BiFunction<? super K,? super V,? extends V> rf)
V computeIfAbsent(K key, Function<? super K,? extends V> mf)
V computeIfPresent(K key, BiFunction<? super K,? super V,? extends V> rf)
V merge(K key, V value, BiFunction<? super V,? super V,? extends V> rf)
V put(K key, V value)
void putAll(Map<? extends K,? extends V> m)
V putIfAbsent(K key, V value)
V remove(Object key)
boolean remove(Object key, Object value)
V replace(K key, V value)
boolean replace(K key, V oldValue, V newValue)
void replaceAll(BiFunction<? super K,? super V,? extends V> f)

Regardless the way of creating immutable map (of or ofEntries method) we won't be able to instantiate it with key, value or whole entry equal to null. Below are the examples of code that will throw NullPointerException:

// throws NullPointerExcepton because of null key
Map<String, Long> age = Map.of(null, 59L, "Steve", 61L);

// throws NullPointerExcepton because of null value
Map<String, Long> age = Map.of("Bruce", null, "Steve", 61L);

// throws NullPointerExcepton because of null entry
Map<String, Long> age = Map.ofEntries(Map.entry("Bruce", 59L), null);

Similarily to immutable set, we cannot create a map with duplicate values. Trying to do so will end up with throwing IllegalArgumentException:

Map<String, Long> age = Map.of("Bruce", 59L, "Bruce", 59L);

Map<String, Long> age = Map.ofEntries(Map.entry("Bruce", 59L),
                                      Map.entry("Bruce", 59L));

And here are some examples of how can we properly create immutable maps in Java 9:

Map<String, Long> age = Map.of("Bruce", 59L, "Steve", 61L, "Dave", 60L,
                               "Adrian", 60L, "Janick", 60L, "Nicko", 65L);

Map<String, Long> age = Map.ofEntries(Map.entry("Bruce", 59L),
                                      Map.entry("Steve", 61L),
                                      Map.entry("Dave", 60L),
                                      Map.entry("Adrian", 60L),
                                      Map.entry("Janick", 60L),
                                      Map.entry("Nicko", 65L));


Conclusions


Since Java 9, creating immutable collections is very convenient. We have a set of static factory methods for each interface, that, apart from creating immutable objects, prevent from inserting nulls or duplicates (in Set and Map). We are warned about any problems at creation time, and we are secured from getting NullPointerExceptions while traversing collection elements. It is a tiny feature, but surely a useful one!


Monday, 18 September 2017

Following OOP principles - hermetization

Have you ever looked at the code that you had written couple of months ago and asked yourself: “who could leave such a mess here?” Have you ever been so lazy that you didn’t think of what accessors/mutators you need, simply hitting “Generate getters and setters” for your entity in IDE? Or maybe you have used lombok’s @Getter/@Setter annotations to get the same effect? Yep, just as I thought! Well, honestly, I did this way too many times, too. The problem is that it is not the only crime that we can commit in order to break some of the basic OOP principles. In this article we will focus on hermetization (information hiding) but other paradigms, like abstraction or encapsulation will be mentioned too, as they all are complementary to each other. In the following paragraphs I will give you an explanation of what is hermetization and how to provide one within Java. We will discuss common pitfalls and some good practices. You will also find here some philosophical considerations and my own judgements and opinions. I am very open for further discussions. I will walk you through an example of a very poorly written classes, which we will improve step by step, looking at hermetization from numerous perspectives. Enjoy!

Hermetization


Hermetization is about hiding information (implementation details) that shouldn’t be visible to clients of a class or a module. As long as we follow the encapsulation paradigm, we enclose information (state) and interface (behavior) in a class. Now an interface (API) is a declaration of how we can interact with a particular object. Should we care about how this interaction is implemented? Should we care about the data representation inside? Should we publish methods modifying internal state, that is supposed to be done automatically? Nope, not at all. We just want to send it a command, get the result, and forget. At the same time we want our internals to stay safe and untouched (intentionally or unintentionally) from the outside.

The less information we expose to the outer world, the less coupled modules we get. Thus, we gain a better separation of classes, which it turn means that we can easily:
  • manipulate the logic/internals inside a class not worrying that we will break our clients,
  • analyse, use, and test such classes, as we have a clear interface
  • reuse them, as independent classes might appear to be useful in some other contexts
  • keep object's state safe.

Example overview


In this article we will focus on a simple business case. We need to provide following things:

  • the ability to create both contact person and a customer,
  • each contact person can have an email address - we want to both present and modify this data,
  • a particular contact person might be assigned to more than one customer
  • each customer can have a name, and a list of contact people
  • we want to store a timestamp of the moment of customers' creation and activation
  • we also want to be able to easily activate a customer and verify if a customer is activated or not
  • we want to present customer's name, contact people, creation date and a flag of activation
  • name and contact person list can be modified in future, but creation date must be set only once during creation phase.

Here is a piece of extremely poorly written code:

public class ContactPerson {
  public long id;
  public String email;
}
public class Customer {
  public long id;
  public String name;
  public Date creationDate;
  public Date activationDate;
  public ArrayList<ContactPerson> contactPeople = new ArrayList<>();
}
public class CustomerService {

  public Customer createCustomer(String name,
                                 ArrayList<ContactPerson> contactPeople) {
    if(contactPeople == null) {
      throw new IllegalArgumentException("Contact people list cannot be null");
    }
    if (StringUtils.isEmpty(name)) {
      throw new IllegalArgumentException("Name cannot be empty");
    }
    final Customer customer = new Customer();
    customer.id = Sequence.nextValue();
    customer.creationDate = new Date();
    customer.name = name;
    customer.contactPeople = contactPeople;
    return customer;
  }

  public ContactPerson createContactPerson(String email) {
    final ContactPerson contactPerson = new ContactPerson();
    contactPerson.id = Sequence.nextValue();
    contactPerson.email = email;
    return contactPerson;
  }

  void activateCustomer(Customer customer) {
    customer.activationDate = new Date();
  }

  boolean isCustomerActive(Customer customer) {
    return customer.activationDate != null;
  }

  void addContactPerson(Customer customer, ContactPerson contactPerson) {
    customer.contactPeople.add(contactPerson);
  }

  void removeContactPerson(Customer customer, ContactPerson contactPerson) {
    customer.contactPeople.removeIf(it -> it.id == contactPerson.id);
  }
}

You can see that we have two model classes and a service fulfilling mentioned business requirements. Let's try to work on this example to make it a shiny one. You can find the final version of this example here.

Please note that we skip here all aspects of concurrency.


Access control, accessors, mutators


The first facility for information hiding that Java gives us is access control mechanism. We have a few options to choose:
  • Private
  • Package-private
  • Protected
  • Public
By default, we are granted a package-private scope which gives us a bit of hermetization out of the box. The language itself suggests that we should keep our modules (packages) independent, reusable - it should be our conscious decision to make a class, a method or a field a public one.

The code from above does what it is supposed to do, but with this approach you face following problems:
  • you cannot change the data representation without modifying client's code - the information about how you store data becomes a part of your API,
  • you cannot perform any extra actions  (like validation) when field is being accessed/modified.
It means we have no hermetization at all. The easiest way of dealing with it is to restrict all instance members' visibility to private scope and define their accessors (getters) and mutators (setters) like below:

public class ContactPerson {
  private long id;
  private String email;

  public long getId() {
    return id;
  }

  public void setId(long id) {
    this.id = id;
  }

  public String getEmail() {
    return email;
  }

  public void setEmail(String email) {
    this.email = email;
  }

  @Override
  public String toString() {
    return "ContactPerson{" +
      "id=" + id +
      ", email='" + email + '\'' +
      '}';
  }
}

public class Customer {
  private long id;
  private String name;
  private Date creationDate;
  private Date activationDate;
  private ArrayList<ContactPerson> contactPeople = new ArrayList<>();

  public long getId() {
    return id;
  }

  public void setId(long id) {
    this.id = id;
  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }

  public Date getCreationDate() {
    return creationDate;
  }

  public void setCreationDate(Date creationDate) {
    this.creationDate = creationDate;
  }

  public Date getActivationDate() {
    return activationDate;
  }

  public void setActivationDate(Date activationDate) {
    this.activationDate = activationDate;
  }

  public ArrayList<ContactPerson> getContactPeople() {
    return contactPeople;
  }

  public void setContactPeople(ArrayList<ContactPerson> contactPeople) {
    this.contactPeople = contactPeople;
  }

  @Override
  public String toString() {
    return "Customer{" +
      "id=" + id +
      ", name='" + name + '\'' +
      ", creationDate=" + creationDate +
      ", activationDate=" + activationDate +
      ", contactPeople=" + contactPeople +
      '}';
  }
}
And now our client's code need to be refactored, too:
public class CustomerService {

  public Customer createCustomer(String name,
                                 ArrayList<ContactPerson> contactPeople) {
    if(contactPeople == null) {
      throw new IllegalArgumentException("Contact people list cannot be null");
    }
    if (StringUtils.isEmpty(name)) {
      throw new IllegalArgumentException("Name cannot be empty");
    }
    final Customer customer = new Customer();
    customer.setId(Sequence.nextValue());
    customer.setCreationDate(new Date());
    customer.setName(name);
    customer.setContactPeople(contactPeople);
    return customer;
  }

  public ContactPerson createContactPerson(String email) {
    final ContactPerson contactPerson = new ContactPerson();
    contactPerson.setId(Sequence.nextValue());
    contactPerson.setEmail(email);
    return contactPerson;
  }

  public void activateCustomer(Customer customer) {
    customer.setActivationDate(new Date());
  }

  public boolean isCustomerActive(Customer customer) {
    return customer.getActivationDate() != null;
  }

  public void addContactPerson(Customer customer, ContactPerson contactPerson) {
    customer.getContactPeople().add(contactPerson);
  }

  public void removeContactPerson(Customer customer, long contactPersonId) {
    customer.getContactPeople().removeIf(it -> it.getId() == contactPersonId);
  }
}

You may think now that we are done - our fields are private, accessors and mutators cover implementation details, right? Well, it is a common mistake that we treat objects as data containers. We start with defining a set of fields, and afterwards we declare corresponding getters and setters to each and every field in a class. Then we put the whole logic into some service class making our entity a dumb data structure. As long as a class can manage the values of its fields (e.g. there is no need to call a repository or some external system) it should be done inside this class. In other words, every class should encapsulate both instance fields and business methods, which are nothing more than an abstraction of a real-life domain object, hermetizing implementation details as much as possible. You see - OOP is about composing all paradigms together. Now let's try to answer a few questions, bearing in mind previously defined example's business requirements:

ContactPerson:
  1. Do I really need getters to all fields? Well, I suppose yes - we might want to present customers' contact people information. Okay then, let's leave the getters.
  2. Is ID something that I should set? Do I even need what value should I give it? No. It is something that shouldn't bother us. It should be generated automatically, e.g. by Hibernate. It means that having a setId method we break hermetization! Let's remove this mutator and put its logic to a constructor. For the simplicity of an example - we put a static sequence generator there.
Customer:
  1. Do I really need getters to all fields? Nope. Like we said in the example description, we just want to get information whether a customer is active or not. Exposing getActivationDate method forces our clients to put some logic around activationDate value. It smells badly to me. A customer is able to decide about its activation status using its own fields' values (one field actually). It suggests that we hermetize activation details inside an entity. We simply move isCustomerActive method logic into isActive method inside Customer class, like it is depicted below.
  2. Is ID something I should set? You should know the answer now - it is the same situation like with ContactPerson.
  3. Should I set creationDate in client's code? Well, as the name implies it is a timestamp of object creation and shouldn't be modifiable at any other time. Thus, giving a setter we create a threat that someone will update this value in the future or set something really strange. It is up to entity to decide what time it should set. Let's move it to the constructor, then, and forget about this mutator.
  4. Do I need to set activationDate in client's code? Well, who told you (the client) that we store activation date? Ha! We don't care about data representation again, we just want to be able to activate a customer. What we should do is to remove setActivationDate from the service and create activateCustomer method inside an entity instead.
  5. Do I really need methods that add or remove contact people from my customer in a service class? Well again, collection belongs to a Customer entity, and letting some third parties modify that collection is a crime. Let's move these methods to the entity, too.
  6. Should I keep validations in service? In this case we can validate params without calling external services or resources, so the answer is no. Every condition that must be fulfilled while creating or setting a field should be hermetized inside the entity (in constructors and/or mutators), so that clients don't need to worry about performing such checks. Thus, we will have consistent conditions across all clients. Passing wrong parameters will simply cause an exception. 
Uhh, finally... that was tough. Let's see how our code looks now:

public class ContactPerson {
  private long id;
  private String email;

  public ContactPerson() {
    this.id = Sequence.nextValue();
  }

  public long getId() {
    return id;
  }

  public String getEmail() {
    return email;
  }

  public void setEmail(String email) {
    this.email = email;
  }

  @Override
  public String toString() {
    return "ContactPerson{" +
      "id=" + id +
      ", email='" + email + '\'' +
      '}';
  }
}
public class Customer {
  private long id;
  private String name;
  private Date creationDate;
  private Date activationDate;
  private ArrayList<ContactPerson> contactPeople = new ArrayList<>();

  public Customer() {
    this.id = Sequence.nextValue();
    this.creationDate = new Date();
  }

  public long getId() {
    return id;
  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    if (StringUtils.isEmpty(name)) {
      throw new IllegalArgumentException("Name cannot be empty");
    } else {
      this.name = name;
    }
  }

  public Date getCreationDate() {
    return creationDate;
  }

  public ArrayList<ContactPerson> getContactPeople() {
    return contactPeople;
  }

  public void setContactPeople(ArrayList<ContactPerson> contactPeople) {
    if (contactPeople == null) {
      throw new IllegalArgumentException("Contact people list cannot be null");
    } else {
      this.contactPeople = contactPeople;
    }
  }

  public void activate() {
    this.activationDate = new Date();
  }

  public boolean isActive() {
    return this.activationDate != null;
  }

  public void addContactPerson(ContactPerson contactPerson) {
    this.contactPeople.add(contactPerson);
  }

  public void removeContactPerson(long contactPersonId) {
    this.contactPeople.removeIf(it -> it.getId() == contactPersonId);
  }

  @Override
  public String toString() {
    return "Customer{" +
      "id=" + id +
      ", name='" + name + '\'' +
      ", creationDate=" + creationDate +
      ", activationDate=" + activationDate +
      ", contactPeople=" + contactPeople +
      '}';
  }
}
public class CustomerService {

  public Customer createCustomer(String name, ArrayList<ContactPerson> contactPeople) {
    final Customer customer = new Customer();
    customer.setName(name);
    customer.setContactPeople(contactPeople);
    return customer;
  }

  public ContactPerson createContactPerson(String email) {
    final ContactPerson contactPerson = new ContactPerson();
    contactPerson.setEmail(email);
    return contactPerson;
  }

}


What we can see now, is that our entities are not just simple data stores. They have a real behavior. Implementation details are hidden behind constructors, and business methods. Those business methods are nothing more than an abstraction of a real-world behavior of customer and its contact people. You can see now that abstraction, encapsulation and hermetization are complementary to each other. As a result of our refactoring, the CustomerService does nothing more than just creating objects, so we could even remove this class, and implement proper constructors or factory methods. Nice, huh?

Interface vs implementation


Let's have a look at the list of contact people. It is working, right? But don't we tell our client too much with getContactPeople and setContactPeople methods' signatures? I think we are, as we are using a concrete implementation of a java.util.List interface to declare a type of contact people collection. We are sharing yet another implementation detail here. In such situations what we should do is to use an interface (if such interface exists of course) instead - java.util.List in this case. The only place where we can refer to a specific class is object's construction. This way we both hide data representation and create a possibility to change the implementation from an ArrayList to a LinkedList if needed, without modifying our clients. Isn't it cool?

Don't think that you must always follow this approach. It is completely correct to refer to objects via class when one of the following situations apply to your case. Firstly, when you hava a class that does not implement any interface - then you simply have no choice, and must use class as a type.Secondly, when a class belongs to some class hierarchy, then it is recommended to refer to an object via the base class (usually an abstract one). And finally, your class might implement an interface, but contain some additional methods, not existing in mentioned interface. If and only if your client's code need to call this extra methods - then you need to refer to object via this class instead of using its interface.

It is all about providing both hermetization and flexibility. Let's have a look at Customer class now:
public class Customer {
  private long id;
  private String name;
  private Date creationDate;
  private Date activationDate;
  private List<ContactPerson> contactPeople = new ArrayList<>();

  public Customer() {
    this.id = Sequence.nextValue();
    this.creationDate = new Date();
  }

  public long getId() {
    return id;
  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    if (StringUtils.isEmpty(name)) {
      throw new IllegalArgumentException("Name cannot be empty");
    } else {
      this.name = name;
    }
  }

  public Date getCreationDate() {
    return creationDate;
  }

  public List<ContactPerson> getContactPeople() {
    return contactPeople;
  }

  public void setContactPeople(List<ContactPerson> contactPeople) {
    if (contactPeople == null) {
      throw new IllegalArgumentException("Contact people list cannot be null");
    } else {
      this.contactPeople = contactPeople;
    }
  }

  public void activate() {
    this.activationDate = new Date();
  }

  public boolean isActive() {
    return this.activationDate != null;
  }

  public void addContactPerson(ContactPerson contactPerson) {
    this.contactPeople.add(contactPerson);
  }

  public void removeContactPerson(long contactPersonId) {
    this.contactPeople.removeIf(it -> it.getId() == contactPersonId);
  }

  @Override
  public String toString() {
    return "Customer{" +
      "id=" + id +
      ", name='" + name + '\'' +
      ", creationDate=" + creationDate +
      ", activationDate=" + activationDate +
      ", contactPeople=" + contactPeople +
      '}';
  }
} 

toString()


It might look trivial to override default toString() method, but my experience shows that the way you print your object details is very vital. First of all, there might be some crazy guy that will write an application reading your logs, parsing them and calculating something - yep, I saw that on production. Secondly, lots of people have access to our log files and may read some sensitive information. It means that toString() method should be implemented with the same attention like an API. It should expose only those information, that are accessible programmatically. It is a common mistake that we just hit "Generate toString()" in our IDE, because it usually creates a method printing all your private fields and breaking hermetization by publishing all information about data representation. We have one violation of this rule in Customer class, then. Below you can see how our toString() method should look like:

public class Customer {
  ...

  @Override
  public String toString() {
    return "Customer{" +
      "id=" + id +
      ", name='" + name + '\'' +
      ", creationDate=" + creationDate +
      ", active=" + isActive() + //activation date no longer here!
      ", contactPeople=" + contactPeople +
      '}';
  }
} 


equals() and hashCode()


Another couple of methods common for all objects are equals() and hashCode(). They are also strongly connected with OOP principles, as they are part of your API. Each time when objects of your class are considered to be compared or play the role of a key in some data structure, and their equality is based on some logical assumptions, then you should override Object's default equals() method. It is said that if you can do something inside your class or a library - do it there - don't force your clients to implement it on their own, because you will spread the logic outside of the class and break the hermetization. This problem is similar to what we described in Access control, accessors, mutators chapter, while moving activateCustomer() and isCustomerActive() methods from service to Customer class. Of course you should also remember that overriding equals() implies necessity of implementing hashCode(), too. As soon as you have both methods, your clients doesn't have to worry about deciding how they should compare objects, or wondering if they can use them in a hash map - you hide (hermetize) it inside your class. Below you will find out how equals() and hashCode() could look like in our example (let's assume that all instance fields define object's identity).

Please note that the description of rules which should be followed while implementing equals() and hashCode() are out of the scope of this article.
public class ContactPerson {

  ...
    
  @Override
  public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;

    final ContactPerson that = (ContactPerson) o;

    if (id != that.id) return false;
    return email != null ? email.equals(that.email) : that.email == null;
  }

  @Override
  public int hashCode() {
    int result = (int) (id ^ (id >>> 32));
    result = 31 * result + (email != null ? email.hashCode() : 0);
    return result;
  }

}

public class Customer {

  ...
  @Override
  public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;

    final Customer customer = (Customer) o;

    if (id != customer.id) return false;
    if (name != null ? !name.equals(customer.name) : customer.name != null) return false;
    if (creationDate != null ? !creationDate.equals(customer.creationDate) : customer.creationDate != null) return false;
    if (activationDate != null ? !activationDate.equals(customer.activationDate) : customer.activationDate != null) return false;
    return contactPeople != null ? contactPeople.equals(customer.contactPeople) : customer.contactPeople == null;
  }

  @Override
  public int hashCode() {
    int result = (int) (id ^ (id >>> 32));
    result = 31 * result + (name != null ? name.hashCode() : 0);
    result = 31 * result + (creationDate != null ? creationDate.hashCode() : 0);
    result = 31 * result + (activationDate != null ? activationDate.hashCode() : 0);
    result = 31 * result + (contactPeople != null ? contactPeople.hashCode() : 0);
    return result;
  }
}

Object references


When we operate on objects, we pass their references back and forth, especially when we pass an object as a method parameter or return it as a method's call result. Having a reference to a mutable object we can easily update its state. It means that if it is a reference to an instance field of some other object - we may break its hermetization. Let's have a look at our example. Imagine a situation, when our client calls getCreationDate() method like below:
Date creationDate = customer.getCreationDate();
creationDate.setTime(1504259400000L);

Did we just find a way of changing the state of our Customer object not even calling any of its mutators? Exactly. One of the possibilities to defend ourselves from this kind of threats is using immutable objects (check-out next chapter). Here we will focus on another option, then, that is defensive copying. Defensive copying is about returning copies (references to copies - to be precise) of our mutable instance fields and working on copies of objects passed as constructor or mutator parameters.

Well, in order to secure our creationDate field we will produce a copy. Here are two possible solutions:

  1. Cloning - it is a para-Java mechanism of creating objects without using constructors. There are lots of dangerous things around it, that won't be covered in detail here. It is considered as not-safe and we should avoid it but I also think that we should be aware of what can be possibly done via this technique. In a big short, it is about calling object's protected clone() method. In most cases, classes that enable cloning provide a public equivalent of clone(). What you need to know is that you should use it for defensive copying only if the class is declared final (nobody can extend it). Why is that? It is because there is a real threat that a subclass overrides clone() method which does both cloning and registering reference to original object somewhere, and we will lose hermetization (more about inheritance in furhter chapter) You must be also aware that clone() does just a shallow copy based on an assignment, which means that all instance fields will be initialized with = operator. Thus, if the object you are copying has an instance field which is a reference to some other object, this reference will be copied one to one (instance field of the object's clone will point to the same place in memory - any change within this object may corrupt all clones, unless you implement a deep cloning!). Now, please take a look at a sample cloning (don't use it with Date in a real life, as Date is not a final class):
    public Date getCreationDate() {
      return (Date) creationDate.clone();
    }
    

  2. Copying via constructor or factory method (which may be called a copy constructor) - this is a quick and safe method, free from threats carried by clone(), where you can fully control creation process yourself.
    public Date getCreationDate() {
      return new Date(creationDate.getTime());
    }
    

The same problem refers to the contactPeople collection. Whatever mutable collection you expose to the outter world, anyone can modify it freely without any notice. It means that we should neither return reference to the collection nor save such reference to our instance field within a setter. There are plenty of methods to perform the shallow copy of a collection (shallow in this case means that we will create a completely different collection object, but containing the same references to content elements!). Defensive copying for accessor method is rather straightforward. We could simply use Collections API like below:
public List<ContactPerson> getContactPeople() {
  return Collections.unmodifiableList(contactPeople);
}

Regarding mutator, we could either remove it and force our clients to use add/remove methods (addContactPerson/removeContactPerson) or make a shallow copy manually:
public void setContactPeople(List<ContactPerson> contactPeople) {
  if (contactPeople == null) {
    throw new IllegalArgumentException("Contact people list cannot be null");
  } else {
    this.contactPeople = new ArrayList<>(contactPeople);
  }
}

You should see now that the problem we face here is that (despite defensive copying of collection) we keep references to all elements and enable our clients to modify them. We could discuss now whether shallow copy gives a real hermetization or not, and... it depends. If we have references to immutable objects (please refer to further chapter) - we don't need to worry about hermetiaztion violations during copying. Well, immutables don't even need to be copied at all! If an object is mutable, though, then changing its state means changing indirectly the state of all objects containing its reference. Sometimes when we need to store a reference to a mutable object, we simply cannot copy all its fields. We have a perfect example of that in our code - is there a way of copying id field without using reflection? There is none, as we have id field generated automatically inside the constructor and there is no setter available. Now it is the moment where you should ask: okay, so why don't we restore setId methods? Personally, I consider it as a bad idea, because cloning is a very low level procedure, and it shouldn't influence our abstraction and hermetization. Thus, to workaround this problem we could e.g. create a private constructor that accepts all fields as parameters and provide our own copy() method that would call it and return a proper copy (you see, no reflection needed, no hermetization violations). Take a look:
public class ContactPerson {
  private long id;
  private String email;

  private ContactPerson(long id, String email) {
    this.id = id;
    this.email = email;
  }

  ...
  public ContactPerson copy() {
    return new ContactPerson(this.id, this.email);
  }
}

As soon as we have the ability to make a copy of the elements from our collection we can easily make deep copies inside getter and setter:
public List<ContactPerson> getContactPeople() {
  return this.contactPeople.stream().map(ContactPerson::copy).collect(toList());
}

public void setContactPeople(List<ContactPerson> contactPeople) {
  if (contactPeople == null) {
    throw new IllegalArgumentException("Contact people list cannot be null");
  } else {
    this.contactPeople = contactPeople.stream()
      .map(ContactPerson::copy).collect(toList());
  }
}

If we have a collection of objects of type that we have no authority to modify (e.g. as it comes from some external library), we should just perform a shallow copy and advise other programmers with a proper comment not to mutate the state of these objects.

Inheritance


Inheritance breaks hermetization! Every subclass is based on the logic encapsulated in its superclass, right? In order to work properly, a subclass needs to call its parent's super methods, which means it depends on implementation details of the predecessor. Now a superclass may change, and those changes influence all its subclasses. As a consequence, every alteration might require changes in children classes. Let me show you an example. Let's imagine, that we want to store a number of contact people in an instance variable inside a subclass of a Customer (yes, I know it is just a size() of our collection, but I want to visualise the problem) - let's call it a SpecialCustomer.
public class SpecialCustomer extends Customer {
  private int numberOfContactPeople = 0;

  @Override
  public void setContactPeople(List<ContactPerson> contactPeople) {
    numberOfContactPeople = contactPeople.size();
    super.setContactPeople(contactPeople);
  }

  @Override
  public void addContactPerson(ContactPerson contactPerson) {
    super.addContactPerson(contactPerson);
    numberOfContactPeople++;
  }

  public int getNumberOfContactPeople() {
    return numberOfContactPeople;
  }
}

And here we have a simple test, where we create SpecialCustomer setting one-element ContactPerson list:
@Test
public void specialCustomerShouldHaveContactPeopleCounterEqualToOne() {
  //given
  final ContactPerson contactPerson = 
    new ContactPerson("Steve", "Harris", "steve@gmail.com");
  final List<ContactPerson> contactPeople = new ArrayList<>();
  contactPeople.add(contactPerson);

  //when
  final SpecialCustomer specialCustomer = new SpecialCustomer();
  specialCustomer.setContactPeople(contactPeople);

  //then
  assertEquals(1, specialCustomer.getNumberOfContactPeople());
}

And what we get after running this test is:
java.lang.AssertionError: 
Expected :1
Actual   :2

The reason is that we didn't know, that setContactPeople method from Customer class calls addContactPerson inside, so our counter gets incremented twice. Hermetization of our subclass is broken by the dependency to its parent, and it is a clear example of a strong and dangerous coupling that inheritance bears. Ok, so how to deal with it?

Well, we could of course override every public method and put a desired logic in subclass without calling super methods, but this clearly isn't something that inheritance is for, and of course, we might not be aware e.g. about validations that are implemented in our super class. Another idea might be to use inheritance only when we want to add new methods, not overriding existing ones. Great, but if a method of the same signature is added to the base class, then the problem is back. What if a parent class has some errors? Then we populate all this errors to its children. Okay, I won't keep you in suspense anymore - you should prefer composition over inheritance. We won't discuss this pattern further, but as you know the threats connected with inheritance I would rather like to point out, that if we see a possibility of breaking hermetization, and we don't think our class should be extendable, we should declare it final (and we will do so with our example). If it is not the case, then we should declare as many methods final as possible and document exactly how each method should be overriden properly and hope that other programmers will stick to that rules.

Immutables


We had said it many times before that when we have immutable objects, we don't have to worry about hermetization issues. Sounds great, huh? What are those immutables, then? As the name implies, these are objects that cannot be modified once they are created. It means that they have one, consistent state throughout their lifecycle, and thus they may be safely shared among numerous threads. In terms of hermetization it is an essence of hiding and securing internals. Here are the features that immutable class must provide:

  • All instance fields are private
  • No mutators (setters) - there shouldn't be any possibility of changing the state of existing object
  • Any method that is supposed to change a state creates the object's copy with a properly updated fields
  • No method can be overriden - it is about dealing with threats carried with inheritance
  • All instance fields should be declared final, and thus, should be initialized inside constructor
  • Access to all mutable instance fields must be secured with defensive copying mechanism
As you can see, all those features reflect our previous discussion. Unless there is a strong reason of creating modifiable classes, we should do our best to provide immutable ones. What a wonderful world would it be if we could have our projects built upon immutables only... Basically, yes, it would be, but as everything in this world it has its drawbacks, too, and we should be aware of them. Now if an object is immutable, operations that require returning a new instance might be costly in terms of memory. Imagine copying arrays of thousand of elements each time we call addToYourArray method! Another side-effect is that the "old" instance will be alive as long as something holds its reference (or GC cleans it). Sometimes making immutables is even impossible. You may e.g. use JPA that requires no-arg constructor and you simply cannot declare your fields final and instantiate them at creation time. If we cannot make our classes immutable we should declare as many fields final as possible to keep the hermetization and security at the highest possible level.

When talking about immutables, it is also worth mentioning that immutable objects are desired to be used with so called Value Objects. Not digging deep into the topic, these are the objects that don't have any conceptual identity, and we specify them only by the state of their properties (fields). Keeping value objects immutable ones combines very good abstraction and encapsulation (by converting a logical value into an object) with hermetization, therefore securing their state (value) from any manipulations from the outside.

There are lots of good examples of immutables in core Java libraries: java.lang.String, java.math.BigInteger, java.time.ZonedDateTime. You should now know why didn't we do any defensive copy for String fields. Remember, if you only have an option - use immutables! Yes, we will do so too, we will throw Date away in favour of ZonedDateTime.

How do you think, can we make our classes pure immutable ones? Let's try it with a ContactPerson:
public final class ContactPerson {
  private final long id;
  private final String email;
 
  private ContactPerson(long id, String email) {
    this.id = id;
    this.email = email;
  }

  public long getId() {
    return id;
  }

  public String getEmail() {
    return email;
  }

  @Override
  public String toString() {
    return "ContactPerson{" +
      "email='" + email + '\'' +
      '}';
  }

  @Override
  public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;

    final ContactPerson that = (ContactPerson) o;

    if (id != that.id) return false;
    return email != null ? email.equals(that.email) : that.email == null;
  }

  @Override
  public int hashCode() {
    int result = (int) (id ^ (id >>> 32));
    result = 31 * result + (email != null ? email.hashCode() : 0);
    return result;
  }

  public ContactPerson copy() {
    return new ContactPerson(this.id, his.email);
  }

  public static ContactPerson valueOf(String email) {
    return new ContactPerson(Sequence.nextValue(), email);
  }
}

  • All instance fields private - checked
  • No mutators (setters) - checked
  • Any method that is supposed to change a state must create a copy with a properly updated fields - no such methods
  • No method can be overriden - checked
  • All instance fields final - checked
  • Mutable instance fields must be secured with defensive copying - checked
Success! We made ContactPerson an immutable class. You can see here that we added a static valueOf factory method instead of a public constructor, which gives a convenient way of creating objects (no more new operator) and gives another level of hermetization - client does not need to know what constructors are used inside.

Now can we do the same to a Customer class? Yes, but with a few regards. Firstly, if we declare activationDate final, we won't be able to activate the same object - we will need to create its copy. Similarily, we will need to clone whole customer while adding or removing any contact person. If we are okay with that, let's see how it works:

public class Customer {
  private final long id;
  private final String name;
  private final ZonedDateTime creationDate;
  private final ZonedDateTime activationDate;
  private final List<ContactPerson> contactPeople;
  
  private Customer(long id, String name, ZonedDateTime creationDate,
                   ZonedDateTime activationDate, List<ContactPerson> contactPeople) {
    if (contactPeople == null) {
      throw new IllegalArgumentException("Contact people list cannot be null");
    }
    if (StringUtils.isEmpty(name)) {
      throw new IllegalArgumentException("Name cannot be empty");
    }
    this.id = id;
    this.name = name;
    this.creationDate = creationDate;
    this.activationDate = activationDate;
    this.contactPeople = new ArrayList<>(contactPeople);
  }

  public long getId() {
    return id;
  }

  public String getName() {
    return name;
  }

  public ZonedDateTime getCreationDate() {
    return creationDate;
  }

  public List<ContactPerson> getContactPeople() {
    return Collections.unmodifiableList(contactPeople);
  }

  public Customer activate() {
    return new Customer(this.id, this.name, this.creationDate,
      ZonedDateTime.now(), new ArrayList<>(this.contactPeople));
  }

  public boolean isActive() {
    return this.activationDate != null;
  }

  public Customer addContactPerson(ContactPerson contactPerson) {
    final List<ContactPerson> newContactPersonList =
      new ArrayList<>(this.contactPeople);
    newContactPersonList.add(contactPerson);
    return new Customer(this.id, this.name, this.creationDate,
      this.activationDate, newContactPersonList);
  }

  public Customer removeContactPerson(long contactPersonId) {
    final List<ContactPerson> newContactPersonList =
      new ArrayList<>(this.contactPeople);
    newContactPersonList.removeIf(it -> it.getId() == contactPersonId);
    return new Customer(this.id, this.name, this.creationDate,
      this.activationDate, newContactPersonList);
  }

  @Override
  public String toString() {
    return "Customer{" +
      "id=" + id +
      ", name='" + name + '\'' +
      ", creationDate=" + creationDate +
      ", activationDate=" + activationDate +
      ", contactPeople=" + contactPeople +
      '}';
  }

  @Override
  public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;

    final Customer customer = (Customer) o;

    if (id != customer.id) return false;
    if (name != null ? !name.equals(customer.name) : customer.name != null) return false;
    if (creationDate != null ? !creationDate.equals(customer.creationDate) : customer.creationDate != null) return false;
    if (activationDate != null ? !activationDate.equals(customer.activationDate) : customer.activationDate != null) return false;
    return contactPeople != null ? contactPeople.equals(customer.contactPeople) : customer.contactPeople == null;
  }

  @Override
  public int hashCode() {
    int result = (int) (id ^ (id >>> 32));
    result = 31 * result + (name != null ? name.hashCode() : 0);
    result = 31 * result + (creationDate != null ? creationDate.hashCode() : 0);
    result = 31 * result + (activationDate != null ? activationDate.hashCode() : 0);
    result = 31 * result + (contactPeople != null ? contactPeople.hashCode() : 0);
    return result;
  }

  public static Customer valueOf(String name, List<ContactPerson> contactPeople) {
    return new Customer(Sequence.nextValue(), name, ZonedDateTime.now(),
      null, contactPeople);
  }
}

  • All instance fields private - checked
  • No mutators (setters) - checked
  • Any method that is supposed to change a state must create a copy with a properly updated fields - checked
  • No method can be overriden - checked
  • All instance fields final - checked
  • Mutable instance fields must be secured with defensive copying - checked
Please note that in getContactPeople method we got back to a shallow copy solution. This is because ContactPerson is an immutable class so we don't need to copy its objects any more. All right, we have our classes hermetized!

Serialization


It is a common thing that we send our objects throughout the network or persist them somehow (in a queue, event store, etc.). In order to convert object to some unified binary form we often use Java serialization mechanism. We are not going to describe how serialization works but rather point out its connection with hermetization.

Well, what my experience shows, there is something about serialization that makes people think that the only thing they should do is to implement Serializable interface. Unfortunately it might be illusive. Let me explain.

Deserialization restores the private state of an object. It means that we can treat readObject method as another public constructor. Thus, we cannot trust the external byte stream (like we don't trust our clients) that it will always contain proper data. If there are any validations implemented in our constructors or setters, they should be applied within readObject method, too. It will prevent us from initializing corrupted object.

Another threat that we are exposed to is that the serialized byte stream might contain so called back references to objects that are already inside the stream. If these objects are mutable, then we face the same problem like we described in Object references chapter. You should know by now, that what we should do inside readObject method is a defensive copy of all mutable objects.

In case of ContactPerson class, default serialized form of an object corresponds to its logical content, and we don't have any references to mutable objects nor validations there. As a result it is enough to simply impement Serializable interface, like below:

public final class ContactPerson implements Serializable {
  private static final long serialVersionUID = 1L;
  ...
}

In Customer class, apart from implementing Serializable interface, we need to override readObject method and both put appropriate validations and make defensive copies of all instance fields that are mutable.

public class Customer implements Serializable {
  private static final long serialVersionUID = 1L;

  private final long id;
  private final String name;
  private final ZonedDateTime creationDate;
  private final ZonedDateTime activationDate;
  private List<ContactPerson> contactPeople;

  ...

  private void readObject(ObjectInputStream s) throws IOException, ClassNotFoundException {
    s.defaultReadObject();
    if (contactPeople == null) {
      throw new NotSerializableException("Contact people list cannot be null");
    }
    if (StringUtils.isEmpty(name)) {
      throw new NotSerializableException("Name cannot be empty");
    }
    contactPeople = new ArrayList<>(contactPeople);
  }
}

One thing that we needed to do was removing the final keyword from contactPeople field. It is because we are doing defensive copying (shallow one) after the object is constructed - readObject works like constructor but it isn't one. Here we partially break one of the rules from immutable object definition, but having defensive copy mechanism for this field, we will still have effectively immutable object and safe deserialization, too.

Conclusions


As you could see in this long article, there is a lot of philosophy around OOP concepts, which is not that simple as it might sometimes seem to be. You had a chance to see that all OOP concepts are complementary, and overlap each other. We cannot talk about hermetization, not mentioning encapsulation, abstraction or inheritance. Here are some rules that you should follow to have hermetization in your code:

  • Create immutables when possible.
  • Keep access to instance members as restricted as possible.
  • Give your class a public access modifier only when you are sure that you want to expose it outside the package.
  • Declare accessors only to fields you want to expose to the outter world.
  • Declare modifiers only to fields that really might be changed in the future and that are not to be set automatically.
  • Give your class a behavior, so that methods reflect the abstraction, hiding data representation and other implementation details.
  • If you have instance fields that are references to mutable objects, provide defensive copying in getters, setters and constructors.
  • Try to keep your class final. If you want your class to be extended, declare as many methods final as possible, and document the class clearly.
  • Always prefer interfaces (or base classes from class hierarchy) rather than their implementations as types of fields/params/method return values.
  • Override toString() method so that it prints information accessible programmatically from the outside of a class - it is like your API.
  • If your objects are supposed to be compared to each other or used as keys in data structure, provide your own equals() and hashCode() methods.
  • Prefer composition over inheritance.
  • Put proper validations and defensive copying (if needed) inside readObject method while implementing Serializable interface.