This presentation: * covers basics of caching and popular cache types * explains evolution from simple cache to distributed, and from distributed to IMDG * not describes usage of NoSQL solutions for caching * is not intended for products comparison or for promotion of Hazelcast as the best solution

1. From cache to in-memory data grid.
Introduction to Hazelcast.
By Taras Matyashovsky

2. Introduction

3. About me
• Software engineer/TL
• Worked for outsource companies, product
companies and tried myself in startups/
freelancing
• 7+ years production Java experience
• Fan of Agile methodologies, CSM

4. What?
• This presentation:
• covers basics of caching and popular cache
types
• explains evolution from simple cache to
distributed, and from distributed to IMDG
• not describes usage of NoSQL solutions for
caching
• is not intended for products comparison or
for promotion of Hazelcast as the best
solution

5. Why?
• to expand horizons regarding modern
distributed architectures and solutions
• to share experience from my current
project where Infinispan was replaced
with Hazelcast as in-memory distributed
cache solution

6. Agenda
1st part:
• Why software caches?
• Common cache attributes
• Cache access patterns
• Cache types
• Distributed cache vs. IMDG

7. Agenda
2nd part:
• Hazelcast in a nutshell
• Hazelcast configuration
• Live demo sessions
• in-memory distributed cache
• write-through cache with Postgres as storage
• search in distributed cache
• parallel processing using executor service and entry
processor
• Infinispan vs. Hazelcast
• Best practices and personal recommendations

8. Caching Basics

9. Why Software Caching?
• application performance:
• many concurrent users
• time and costs overhead to access
application’s data stored in RDBMS or file
system
• database-access bottlenecks caused by too
many simultaneous requests

10. So Software Caches
• improve response times by reducing data access
latency
• offload persistent storages by reducing number
of trips to data sources
• avoid the cost of repeatedly creating objects
• share objects between threads
• only work for IO-bound applications

11. So Software Caches
are essential for modern
high-loaded applications

12. But
• memory size
• is limited
• can become unacceptably huge
• synchronization complexity
• consistency between the cached data state
and data source’s original data
• durability
• correct cache invalidation
• scalability

13. Common Cache Attributes
• maximum size, e.g. quantity of entries
• cache algorithm used for invalidation/eviction,
e.g.:
• least recently used (LRU)
• least frequently used (LFU)
• FIFO
• eviction percentage
• expiration, e.g.:
• time-to-live (TTL)
• absolute/relative time-based expiration

14. Cache Access Patterns
• cache aside
• read-through
• refresh-ahead
• write-through
• write-behind

15. Cache Aside Pattern
• application is responsible for reading and writing
from the storage and the cache doesn't interact
with the storage at all
• the cache is “kept aside” as a faster and more
scalable in-memory data store
Client
Cache
Storage

16. Read-Through/Write-Through
• the application treats cache as the main data
store and reads/writes data from/to it
• the cache is responsible for reading and writing
this data to the database
Client Cache Storage

17. Write-Behind Pattern
• modified cache entries are asynchronously
written to the storage after a configurable delay
Client Cache Storage

18. Refresh-Ahead Pattern
• automatically and asynchronously reload
(refresh) any recently accessed cache entry from
the cache loader prior to its expiration
Client Cache Storage

19. Cache Strategy Selection
RT/WT vs. cache-aside:
• RT/WT simplifies application code
• cache-aside may have blocking behavior
• cache-aside may be preferable when there are
multiple cache updates triggered to the same
storage from different cache servers

20. Cache Strategy Selection
Write-through vs. write-behind:
• write-behind caching may deliver considerably
higher throughput and reduced latency
compared to write-through caching
• implication of write-behind caching is that
database updates occur outside of the cache
transaction
• write-behind transaction can conflict with an
external update

21. Cache Types

22. Cache Types
• local cache
• replicated cache
• distributed cache
• remote cache
• near cache

23. Local Cache
a cache that is local to
(completely contained within)
a particular cluster node

24. Local Cache
Pros:
• simplicity
• performance
• no serialization/deserialization overhead
Cons:
• not a fault-tolerant
• scalability

25. Local Cache
Solutions:
• EhCache
• Google Guava
• Infinispan local cache mode

26. Replicated Cache
a cache that replicates its data
to all cluster nodes

27. Get in Replicated Cache
Each cluster node (JVM) accesses the data from its
own memory, i.e. local read:

28. Put in Replicated Cache
Pushing the new version of the data to all other
cluster nodes:

29. Replicated Cache
Pros:
• best read performance
• fault–tolerant
• linear performance scalability for reads
Cons:
• poor write performance
• additional network load
• poor and limited scalability for writes
• memory consumption

30. Replicated Cache
Solutions:
• open-source:
• Infinispan
• commercial:
• Oracle Coherence
• EhCache + Terracota

31. Distributed Cache
a cache that partitions its data
among all cluster nodes

32. Get in Distributed Cache
Access often must go over the network to another
cluster node:

33. Put in Distributed Cache
Resolving known limitation of replicated cache:

34. Put in Distributed Cache
• the data is being sent to a primary cluster node
and a backup cluster node if backup count is 1
• modifications to the cache are not considered
complete until all backups have acknowledged
receipt of the modification, i.e. slight
performance penalty
• such overhead guarantees that data consistency
is maintained and no data is lost

35. Failover in Distributed Cache
Failover involves promoting backup data to be
primary storage:

36. Local Storage in Distributed Cache
Certain cluster nodes can be configured to store
data, and others to be configured to not store
data:

37. Distributed Cache
Pros:
• linear performance scalability for reads and
writes
• fault-tolerant
Cons:
• increased latency of reads (due to network
round-trip and serialization/deserialization
expenses)

38. Distributed Cache Summary
Distributed in-memory key/value stores
supports a simple set of “put” and “get”
operations and optionally read-through and
write-through behavior for writing and
reading values to and from underlying
disk-based storage such as an RDBMS

39. Distributed Cache Summary
Depending on the product additional
features like:
• ACID transactions
• eviction policies
• replication vs. partitioning
• active backups
also became available as the products
matured

40. Distributed Cache
Solutions:
• open-source:
• Infinispan
• Hazelcast
• NoSQL storages, e.g. Redis, Cassandra,
MongoDB, etc.
• commercial:
• Oracle Coherence
• Terracota

41. Remote Cache
a cache that is located remotely and
should be accessed by a client(s)

42. Remote Cache
Majority of existing distributed/replicated
caches solutions support 2 modes:
• embedded mode
• when cache instance is started within the same JVM
as your application
• client-server mode
• when remote cache instance is started and clients
connect to it using a variety of different protocols

43. Remote Cache
Solutions:
• Infinispan remote cache mode
• Hazelcast client-server mode
• Memcached

44. Near Cache
a hybrid cache;
it typically fronts a distributed cache or a
remote cache with a local cache

45. Get in Near Cache
When an object is fetched from remote node, it is
put to local cache, so subsequent requests are
handled by local node retrieving from local cache:

46. Near Cache
Pros:
• it is best used for read only data
Cons:
• increases memory usage since the near cache
items need to be stored in the memory of the
member
• reduces consistency

47. In-memory Data Grid

48. In-memory Data Grid (IMDG)

49. In-memory Data Grid
In-memory distributed cache plus:
• ability to support co-location of computations
with data in a distributed context and move
computation to data
• distributed MPP processing based on standard
SQL and/or Map/Reduce, that allows to
effectively compute over data stored in-memory
across the cluster

50. IMDC vs. IMDG
• in-memory distributed caches were
developed in response to a growing need
for data high-availability
• in-memory data grids were developed to
respond to the growing complexities of
data processing

51. IMDG in a nutshell
Adding distributed SQL and/or MapReduce
type processing required a complete
re-thinking of distributed caches, as focus
has shifted from pure data management to
hybrid data and compute management

52. In-memory Data Grid Solutions

53. Hazelcast

54. Hazelcast
The leading open source
in-memory data grid
free alternative to proprietary solutions,
such as Oracle Coherence,
VMWare Pivotal Gemfire and
Software AG Terracotta

55. Hazelcast Use-Cases
• scale your application
• share data across cluster
• partition your data
• balance the load
• send/receive messages
• process in parallel on many JVMs, i.e. MPP

56. Hazelcast Features
• dynamic clustering, backup, discovery,
fail-over
• distributed map, queue, set, list, lock,
semaphore, topic, executor service, etc.
• transaction support
• map/reduce API
• Java client for accessing the cluster
remotely

57. Hazelcast Configuration
• programmatic configuration
• XML configuration
• Spring configuration
Nuance:
It is very important that the configuration on all
members in the cluster is exactly the same,
it doesn’t matter if you use the XML based
configuration or the programmatic configuration.

58. Sample Application

59. Live Demo “Configuration”

60. Sample Application
Technologies:
• Spring Boot 1.0.1
• Hazelcast 3.2
• Postgres 9.3
Application:
• RESTful web service to get/put data from/to cache
• RESTful web service to execute tasks in the cluster
• one Instance of Hazelcast per application
* Some samples are not optimal and created just to demonstrate usage of existing Hazelcast API

61. Global Hazelcast Configuration
Defined global Hazelcast configuration in separate
config in common module. It contains skeleton for
future Hazelcast instance as well as global
configuration settings:
• instance configuration skeleton
• common properties
• group name and password
• TCP based network configuration
• join config
• multicast and TCP/IP config
• default distributed map configuration skeleton

62. Hazelcast Instance
Each module that uses Hazelcast for distributed
cache should have its own separate Hazelcast
instance.
The “Hazelcast Instance” is a factory for creating
individual cache objects.
Each cache has a name and potentially distinct
configuration settings (expiration, eviction,
replication, and more).
Multiple instances can live within the same JVM.

63. Hazelcast Cluster Group
Groups are used in order to have multiple isolated
clusters on the same network instead of a single
cluster.
JVM can host multiple Hazelcast instances (nodes).
Each node can only participate in one group and it
only joins to its own group, does not mess with
others.
In order to achieve this group name and group
password configuration properties are used.

64. Hazelcast Network Config
In our environment multicast mechanism for
joining the cluster is not supported, so only TCP/IP-cluster
approach will be used.
In this case there should be a one or more well
known members to connect to.

65. Live Demo “Map Store”

66. Hazelcast Map Store
• useful for reading and writing map entries from
and to an external data source
• one instance per map per node will be created
• word of caution: the map store should NOT call
distributed map operations, otherwise you
might run into deadlocks

67. Hazelcast Map Store
• map pre-population via loadAllKeys method that
returns the set of all “hot” keys that need to be
loaded for the partitions owned by the member
• write through vs. write behind using “write-delay-
seconds” configuration (0 or bigger)
• MapLoaderLifecycleSupport to be notified of
lifecycle events, i.e. init and destroy

68. Live Demo “Executor Service”

69. Hazelcast Executor Service
• extends the java.util.concurrent.ExecutorService,
but is designed to be used in a distributed
environment
• scaling up via threads pool size
• scaling out is automatic via addition of new
Hazelcast instances

70. Hazelcast Executor Service
• provides different ways to route tasks:
• any member
• specific member
• the member hosting a specific key
• all or subset of members
• supports execution callback

71. Hazelcast Executor Service
Drawbacks:
• work-queue has no high availability:
• each member will create local ThreadPoolExecutors
with ordinary work-queues that do the real work but
not backed up by Hazelcast
• work-queue is not partitioned:
• it could be that one member has a lot of unprocessed
work, and another is idle
• no customizable load balancing

72. Hazelcast Features
More useful features:
• entry listener
• transactions support, e.g. local, distributed
• map reduce API out-of-the-box
• custom serialization/deserialization mechanism
• distributed topic
• clients

73. Hazelcast Missing Features
Missing useful features:
• update configuration in running cluster
• load balancing for executor service

74. Infinispan vs. Hazelcast

75. Infinispan vs. Hazelcast
Infinispan Hazelcast
Pros • backed by relatively large
company for use in largely
distributed environments
(JBoss)
• been in active use for
several years
• well-written documentation
• a lot of examples of different
configurations as well as
solutions to common
problems
• easy setup
• more performant than
Infinispan
• simple node/cluster
discovery mechanism
• relies on only 1 jar to be
included on classpath
• brief documentation
completed with simple
code samples

76. Infinispan vs. Hazelcast
Infinispan Hazelcast
Cons • relies on JGroups that
proven to be buggy
especially under high load
• configuration can be
overly complex
• ~9 jars are needed in
order to get Infinispan up
and running
• code appears very
complex and hard to
debug/trace
• backed by a startup based
in Palo Alto and Turkey,
just received Series A 2.5
M funding from Bain
Capital Ventures
• customization points are
fairly limited
• some exceptions can be
difficult to diagnose due to
poorly written exception
messages
• still quite buggy

77. Hazelcast Summary

78. Best Practices
• each specific Hazelcast instance should have its
unique instance name
• each specific Hazelcast instance should have its
unique group name and password
• each specific Hazelcast instance should start on
separate port according to predefined ranges

79. Personal Recommendations
• use XML configuration in production, but don’t
use spring:hz schema. Our Spring based “lego
bricks” approach for building resulting Hazelcast
instance is quite decent.
• don’t use Hazelcast for local caches as it was
never designed with that purpose and always
performs serialization/deserialization
• don’t use library specific classes, use common
collections, e.g. ConcurrentMap, and you will be
able to replace underlying cache solution easily

80. Hazelcast Drawbacks
• still quite buggy
• poor documentation for more complex
cases
• enterprise edition costs money, but
includes:
• elastic memory
• JAAS security
• .NET and C++ clients

81. Q/A?

82. Thank you!
by Taras Matyashovsky

83. References
• http://docs.oracle.com/cd/E18686_01/coh.37/e18677/cache_intro.htm
• http://coherence.oracle.com/display/COH31UG/Read-Through,+Write-
Through,+Refresh-Ahead+and+Write-Behind+Caching
• http://blog.tekmindsolutions.com/oracle-coherence-diffrence-between-replicated-
cache-vs-partitioneddistributed-cache/
• http://www.slideshare.net/MaxAlexejev/from-distributed-caches-to-inmemory-data-
grids
• http://www.slideshare.net/jaxlondon2012/clustering-your-application-with-hazelcast
• http://www.gridgain.com/blog/fyi/cache-data-grid-database/
• http://gridgaintech.wordpress.com/2013/10/19/distributed-caching-is-dead-long-
live/
• http://www.hazelcast.com/resources/the-book-of-hazelcast/
• https://labs.consol.de/java-caches/part-3-3-peer-to-peer-with-hazelcast/
• http://hazelcast.com/resources/thinking-distributed-the-hazelcast-way/
• https://github.com/tmatyashovsky/hazelcast-samples/