6. Performance
Amount of useful work accomplished by a
computer system compared to the time and
resources used
7. Scalability
Capability of a system to increase the amount of
useful work as resources and load are added to
the system
8. Scalability
• A system that performs fast with 10 users
might not do so with 1000 - it doesn’t scale
• Designing for scalability always decreases
performance
12. Scalability is about
parallelizing
• Parallel decomposition allows division of
work
• Parallelizing might mean more work
• There’s almost always a part of serial
computation
15. Vertical Scalability
Scale Up
• Bigger, meaner machines
- More cores (and more powerful)
- More memory
- Faster local storage
• Limited
- Technical constraints
- Cost - big machines get exponentially
expensive
16. Shared State
• Need to use those cores
• Java - shared-state concurrency
- Mutable state protected with locks
- Hard to get right
- Most developers don’t have experience
writing multithreaded code
17. This is how they look
like
public static synchronized SomeObject getInstance() {
return instance;
}
public SomeObject doConcurrentThingy() {
synchronized(this) {
//...
}
return ..;
}
18. Single vs Multi-threaded
• Single-threaded
- No scheduling cost
- No synchronization cost
• Multi-threaded
- Context Switching (high cost)
- Memory Synchronization (memory barriers)
- Blocking
19. Lock Contention
Little’s Law
The average number of customers in a stable
system is equal to their average arrival rate
multiplied by their average time in the system
20. Reducing Contention
• Reduce lock duration
• Reduce frequency with which locks are
requested (stripping)
• Replace exclusive locks with other mechanisms
- Concurrent Collections
- ReadWriteLocks
- Atomic Variables
- Immutable Objects
21. Concurrent Collections
• Use lock stripping
• Includes putIfAbsent() and replace()
methods
• ConcurrentHashMap has 16 separate locks by
default
• Don’t reinvent the wheel
22. ReadWriteLocks
• Pair of locks
• Read lock can be held by multiple
threads if there are no writers
• Write lock is exclusive
• Good improvements if object as fewer
writers
23. Atomic Variables
• Allow to make check-update type of
operations atomically
• Without locks - use low-level CPU
instructions
• It’s volatile on steroids (visibility +
atomicity)
24. Immutable Objects
• Immutability makes concurrency simple - thread-
safety guaranteed
• An immutable object is:
- final
- fields are final and private
- Constructor constructs the object completely
- No state changing methods
- Copy internal mutable objects when receiving
or returning
25. JVM issues
• Caching is useful - storing stuff in memory
• Larger JVM heap size means longer garbage
collection times
• Not acceptable to have long pauses
• Solutions
- Maximum size for heap 2GB/4GB
- Multiple JVMs per machine
- Better garbage collectors: G1 might help
26. Scaling Up: Other
Approaches
• Change the paradigm
- Actors (Erlang and Scala)
- Dataflow programming (GParallelizer)
- Software Transactional Memory
(Pastrami)
- Functional languages, such as Clojure
27. Scaling Up: Other
Approaches
• Dedicated JVM-friendly hardware
- Azul Systems is amazing
- Hundreds of cores
- Enormous heap sizes with negligible gc
pauses
- HTM included
- Built-in lock elision mechanism
30. Horizontal Scalability
Scale Out
• Big machines are expensive - 1 x 32 core
normally much more expensive than 4 x
8 core
• Increase throughput by adding more
machines
• Distributed Systems research revisited -
not new
42. Challenges
• How do we route requests to servers?
• How do distribute data between servers?
• How do we handle failures?
• How do we keep our cache consistent?
• How do we handle load peaks?
44. Technique #1: Partitioning
• Each server handles a subset of data
• Improves scalability by parallelizing
• Requires predictable routing
• Introduces problems with locality
• Move work to where the data is!
48. Technique #3: Messaging
• Use message passing, queues and pub/sub
models - JMS
• Improves reliability easily
• Helps deal with peaks
- The queue keeps filling
- If it gets too big, extra requests are
rejected
49. Solution #1: De-
normalize DB
• Faster queries
• Additional work to generate tables
• Less space efficiency
• Harder to maintain consistency
50. Solution #2: Non-SQL
Database
• Why not remove the relational part
altogether
• Bad for complex queries
• Berkeley DB is a prime example
51. Solution #3: Distributed
Key/Value Stores
• Highly scalable - used in the largest websites in the
world, based on Amazon’s Dynamo and Google’s
BigTable
• Mostly open source
• Partitioned
• Replicated
• Versioned
• No SPOF
• Voldemort (LinkedIn), Cassandra (Facebook) and HBase
are written in Java
62. Solution #4:
MapReduce
• Google’s algorithm to split work, process it
and reduce to an answer
• Used for offline processing of large
amounts of data
• Hadoop is used everywhere! Other options
such as GridGain exist
63. Solution #5: Data Grid
• Data (and computations)
• In-memory - low response times
• Database back-end (SQL or not)
• Partitioned - operations on data executed in
specific partition
• Replicated - handles failover automatically
• Transactional
64. Solution #5: Data Grid
• It’s a distributed cache + computational
engine
• Can be used as a cache with JPA and the like
• Oracle Coherence is very good.
• Terracotta, Gridgain, Gemfire, Gigaspaces,
Velocity (Microsoft) and Websphere
extreme scale (IBM)
65. Retrospective
• You need to scale up and out
• Write code thinking of hundreds of cores
• Relational might not be the way to go
• Cache whenever you can
• Be aware of data locality
66. Q &A
Thanks for listening!
Ruben Badaró
http://www.zonaj.org