Benchmark OpenERP : testez les performances et la robustesse d'OpenERP sur vos volumes de données.
Le benchmark technique d'OpenERP propose des scripts automatiques pour tester la charge d'OpenERP vis-à-vis de vos données et de votre activité.
1. OPENERP BENCHMARK
How to test performance and
robustness against your own volumes ?
Cyril MORISSE ( @cmorisse )
2. Introduction
At Audaxis, we decided to invest in a methodology
to answer 3 kinds of questions:
Is OpenERP able to manage my business
Is this hardware configuration able to sustain our
operations load ? Until when ?
What hardware should we buy to run OpenERP for the
next 3 years ?
3. What to measure ?
You must find the most challenging business processes and
conditions then express them in terms of what you can measure
Eg. Can OpenERP handle 240.000 orders per years with 40 users ?
You can’t benchmark on this base
240.000 orders per year is 2,5 orders per hour per user (on a 240
working days of 10 hours per year base)
Note: that this is one order every 36s too !
What you can measure is:
Can my system maintains response time < 1 s per transaction over 10
hours with :
40 users entering 1 order every 16 minutes during 10 (peak hours intensity)
That’s 1 order every 24s
4. What to measure (continued)
What does «good response time» technically means ?
JSON-RPC requests response time must be kept around
1s for the system to be felt as responsive.
Funkload measure responsiveness for you using APDEX
OpenERP
Database
OpenERP
Application
Server
User OpenERP
Web Client
Enter fields,
Press
buttons
JSON-RPC SQL
Responsiveness
1 Sale Order with 5 lines, “Confirmed” then “Invoiced”
~ 20 ~ 150 ~ 6700
5. How to measure it
Use Funkload
Develop a test case which simulates user interactions (via XML-RPC…)
Install « Funkload monitor » on your servers
Define and run Funkload benchmarks with:
A test case
List of cycle defining the number of concurrent users
Eg. 100,100,100,100
Cycle duration
Frequency (as a period : one test every x seconds)
You will get a (quite) fancy report with:
Test execution time
Individual request execution time
Main server metrics (CPU, RAM, Network Load)
9. Database size really matters
You must bench with real volume of data !!!
0
1
2
3
4
5
6
7
8
0 100000 200000 300000 400000 500000
OpenERP Response Time evolution over Sales Order
count
Response Time (s)
10. Database size really matters(continued)
In fact, you will need several benchmarks:
Bench-filler
Use these benchmarks to fill up your database (customers,
products, …)
Bench-reference
2 hours with target activity level
Bench-robustness
10 (several) hours with target activity level
Bench-limit
We run this one at the end of the bench to determine how many
users more the system will support at target activity level
11. Plan your architecture
but focus on the database
Configure your database ! At least use pgtune !!!
ORM may be your enemy ; think about the roundtrips involved by your code
Don’t split Application and Database servers too early !
If your database is not too big : think SSD (From 20x to 250x more IOPS than HDD)*
OpenERP Database
OpenERP
Application Server
OpenERP
Web Client
JSON-RPC SQL
Horizontal Scaling
VerticalScaling
12. Tools & References
Funkload ( http://funkload.nuxeo.org )
Benchmark (and Test) your OpenERP Server and others
Muppy ( http://bitbucket.org/cmorisse/muppy )
“Make your OpenERP Servers behave as puppets”
Install OpenERP platforms everywhere (in really alpha stage)
Pgbadger ( http://dalibo.github.io/pgbadger )
Analyse SQL requests processed by your PostgreSQL Server
Pgtune ( https://github.com/gregs1104/pgtune )
Helps you to configure your PostgreSQL (basically)
Pg_activity (https://github.com/julmon/pg_activity )
Top like application for PostgreSQL Server
OpenERP buildout Recipe ( https://pypi.python.org/pypi/anybox.recipe.openerp )
A set of Ugly scripts to automate the launch of Funkload Filler benchmarks
Theses tools allows to measure performance over database size
http://www.apdex.org
SQL Databases Don’t Scale by Adam WIGGINS
http://adam.heroku.com/past/2009/7/6/sql_databases_dont_scale/
HDD vs SSD by IBM
http://www-03.ibm.com/systems/power/hardware/peripherals/ssd/hdd_ssd.html