SlideShare une entreprise Scribd logo
1  sur  39
Télécharger pour lire hors ligne
vFabric SQLFire
Fast meets scalable in VMware's new
      distributed SQL database.
Speed Matters.




        Users demand fast applications and fast websites.
           The database is the hardest thing to scale.
What Is SQLFire?
• SQLFire is a memory-optimized, distributed SQL
  database.
• SQLFire satisfies modern application needs by
  adopting a more scalable design than traditional
  RDBMS.
• SQLFire focuses on Speed, Scale and SQL.
SQLFire: Speed, Scale, SQL.
       Speed                      Scale                          SQL
• Memory-optimized for   • Horizontally scalable.      • Familiar SQL interface.
  maximum speed and      • Add or remove nodes at      • SQL 92 compliant.
  minimum latency.         any time for more           • JDBC and ADO.NET
                           capacity or availability.     interfaces.
SQLFire Speed Through Memory Optimization.
• Memory Optimization
  – Latency is tracked to the millisecond.
  – Use memory to get an advantage.
• SQLFire can use disk too.
  – Data can be asynchronously persisted to disk.
  – Append-only logfile format for best update
    performance.
  – Disaster prevention and for larger databases.
What’s Really Great About Memory?




       Disk seeks are 100,000 times slower than memory reads!
                        (10,000,000 ns = 10 ms)
Who Needs That Kind Of Latency Anyway?
• Major online travel site.
• Minimizing page load time a must.
  – Did all the usual tricks including minimizing server
    round-trips, etc.
• Required all data access to complete in less than
  20ms – Always!
• Selected a memory-optimized database from
  VMware.
The Things People Do For Speed.
• Many architectures have appeared to make data faster.
• Many architectures
   –
  forMemcached on top of SQL Databases.
      speeding up data
   – Read-only slave nodes.
  have appeared lately.
   – Clustering / RAC.
• AllNoSQL. from
   – stem
• Clear need to overcome traditional database limitations.
   – Relational databases were not built to serve thousands (or
     even hundreds) of users at once.
   – RDBMS were not built for the low latency users expect.
Don’t Mask The Problem, Solve It!

   Browser Tier             Browser Tier




  Application Tier        Application Tier



 Memcache + SQL:              SQLFire
  Multi-data model      Horizontally Scalable
to overcome the DB.        SQL Database
What Is Horizontal Scalability?
• Software that makes
  multiple computers
  appear as one system.
• Horizontal scale gives:
   – Better performance.
   – More capacity.
   – Higher availability.
SQLFire Horizontal Scalability.
• SQLFire is a horizontally-scalable SQL Database.
• SQLFire is horizontally scalable from the ground up.
   –   Data inserts.
   –   Key lookup.
   –   Stored procedures / functions.
   –   Joins.
   –   Transactions.
   –   More.
SQLFire and SQL.
• SQLFire syntax is based on the SQL-92 standard.
• SQLFire extensions are to Data Definition
  Language (DDL), e.g. CREATE TABLE.
• SQLFire ships with JDBC and ADO.NET drivers.
  – Built-in, transparent high availability.
COMPARING SQLFIRE AND OTHER
DATABASES
SQLFire Challenges Traditional DB Design, Not SQL.
• Traditional RDBMS: Too much focus on ACID consistency.
• Traditional RDBMS: Too much contention for disk.
The database world is changing. A lot!
• Many new data models (NoSQL) are emerging
    –   Key-value
    –   Column family (inspired by Google BigTable)
    –   Document
    –   Graph
• Most focus on making model less rigid than SQL
• Consistency model is not ACID
             Low scale                 High scale     Very high scale

           STRICT – Full                Tunable
                                                         Eventual
          ACID (RDBMS)                Consistency
• Different tradeoffs for different goals
SQLFire At A Glance.
SQLFire Versus Other SQL Databases.

   Attribute                    SQLFire                            Other SQL DBs
  DB Interface     Standard SQL.                       Standard SQL.
                   Tunable. Mix of eventual            High consistency.
Data Consistency
                   consistency and high consistency.
  Transactions     Supported.                          Very strong support.
 Scaling Model     Scale out, commodity servers.       Scale up.
SQLFire Versus NoSQL.
   Attribute                        NoSQL                               SQLFire
  DB Interface       Idiosyncratic (i.e. each is custom).   Standard SQL.
    Querying         Idiosyncratic or not present.          SQL Queries.
Data Consistency     Tunable, most favor eventual           Tunable, favors high consistency.
                     consistency.
  Transactions       Weak or not present.                   Linearly scalable transaction
                                                            model.
 Interface Design    Designed for simplicity.               Designed for compatibility.
   Data Model        Wide variety of different models.      Relational model.
Schema Flexibility   Focus on extreme flexibility,          SQL model, requires DB
                     dynamism.                              migrations, etc.
Illustrating some of SQLFire’s key features.

HANDS ON WITH SQLFIRE
SQLFire Tables Are Replicated By Default.
1    CREATE TABLE sales
                                                              SQLFire Node 1
2      (product_id int, store_id int,
                                                                  Replica
3      price float);                         sales
4
5
6                                                             SQLFire Node 2
7                                                                 Replica
8                                        Best for small and
9                                       frequently accessed
                                                data.
10
Partitioned Tables Are Split Among Members.
1    CREATE TABLE sales
                                                         SQLFire Node 1
2      (product_id int, store_id int,
                                                             Replica
3      price float)                       sales
                                                           Partition 1
4    PARTITION BY
5      COLUMN (product_id);
6                                                        SQLFire Node 2
7                                                            Replica
8
                                        Best for large     Partition 2
9
                                         data sets.
10
Types Of Partitioning In SQLFire.
      Type                    Purpose                                     Example
                    Built-in hashing algorithm
Hash Partitioning
                    splits data at random across     PARTITION BY COLUMN (customer_id);
   (Default)
                    available servers.
                    Manually divide data across      PARTITION BY LIST (home_state)
      List          servers based on discrete         (VALUES (‘CA’, ‘WA’),
                    criteria.                          VALUES (‘TX’, ‘OK’));
                    Manually divide data across      PARTITION BY RANGE (date)
     Range          servers based on continuous       (VALUES BETWEEN ‘2008-01-01’ AND ‘2008-12-31’,
                    criteria.                          VALUES BETWEEN ‘2009-01-01’ AND ‘2009-12-30’);
                    Fully dynamic division of data
   Expression       based on function execution.     PARTITION BY (MONTH(date));
                    Can use UDFs.
Redundancy Increases Availability.
1    CREATE TABLE sales
                                                                SQLFire Node 1
2      (product_id int, store_id int,
                                                                    Replica
3      price float)                           sales
                                                                  Partition 1
4    PARTITION BY
                                                                  Partition 2*
5      COLUMN (product_id);
6      REDUNDANCY 1;                                            SQLFire Node 2
7                                                                   Replica
8
                                        All data is available     Partition 2
9
                                           if Node 1 fails.       Partition 1*
10
Collocate Data For Fast Joins.
1    CREATE TABLE sales                 Related data placed   SQLFire Node 1
2      (product_id int, store_id int,   on the same node.
                                                                  Replica
3      price float)
                                                                Customer 1
4    PARTITION BY                              C1             Customer 1 Sales
5      COLUMN (product_id);
6      COLOCATE WITH customers;                               SQLFire Node 2
7                                              C2                 Replica
8
                                         SQLFire can join       Customer 2
9                                         tables without      Customer 2 Sales
10                                        network hops.
Why Collocation Matters.
1    -- Biggest customers in CA.
                                                         SQLFire Node 1
2    SELECT sum(value) AS total
                                   Since we collocated
                                                             Replica
3     FROM sales, customer
                                   customers and sales     Customer 1
4     WHERE sales.customerid =      this query execute
                                      independently      Customer 1 Sales
5     customer.id AND
                                      on each node.
6     customer.state = “CA”                              SQLFire Node 2
7     ORDER BY total DESC;               Result:
                                    The query scales         Replica
8                                       linearly!          Customer 2
9
                                                         Customer 2 Sales
10
Scaling Functions and Stored Procedures.
1    CALL maxSales              SQLFire uses data-   maxSales on
2                                aware routing to     local data
       WITH RESULT PROCESSOR
                               route processing to
3        maxSalesReducer            the data.
4     ON TABLE sales;
5
                                  maxSalesReducer
6
7
8                              Result Processors
9                              give map/reduce       maxSales on
                                 functionality.       local data
10
Some SQLFire Features Not Discussed Include…
• Server Groups:
    – Granular control of data placement or restrict data to certain
      servers.
• Disk persistence and overflow.
    – For additional availability and when using SQLFire as a primary
      database.
•   Caching / Operational Data Store.
•   Transactions.
•   Indexes.
•   Much more!                                      Scan Me To Learn More
                                                    About These Features!
SQLFire In Action

DEMO
SQLFIRE RELEASE DETAILS
SQLFire Releases.
• SQLFire Professional 1.0:
   –   Release: December 13, 2011.
   –   Part of the vFabric Advanced suite. (Per VM)
   –   Also available standalone. (Per CPU)
   –   Limit of 2 connected nodes per database.
• SQLFire Enterprise 1.0:
   – Coming 1H 2012.
   – Unlimited nodes.
   – Optional SQLFire Enterprise WAN Upgrade for active/active
     global databases.
SQLFire WAN.
• Same database in multiple datacenters or the cloud.
• Fully active-active with asynchronous replication.
• Coming 1H 2012 with SQLFire Enterprise.
Download SQLFire Today!
• Beta available now on our community site.
   – http://tinyurl.com/SQLFire
• General availability December 2011.
   – vFabric Suite or standalone.
   – Free for development up to 3 nodes.
• Follow us on Twitter:
   – http://twitter.com/vfabricsqlfire        Scan To Learn More
QUESTIONS?
DEMO DETAILS
Demo Details
• 2 VMs.
   – ubuntu
   – livecd
• Schema:
   – Table called record
      • threadid: int, value int
      • Each value is validated upon insert.
      • The validate constraint check is extremely CPU intensive.
• Client: 5 threads inserting simultaneously.
• Total of 2500 records to be inserted.
Demo State 1
                  Ubuntu          Ubuntu
                 (locator)      (database)
        Insert




                             validate (very slow!)




                             Insert Complete
Create Table Code:
CREATE TABLE record
  (threadid int, value int
  CONSTRAINT MY_CK CHECK (validate(value) = 1))
  PARTITION BY COLUMN (threadid);

Note: The validate function takes a long time to
run.
Demo State 2
                      Ubuntu          Ubuntu                  livecd
                     (locator)      (database)              (database)
            Insert




             Note:
                                 validate (very slow!)   validate (very slow!)
       No client change!



                                 Insert Complete         Insert Complete
Source Code
• Base Directory:
   – https://github.com/cartershanklin/SQLFireStuff
• Java UDF:
   – https://github.com/cartershanklin/SQLFireStuff/blob/mas
     ter/java/examples/Validate.java
• Jython Test Script
   – https://github.com/cartershanklin/SQLFireStuff/blob/mas
     ter/jython/slowInserts.py
• Contact: @cshanklin

Contenu connexe

Tendances

Bank Data Frank Peterson DB2 10-Early_Experiences_pdf
Bank Data   Frank Peterson DB2 10-Early_Experiences_pdfBank Data   Frank Peterson DB2 10-Early_Experiences_pdf
Bank Data Frank Peterson DB2 10-Early_Experiences_pdfSurekha Parekh
 
MySQL Cluster NoSQL Memcached API
MySQL Cluster NoSQL Memcached APIMySQL Cluster NoSQL Memcached API
MySQL Cluster NoSQL Memcached APIMat Keep
 
DB2 and storage management
DB2 and storage managementDB2 and storage management
DB2 and storage managementCraig Mullins
 
Microsoft SQL Server Distributing Data with R2 Bertucci
Microsoft SQL Server Distributing Data with R2 BertucciMicrosoft SQL Server Distributing Data with R2 Bertucci
Microsoft SQL Server Distributing Data with R2 BertucciMark Ginnebaugh
 
Throughput comparison: Dell PowerEdge R720 drive options
Throughput comparison: Dell PowerEdge R720 drive optionsThroughput comparison: Dell PowerEdge R720 drive options
Throughput comparison: Dell PowerEdge R720 drive optionsPrincipled Technologies
 
Gustavo Garnica: Evolución de la Plataforma Java y lo que Significa para Ti
Gustavo Garnica: Evolución de la Plataforma Java y lo que Significa para TiGustavo Garnica: Evolución de la Plataforma Java y lo que Significa para Ti
Gustavo Garnica: Evolución de la Plataforma Java y lo que Significa para TiSoftware Guru
 
Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...
Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...
Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...DataStax Academy
 
The IBM eX5 Memory Advantage How Additional Memory Capacity on eX5 Can Benefi...
The IBM eX5 Memory Advantage How Additional Memory Capacity on eX5 Can Benefi...The IBM eX5 Memory Advantage How Additional Memory Capacity on eX5 Can Benefi...
The IBM eX5 Memory Advantage How Additional Memory Capacity on eX5 Can Benefi...IBM India Smarter Computing
 
Exadata is still oracle
Exadata is still oracleExadata is still oracle
Exadata is still oracleugur candan
 
Increased database performance and reduced costs with Dell PowerEdge FX2 & VM...
Increased database performance and reduced costs with Dell PowerEdge FX2 & VM...Increased database performance and reduced costs with Dell PowerEdge FX2 & VM...
Increased database performance and reduced costs with Dell PowerEdge FX2 & VM...Principled Technologies
 
SQL Server 2008 R2 Parallel Data Warehouse
SQL Server 2008 R2 Parallel Data WarehouseSQL Server 2008 R2 Parallel Data Warehouse
SQL Server 2008 R2 Parallel Data WarehouseMark Ginnebaugh
 
Solving the DB2 LUW Administration Dilemma
Solving the DB2 LUW Administration DilemmaSolving the DB2 LUW Administration Dilemma
Solving the DB2 LUW Administration DilemmaRandy Goering
 
Dell Acceleration Appliance for Databases 2.0 and Microsoft SQL Server 2014: ...
Dell Acceleration Appliance for Databases 2.0 and Microsoft SQL Server 2014: ...Dell Acceleration Appliance for Databases 2.0 and Microsoft SQL Server 2014: ...
Dell Acceleration Appliance for Databases 2.0 and Microsoft SQL Server 2014: ...Principled Technologies
 
Improve DB2 z/OS Test Data Management
Improve DB2 z/OS Test Data ManagementImprove DB2 z/OS Test Data Management
Improve DB2 z/OS Test Data Managementsoftbasemarketing
 
Dell PowerEdge M520 server solution: Energy efficiency and database performance
Dell PowerEdge M520 server solution: Energy efficiency and database performanceDell PowerEdge M520 server solution: Energy efficiency and database performance
Dell PowerEdge M520 server solution: Energy efficiency and database performancePrincipled Technologies
 
Using Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisUsing Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisScaleOut Software
 

Tendances (18)

DBA101
DBA101DBA101
DBA101
 
Ta3
Ta3Ta3
Ta3
 
Bank Data Frank Peterson DB2 10-Early_Experiences_pdf
Bank Data   Frank Peterson DB2 10-Early_Experiences_pdfBank Data   Frank Peterson DB2 10-Early_Experiences_pdf
Bank Data Frank Peterson DB2 10-Early_Experiences_pdf
 
MySQL Cluster NoSQL Memcached API
MySQL Cluster NoSQL Memcached APIMySQL Cluster NoSQL Memcached API
MySQL Cluster NoSQL Memcached API
 
DB2 and storage management
DB2 and storage managementDB2 and storage management
DB2 and storage management
 
Microsoft SQL Server Distributing Data with R2 Bertucci
Microsoft SQL Server Distributing Data with R2 BertucciMicrosoft SQL Server Distributing Data with R2 Bertucci
Microsoft SQL Server Distributing Data with R2 Bertucci
 
Throughput comparison: Dell PowerEdge R720 drive options
Throughput comparison: Dell PowerEdge R720 drive optionsThroughput comparison: Dell PowerEdge R720 drive options
Throughput comparison: Dell PowerEdge R720 drive options
 
Gustavo Garnica: Evolución de la Plataforma Java y lo que Significa para Ti
Gustavo Garnica: Evolución de la Plataforma Java y lo que Significa para TiGustavo Garnica: Evolución de la Plataforma Java y lo que Significa para Ti
Gustavo Garnica: Evolución de la Plataforma Java y lo que Significa para Ti
 
Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...
Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...
Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...
 
The IBM eX5 Memory Advantage How Additional Memory Capacity on eX5 Can Benefi...
The IBM eX5 Memory Advantage How Additional Memory Capacity on eX5 Can Benefi...The IBM eX5 Memory Advantage How Additional Memory Capacity on eX5 Can Benefi...
The IBM eX5 Memory Advantage How Additional Memory Capacity on eX5 Can Benefi...
 
Exadata is still oracle
Exadata is still oracleExadata is still oracle
Exadata is still oracle
 
Increased database performance and reduced costs with Dell PowerEdge FX2 & VM...
Increased database performance and reduced costs with Dell PowerEdge FX2 & VM...Increased database performance and reduced costs with Dell PowerEdge FX2 & VM...
Increased database performance and reduced costs with Dell PowerEdge FX2 & VM...
 
SQL Server 2008 R2 Parallel Data Warehouse
SQL Server 2008 R2 Parallel Data WarehouseSQL Server 2008 R2 Parallel Data Warehouse
SQL Server 2008 R2 Parallel Data Warehouse
 
Solving the DB2 LUW Administration Dilemma
Solving the DB2 LUW Administration DilemmaSolving the DB2 LUW Administration Dilemma
Solving the DB2 LUW Administration Dilemma
 
Dell Acceleration Appliance for Databases 2.0 and Microsoft SQL Server 2014: ...
Dell Acceleration Appliance for Databases 2.0 and Microsoft SQL Server 2014: ...Dell Acceleration Appliance for Databases 2.0 and Microsoft SQL Server 2014: ...
Dell Acceleration Appliance for Databases 2.0 and Microsoft SQL Server 2014: ...
 
Improve DB2 z/OS Test Data Management
Improve DB2 z/OS Test Data ManagementImprove DB2 z/OS Test Data Management
Improve DB2 z/OS Test Data Management
 
Dell PowerEdge M520 server solution: Energy efficiency and database performance
Dell PowerEdge M520 server solution: Energy efficiency and database performanceDell PowerEdge M520 server solution: Energy efficiency and database performance
Dell PowerEdge M520 server solution: Energy efficiency and database performance
 
Using Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisUsing Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data Analysis
 

Similaire à SQLFire Webinar

Solving performance problems in MySQL without denormalization
Solving performance problems in MySQL without denormalizationSolving performance problems in MySQL without denormalization
Solving performance problems in MySQL without denormalizationdmcfarlane
 
Akiban Technologies: Renormalize
Akiban Technologies: RenormalizeAkiban Technologies: Renormalize
Akiban Technologies: RenormalizeAriel Weil
 
Akiban Technologies: Renormalize
Akiban Technologies: RenormalizeAkiban Technologies: Renormalize
Akiban Technologies: RenormalizeAriel Weil
 
NewSQL - Deliverance from BASE and back to SQL and ACID
NewSQL - Deliverance from BASE and back to SQL and ACIDNewSQL - Deliverance from BASE and back to SQL and ACID
NewSQL - Deliverance from BASE and back to SQL and ACIDTony Rogerson
 
NewSQL Database Overview
NewSQL Database OverviewNewSQL Database Overview
NewSQL Database OverviewSteve Min
 
android sqlite
android sqliteandroid sqlite
android sqliteDeepa Rani
 
SQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerSQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerMichael Rys
 
MySQL New Features -- Sunshine PHP 2020 Presentation
MySQL New Features -- Sunshine PHP 2020 PresentationMySQL New Features -- Sunshine PHP 2020 Presentation
MySQL New Features -- Sunshine PHP 2020 PresentationDave Stokes
 
Building scalable application with sql server
Building scalable application with sql serverBuilding scalable application with sql server
Building scalable application with sql serverChris Adkin
 
My sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlMy sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlsqlhjalp
 
Rapid SQL Datasheet - The Intelligent IDE for SQL Development
Rapid SQL Datasheet - The Intelligent IDE for SQL DevelopmentRapid SQL Datasheet - The Intelligent IDE for SQL Development
Rapid SQL Datasheet - The Intelligent IDE for SQL DevelopmentEmbarcadero Technologies
 
fdocuments.in_introduction-to-ibatis.ppt
fdocuments.in_introduction-to-ibatis.pptfdocuments.in_introduction-to-ibatis.ppt
fdocuments.in_introduction-to-ibatis.pptBruceLee275640
 
Data Handning with Sqlite for Android
Data Handning with Sqlite for AndroidData Handning with Sqlite for Android
Data Handning with Sqlite for AndroidJakir Hossain
 
SQL Server Developer 70-433
SQL Server Developer 70-433SQL Server Developer 70-433
SQL Server Developer 70-433jasonyousef
 
High-Performance Storage Services with HailDB and Java
High-Performance Storage Services with HailDB and JavaHigh-Performance Storage Services with HailDB and Java
High-Performance Storage Services with HailDB and Javasunnygleason
 

Similaire à SQLFire Webinar (20)

Solving performance problems in MySQL without denormalization
Solving performance problems in MySQL without denormalizationSolving performance problems in MySQL without denormalization
Solving performance problems in MySQL without denormalization
 
Akiban Technologies: Renormalize
Akiban Technologies: RenormalizeAkiban Technologies: Renormalize
Akiban Technologies: Renormalize
 
Akiban Technologies: Renormalize
Akiban Technologies: RenormalizeAkiban Technologies: Renormalize
Akiban Technologies: Renormalize
 
NewSQL - Deliverance from BASE and back to SQL and ACID
NewSQL - Deliverance from BASE and back to SQL and ACIDNewSQL - Deliverance from BASE and back to SQL and ACID
NewSQL - Deliverance from BASE and back to SQL and ACID
 
NewSQL Database Overview
NewSQL Database OverviewNewSQL Database Overview
NewSQL Database Overview
 
android sqlite
android sqliteandroid sqlite
android sqlite
 
SQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerSQL and NoSQL in SQL Server
SQL and NoSQL in SQL Server
 
Why you should(n't) run your databases in the cloud
Why you should(n't) run your databases in the cloudWhy you should(n't) run your databases in the cloud
Why you should(n't) run your databases in the cloud
 
SQL 3.pptx
SQL 3.pptxSQL 3.pptx
SQL 3.pptx
 
Sql no sql
Sql no sqlSql no sql
Sql no sql
 
MySQL New Features -- Sunshine PHP 2020 Presentation
MySQL New Features -- Sunshine PHP 2020 PresentationMySQL New Features -- Sunshine PHP 2020 Presentation
MySQL New Features -- Sunshine PHP 2020 Presentation
 
Building scalable application with sql server
Building scalable application with sql serverBuilding scalable application with sql server
Building scalable application with sql server
 
Day2
Day2Day2
Day2
 
NoSQL
NoSQLNoSQL
NoSQL
 
My sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlMy sql crashcourse_intro_kdl
My sql crashcourse_intro_kdl
 
Rapid SQL Datasheet - The Intelligent IDE for SQL Development
Rapid SQL Datasheet - The Intelligent IDE for SQL DevelopmentRapid SQL Datasheet - The Intelligent IDE for SQL Development
Rapid SQL Datasheet - The Intelligent IDE for SQL Development
 
fdocuments.in_introduction-to-ibatis.ppt
fdocuments.in_introduction-to-ibatis.pptfdocuments.in_introduction-to-ibatis.ppt
fdocuments.in_introduction-to-ibatis.ppt
 
Data Handning with Sqlite for Android
Data Handning with Sqlite for AndroidData Handning with Sqlite for Android
Data Handning with Sqlite for Android
 
SQL Server Developer 70-433
SQL Server Developer 70-433SQL Server Developer 70-433
SQL Server Developer 70-433
 
High-Performance Storage Services with HailDB and Java
High-Performance Storage Services with HailDB and JavaHigh-Performance Storage Services with HailDB and Java
High-Performance Storage Services with HailDB and Java
 

Dernier

UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 

Dernier (20)

UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 

SQLFire Webinar

  • 1. vFabric SQLFire Fast meets scalable in VMware's new distributed SQL database.
  • 2. Speed Matters. Users demand fast applications and fast websites. The database is the hardest thing to scale.
  • 3. What Is SQLFire? • SQLFire is a memory-optimized, distributed SQL database. • SQLFire satisfies modern application needs by adopting a more scalable design than traditional RDBMS. • SQLFire focuses on Speed, Scale and SQL.
  • 4. SQLFire: Speed, Scale, SQL. Speed Scale SQL • Memory-optimized for • Horizontally scalable. • Familiar SQL interface. maximum speed and • Add or remove nodes at • SQL 92 compliant. minimum latency. any time for more • JDBC and ADO.NET capacity or availability. interfaces.
  • 5. SQLFire Speed Through Memory Optimization. • Memory Optimization – Latency is tracked to the millisecond. – Use memory to get an advantage. • SQLFire can use disk too. – Data can be asynchronously persisted to disk. – Append-only logfile format for best update performance. – Disaster prevention and for larger databases.
  • 6. What’s Really Great About Memory? Disk seeks are 100,000 times slower than memory reads! (10,000,000 ns = 10 ms)
  • 7. Who Needs That Kind Of Latency Anyway? • Major online travel site. • Minimizing page load time a must. – Did all the usual tricks including minimizing server round-trips, etc. • Required all data access to complete in less than 20ms – Always! • Selected a memory-optimized database from VMware.
  • 8. The Things People Do For Speed. • Many architectures have appeared to make data faster. • Many architectures – forMemcached on top of SQL Databases. speeding up data – Read-only slave nodes. have appeared lately. – Clustering / RAC. • AllNoSQL. from – stem • Clear need to overcome traditional database limitations. – Relational databases were not built to serve thousands (or even hundreds) of users at once. – RDBMS were not built for the low latency users expect.
  • 9. Don’t Mask The Problem, Solve It! Browser Tier Browser Tier Application Tier Application Tier Memcache + SQL: SQLFire Multi-data model Horizontally Scalable to overcome the DB. SQL Database
  • 10. What Is Horizontal Scalability? • Software that makes multiple computers appear as one system. • Horizontal scale gives: – Better performance. – More capacity. – Higher availability.
  • 11. SQLFire Horizontal Scalability. • SQLFire is a horizontally-scalable SQL Database. • SQLFire is horizontally scalable from the ground up. – Data inserts. – Key lookup. – Stored procedures / functions. – Joins. – Transactions. – More.
  • 12. SQLFire and SQL. • SQLFire syntax is based on the SQL-92 standard. • SQLFire extensions are to Data Definition Language (DDL), e.g. CREATE TABLE. • SQLFire ships with JDBC and ADO.NET drivers. – Built-in, transparent high availability.
  • 13. COMPARING SQLFIRE AND OTHER DATABASES
  • 14. SQLFire Challenges Traditional DB Design, Not SQL. • Traditional RDBMS: Too much focus on ACID consistency. • Traditional RDBMS: Too much contention for disk.
  • 15. The database world is changing. A lot! • Many new data models (NoSQL) are emerging – Key-value – Column family (inspired by Google BigTable) – Document – Graph • Most focus on making model less rigid than SQL • Consistency model is not ACID Low scale High scale Very high scale STRICT – Full Tunable Eventual ACID (RDBMS) Consistency • Different tradeoffs for different goals
  • 16. SQLFire At A Glance.
  • 17. SQLFire Versus Other SQL Databases. Attribute SQLFire Other SQL DBs DB Interface Standard SQL. Standard SQL. Tunable. Mix of eventual High consistency. Data Consistency consistency and high consistency. Transactions Supported. Very strong support. Scaling Model Scale out, commodity servers. Scale up.
  • 18. SQLFire Versus NoSQL. Attribute NoSQL SQLFire DB Interface Idiosyncratic (i.e. each is custom). Standard SQL. Querying Idiosyncratic or not present. SQL Queries. Data Consistency Tunable, most favor eventual Tunable, favors high consistency. consistency. Transactions Weak or not present. Linearly scalable transaction model. Interface Design Designed for simplicity. Designed for compatibility. Data Model Wide variety of different models. Relational model. Schema Flexibility Focus on extreme flexibility, SQL model, requires DB dynamism. migrations, etc.
  • 19. Illustrating some of SQLFire’s key features. HANDS ON WITH SQLFIRE
  • 20. SQLFire Tables Are Replicated By Default. 1 CREATE TABLE sales SQLFire Node 1 2 (product_id int, store_id int, Replica 3 price float); sales 4 5 6 SQLFire Node 2 7 Replica 8 Best for small and 9 frequently accessed data. 10
  • 21. Partitioned Tables Are Split Among Members. 1 CREATE TABLE sales SQLFire Node 1 2 (product_id int, store_id int, Replica 3 price float) sales Partition 1 4 PARTITION BY 5 COLUMN (product_id); 6 SQLFire Node 2 7 Replica 8 Best for large Partition 2 9 data sets. 10
  • 22. Types Of Partitioning In SQLFire. Type Purpose Example Built-in hashing algorithm Hash Partitioning splits data at random across PARTITION BY COLUMN (customer_id); (Default) available servers. Manually divide data across PARTITION BY LIST (home_state) List servers based on discrete (VALUES (‘CA’, ‘WA’), criteria. VALUES (‘TX’, ‘OK’)); Manually divide data across PARTITION BY RANGE (date) Range servers based on continuous (VALUES BETWEEN ‘2008-01-01’ AND ‘2008-12-31’, criteria. VALUES BETWEEN ‘2009-01-01’ AND ‘2009-12-30’); Fully dynamic division of data Expression based on function execution. PARTITION BY (MONTH(date)); Can use UDFs.
  • 23. Redundancy Increases Availability. 1 CREATE TABLE sales SQLFire Node 1 2 (product_id int, store_id int, Replica 3 price float) sales Partition 1 4 PARTITION BY Partition 2* 5 COLUMN (product_id); 6 REDUNDANCY 1; SQLFire Node 2 7 Replica 8 All data is available Partition 2 9 if Node 1 fails. Partition 1* 10
  • 24. Collocate Data For Fast Joins. 1 CREATE TABLE sales Related data placed SQLFire Node 1 2 (product_id int, store_id int, on the same node. Replica 3 price float) Customer 1 4 PARTITION BY C1 Customer 1 Sales 5 COLUMN (product_id); 6 COLOCATE WITH customers; SQLFire Node 2 7 C2 Replica 8 SQLFire can join Customer 2 9 tables without Customer 2 Sales 10 network hops.
  • 25. Why Collocation Matters. 1 -- Biggest customers in CA. SQLFire Node 1 2 SELECT sum(value) AS total Since we collocated Replica 3 FROM sales, customer customers and sales Customer 1 4 WHERE sales.customerid = this query execute independently Customer 1 Sales 5 customer.id AND on each node. 6 customer.state = “CA” SQLFire Node 2 7 ORDER BY total DESC; Result: The query scales Replica 8 linearly! Customer 2 9 Customer 2 Sales 10
  • 26. Scaling Functions and Stored Procedures. 1 CALL maxSales SQLFire uses data- maxSales on 2 aware routing to local data WITH RESULT PROCESSOR route processing to 3 maxSalesReducer the data. 4 ON TABLE sales; 5 maxSalesReducer 6 7 8 Result Processors 9 give map/reduce maxSales on functionality. local data 10
  • 27. Some SQLFire Features Not Discussed Include… • Server Groups: – Granular control of data placement or restrict data to certain servers. • Disk persistence and overflow. – For additional availability and when using SQLFire as a primary database. • Caching / Operational Data Store. • Transactions. • Indexes. • Much more! Scan Me To Learn More About These Features!
  • 30. SQLFire Releases. • SQLFire Professional 1.0: – Release: December 13, 2011. – Part of the vFabric Advanced suite. (Per VM) – Also available standalone. (Per CPU) – Limit of 2 connected nodes per database. • SQLFire Enterprise 1.0: – Coming 1H 2012. – Unlimited nodes. – Optional SQLFire Enterprise WAN Upgrade for active/active global databases.
  • 31. SQLFire WAN. • Same database in multiple datacenters or the cloud. • Fully active-active with asynchronous replication. • Coming 1H 2012 with SQLFire Enterprise.
  • 32. Download SQLFire Today! • Beta available now on our community site. – http://tinyurl.com/SQLFire • General availability December 2011. – vFabric Suite or standalone. – Free for development up to 3 nodes. • Follow us on Twitter: – http://twitter.com/vfabricsqlfire Scan To Learn More
  • 35. Demo Details • 2 VMs. – ubuntu – livecd • Schema: – Table called record • threadid: int, value int • Each value is validated upon insert. • The validate constraint check is extremely CPU intensive. • Client: 5 threads inserting simultaneously. • Total of 2500 records to be inserted.
  • 36. Demo State 1 Ubuntu Ubuntu (locator) (database) Insert validate (very slow!) Insert Complete
  • 37. Create Table Code: CREATE TABLE record (threadid int, value int CONSTRAINT MY_CK CHECK (validate(value) = 1)) PARTITION BY COLUMN (threadid); Note: The validate function takes a long time to run.
  • 38. Demo State 2 Ubuntu Ubuntu livecd (locator) (database) (database) Insert Note: validate (very slow!) validate (very slow!) No client change! Insert Complete Insert Complete
  • 39. Source Code • Base Directory: – https://github.com/cartershanklin/SQLFireStuff • Java UDF: – https://github.com/cartershanklin/SQLFireStuff/blob/mas ter/java/examples/Validate.java • Jython Test Script – https://github.com/cartershanklin/SQLFireStuff/blob/mas ter/jython/slowInserts.py • Contact: @cshanklin

Notes de l'éditeur

  1. Speed matters a lot these days. The major internet companies have known this for a long time, they find that websites and apps that load quickly and are responsive  are the key to more satisfied users and ultimately to higher revenue.I took a couple of infographics here from a website optimization company called strangeloop and they show that internet companies measure speed in milliseconds and measure in very minute ways how each millisecond contributes to their bottom line.These days everybody is trying to make their applications richer and more interactive while at the same time delivering the great performance users expect.There are a lot of pieces that go into making an app fast, but the hardest problem to solve when you've got a lot of users or a lot of data is making the database fast enough. Databases remain the hardest thing to scale and the main inhibitor for people who want to build feature-rich applications that scale to thousands of users.
  2. Let's talk about how SQLFire helps solve this problem.  SQLFire is a memory-optimized, distributed SQL database. We'll talk more about what these things mean in a minute. But SQLFire is built from the ground up to be fast enough and scalable enough for the most demanding of modern applications. Though SQLFire is a SQL database, it is more scalable because it relaxes some of the constraints used by traditional databases, which we'll also discuss further.
  3. SQLFire is built around the principles of Speed, Scale and SQL.- Speed through memory optimization.  SQLFire's data structures and internal architecture take the view that data is intended to reside mostly in memory rather than on disk. SQLFire can also use disk but does it in a way that doesn't compromise in-memory speed.- Scale meaning SQLFire is built from the ground up to be horizontally scalable, meaning if you need more capacity, whether it's a bigger database or a faster database, you can simply add servers to get more capacity. Later on I'll show a demo that highlights exactly this point.- Then SQL. Love it or hate it, you probably already know SQL, it's by far the most widely used database access interface. SQLFire implements a subset of the SQL-92 standard and ships with JDBC and ADO.NET interfaces. In fact, SQLFire's SQL drivers are designed with horizontal scale in mind, the application doesn't need to be aware of how many servers are in the SQLFire cluster, and if a node in the cluster fails, applications are automatically re-routed to a working node without any disruption to the application.
  4. Let's drill a bit more into each of these. The screenshot  here is from a service called webpagetest.org and tracks every little detail down to the millisecond. Studies have shown that if an application fails to respond within two seconds, users start to get frustrated, and expectations about speed are growing every year, if an application consistently takes 2 seconds to respond, most users are going to start looking for alternatives. User facing applications need to be extremely fast.By optimizing for memory, latency is minimized and speed is maximized. Although SQLFire is memory-optimize it can also use disk, but SQLFire uses disk in a way that doesn't let it become a bottleneck, writing new data and even updates to disk in an append-only logfile format that won't let the disk become a bottleneck.
  5. Key point to make: Mixed workloads of reads, writes and updates, memory really shines.Why is using memory such a big deal? Jeff Dean from Google put forth a very influential set of figures called numbers everyone should know, and these are useful rules of thumb to use when calculating tradeoffs in different approaches. The main thing to note is the huge differences that will arise in reading and writing to memory versus disk. Memory reads and writes will happen in constant time, while disk reads and writes can become unpredictable based on the number of seeks required. Disk-optimized databases really run into trouble when you have an application that needs to do a lot of simultaneous reading, writing and updating, whereas a memory-optimized database can handle any mixture with very fast and very predictable performance.
  6. Does  low latency really matter? When people optimize websites for example a lot of attention is typically given to things like minimizing server round-trips or writing more efficient javascript. What does this have to do with the database?VMware has a customer that found low latency to be an absolute must. This customer is a major online travel site and found that in order to deliver a good, pleasant, modern web experience to their users, all data access needed to be complete within 20ms, giving the rest of the application a comfortable amount of time to operate, and this was in the presence of reads, writes and updates. This webinar is talking about SQLFire, a product we're just releasing in 1.0 right about now, so this customer is not using SQLFire, but they are using another memory-optimized database called GemFire, which shares a lot of the same technology, and follows the same core principles of speed and low latency.So the answer is yes, user-facing applications really do need low latency, and when you've got a lot of data or a lot of users, in-memory architectures are the way to do it.
  7. These days people are implementing a wide variety of architectures to overcome the limitations of traditional databases. It may be sharding or read-only replication, and many people are turning to NoSQL databases hoping for better scale. The problem is that relational databases are built for querying, not really built to support apps exposed to thousands of users who each expect extremely fast response.
  8. So there are a lot of approaches. In this diagram we see a picture of memcache paired with a SQL database, which is a very common pattern.But as useful as these approaches are there is a problem with them, and that problem is these architectures attempt to hide the problem or defer the problem rather than actually solve the problem. In the process they introduce a lot of complexity and they only take you so far.SQLFire makes a different proposition. Instead of a multi-layered database with different data models, have one low latency database that is fast enough and scalable enough for modern online applications, and the result is a far simpler architecture.
  9. What is horizontal scalability anyway? A horizontally scalable system makes multiple computers appear to be one computer as far as clients or users are concerned. With horizontally scalable systems you can add or remove computers at any time. Adding computers gives a number of benefits including better performance, more capacity and higher availability.
  10. And SQLFire is built from the ground up to be horizontally scalable in every respect, including data access, transactions, function execution etc. Some tradeoffs have to be made to achieve this, specifically you need to give up the notion of full ACID consistency, something we'll talk more about, but SQLFire makes it possible to have a truly dynamic and scalable system without needing to give up the familiar SQL interface.
  11.  Speaking of SQL, SQLFire is based on the SQL-92 standard. SQLFire uses extensions to the standard, it has to to truly realize a horizontally scalable SQL database, but these extensions are limited to the so-called data definition language, things like creating tables. Once tables are set up the application uses the tables and doesn't need to know anything about these extensions. So for the most part applications don't need to worry about any nonstandard syntax.
  12. Let's take a moment to compare SQLFire with other databases. First let's take a look at traditional SQL databases. The scalability limits of traditional RDBMS mainly flow from two factors, first an insistence on strict ACID consistency makes it almost impossible to run on more than one computer. These days everybody wants to run on commodity hardware but you quickly hit a limit with traditional RDBMS because you can't scale it out. Second is traditional databases are disk-optimized, their data structures, their optimizations are all with the idea that they need to optimize the relatively slow medium of disk. This is really holding databases back in this era of cheap memory, where terabyte servers can be bought for well under $50,000.
  13. <-- Strict(Full ACID) ----FIFO(tunable) ---- Eventual ---> (Inpired by Amazon dynamo) RDBMS is synonymous with ACID Tunable: ACID transactions is a choice; by default it could be FIFO Eventual: All bets are off ... you may write and read back and get a different answer or multiple answers (netflix example)
  14. Let's turn to the quickly changing world of new databases and NoSQL. There are a lot of new databases out there trying to solve the database problem in their own way, offering all sorts of different data models, and most of them moving away from the notion of strict ACID consistency.
  15. Let's turn now to a hands-on look at some SQLFire features.On the left we're going to have the SQL code you can use in SQLFire and on the right we'll talk about what the code actually does.For starters we'll create a very simple table, in just the same way you would create it in other databases. By default tables in SQLFire are replicated across all nodes in the SQLFire cluster.That means, for one thing, that if a server crashes all the data in that table is still available. This approach is best for small datasets and data that is frequently accessed or used in joins.
  16. Partitioning data is more sophisticated and more interesting. SQLFire has a keyword, "PARTITION BY", which tells SQLFire that the data in that table should be split up across all available nodes.This approach is a must for large datasets.
  17. There are a lot of different ways to partition data in SQLFire, by default SQLFire will try to evenly distribute data at random across all servers If that's not good enough you can exert a lot of control over how data is divided and distributed using list, range or expression based partitioning.
  18. Partitioning creates a challenge, by default data lives only on one node and if you lose that node the data is offline. We can solve that with the redundancy keyword. Using this causes SQLFire to keep multiple copies of the data on different servers so that if you lose a node, all the data is still available. Redundancy is usually a good idea and you can even keep data in 3 or 4 different servers at once. Most typically you're going to want a redundancy of 1.
  19. Co-location is a key feature that allows SQLFire to be a real SQL database and horizontally scalable at the same time. When I talk to people who know distributed databases they usually ask "how do you do distributed joins?" The answer is, we don't. Instead we allow related data to be grouped together on the same physical node. This is done with the COLOCATE WITH keyword, which associates tables together based on a foreign key and keeps related rows on the same server. In this example we have customer 1 and customer 2 stored on different nodes. The COLOCATE WITH keyword lets me ensure that sales records from customer 1 end up on node 1 and records from customer 2 end up on node 2.
  20. Why does this matter? Later if I need to run a query that joins the data from these tables, the join can run independently on each server in the SQLFire cluster without having to talk to the others. In other words we have linearly scaled the join query by allowing it to run on each node independently. When you talk about a SQL database built from the ground up to be horizontally scalable, these are the sorts of features you can't live without.
  21. We can also look at functions and stored procedures. when you execute a function, SQLFire uses data-aware routing to execute the function on all the nodes that contains the relevant data. And in addition to that you can execute the job across all the nodes and if you need a result you can process results in a RESULT PROCESSOR. In this way you get a map/reduce like model which allows you to take full advantage of your horizontally scaled system in a SQL way.Good to point out that functions are in pure Java.