The document provides an introduction to table partitioning in SQL Server. It explains what a partitioned table is, the key components like partition key, partition function, and partition scheme. It discusses why table partitioning is used, including benefits like partition elimination to improve query performance, ability to backup and restore partitions individually, and perform maintenance tasks like indexing and statistics updates on partitions. It also covers techniques like partition switching which allows fast loading and archiving of data without physically moving it. The document uses examples and diagrams to illustrate these concepts and components of table partitioning.
10. What is a partitioned table?
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
Data is physically stored in groups
of rows called partitions
Each partition can be accessed
and maintained separately
Partitioning is not visible to users,
it behaves like one logical table
11. Partition Key
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
Data is partitioned based on a
single column, the Partition Key
The Partition Key should always be
used as a filter in queries
This ensures Partition Elimination:
only relevant partitions are accessed
12. Partition Function
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
The Partition Function defines how to
partition the data
It specifies boundary values, the points
between two partitions
It specifies if the boundary value
belongs to its left (upper) partition or
its right (lower) partition
14. Partition Function: Range Left and Range Right
Range Left means the boundary value is
the last value in the left partition
CREATE PARTITION FUNCTION
pfLeft (INT) AS RANGE LEFT
FOR VALUES (20,30,40);
Range Right means the boundary value
is the first value in the right partition
CREATE PARTITION FUNCTION
pfRight (INT) AS RANGE RIGHT
FOR VALUES (20,30,40);
...20 21-30 41...31-40 ...19 20-39 40...30-39
20 30 40 20 30 40
15. Partition Scheme
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
The Partition Scheme maps logical
partitions to physical filegroups
Filegroups?
Data files on one or more disks
Backed up and restored individually
Can be Read-Only
18. How are partitions mapped to filegroups?
...20 21-30 41...31-40 ...19 20-39 40...30-39
20 30 40 20 30 40
The partition function specified the boundary values and partitions:
19. How are partitions mapped to filegroups?
CREATE PARTITION SCHEME
psLeft AS PARTITION pfLeft
TO (FG1, FG2, FG3, FG4);
CREATE PARTITION SCHEME
psRight AS PARTITION pfRight
TO (FG1, FG2, FG3, FG4);
...20 21-30 41...31-40 ...19 20-39 40...30-39
20 30 40 20 30 40
The partition scheme uses the partition function...
20. How are partitions mapped to filegroups?
CREATE PARTITION SCHEME
psLeft AS PARTITION pfLeft
TO (FG1, FG2, FG3, FG4);
CREATE PARTITION SCHEME
psRight AS PARTITION pfRight
TO (FG1, FG2, FG3, FG4);
...20 21-30 41...31-40 ...19 20-39 40...30-39
20 30 40 20 30 40
FG1 FG2 FG4FG3 FG1 FG2 FG4FG3
...to map each partition to filegroups:
21. Filegroups
Partition Scheme
Partitioned Table
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
A partitioned table is created on
a partition scheme instead of
directly on a filegroup
The partition scheme uses the
partition key to store rows in the
correct partition and filegroup
based on the definition specified
in the partition function
26. Backup & Restore Partitions
Filegroups can be backed up
and restored individually
If each partition is mapped to
a separate filegroup, partitions
with the most critical data can
be restored first
27. Backup & Restore Partitions
Filegroups can be read-only
If each partition is mapped
to a separate filegroup,
partitions with historical,
unchanging data can be
excluded from regular
backups
28. Index Maintenance per Partition
Rebuild and reorganize indexes per
partition
Rebuild indexes online per partition
was introduced in SQL Server 2014
Set data compression per
partition
ALTER INDEX IndexName
ON TableName
REBUILD PARTITION = 2
WITH (ONLINE = ON);
29. Statistics Maintenance per Partition
Update statistics on specific
partitions instead of scanning
and updating the whole table
UPDATE STATISTICS
TableName (StatisticsName)
WITH RESAMPLE
ON PARTITIONS (3,5);
CREATE STATISTICS
StatisticsName ON
TableName (ColumnName)
WITH INCREMENTAL = ON;
Incremental Statistics was
introduced in SQL Server 2014
30. Load & Archive: Partition Switching
Partitions can be switched between tables,
called switching in or switching out
Partition switching is a metadata operation
that updates the location of the data, no
data is physically moved
Extremely fast compared to inserting
into or deleting from a large table
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
2012-01-01 ... ...
2012-12-31 ... ...
ALTER TABLE Table1
SWITCH PARTITION 5
TO Table2 PARTITION 5;
SELECT
$PARTITION.pf(2014);
31. Load & Archive: Switch out
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
Called switch out when you move data out of a table (archive)
32. Load & Archive: Switch out
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
2012-01-01 ... ...
2012-12-31 ... ...
Called switch out when you move data out of a table (archive)
33. Load & Archive: Switch in
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
2016-01-01 ... ...
2016-12-31 ... ...
Called switch in when you move data into a table (load)
34. Load & Archive: Switch in
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
2016-01-01 ... ...
2016-12-31 ... ...
Called switch in when you move data into a table (load)
35. Sliding Windows
The Sliding Windows technique automates data loading, data archiving
and partition management
It keeps a certain number of partitions by continuously switching out
the oldest data and switching in new data
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2012-01-01 ... ...
2012-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
36. Sliding Windows: Split & Merge
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
Partitions are not actually added or
removed, they are split or merged
Be careful!
Splitting and merging partitions
can cause data movement!!
37. Sliding Windows: Split & Merge
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
Split one partition in two by
adding a new boundary value
ALTER PARTITION FUNCTION pf ()
SPLIT RANGE ('2013-06-01');
Data movement will occur if there is data
on both sides of the new boundary value!
38. Sliding Windows: Split & Merge
2012-01-01 ... ...
2012-12-31 ... ...
2013-01-01 ... ...
2013-12-31 ... ...
2014-01-01 ... ...
2014-12-31 ... ...
2015-01-01 ... ...
2015-12-31 ... ...
Merge two partitions to one by
removing a boundary value
ALTER PARTITION FUNCTION pf ()
MERGE RANGE ('2014-01-01');
Data movement will occur if there is data
on both sides of the old boundary value!
39. Sliding Windows Steps
1. Add new filegroup and file
2. Create switch out (archive) table
3. Create switch in (load) table and load it with new data
4. Split to add new partition
5. Switch in new partition
6. Switch out old partition
7. Merge to remove old partition
8. Delete switch out and switch in tables
9. Delete old file and filegroup
41. Alternative Sliding Windows Steps
1. Add new filegroup and file
2. Create switch out (archive) table
3. Switch out old partition
4. Merge to remove old partition
5. Create switch in (load) table and load it with new data
6. Split to add new partition
7. Switch in new partition
8. Delete switch out and switch in tables
9. Delete old file and filegroup
44. Background: Financial Data Warehouse
Periodic Snapshot Fact Tables
Daily, Weekly and Monthly Balances
SQL Server, DB2, Oracle, .csv, .txt
Loaded at different times during the day
1 2 3 4 5 6 7
Pension Insurance Bank Fund
45. First version
Daily fact table:
Keep 1 day per source
1 2
Pension
6 7
Bank Fund
Monthly fact table:
Keep 36 months per source
3 4 5
Insurance
46. Second version
Daily fact table:
Keep 1 day per source
1 2
Pension
Monthly / Weekly fact table:
Keep 36 months for some sources
Keep 106 weeks for some sources
3 4 5
Insurance
!
6 7
Bank Fund
47. Third version
Daily fact table:
Keep 1 day per source
1 2
Pension
Monthly / Weekly / Daily fact table:
Keep 36 months for some sources
Keep 106 weeks for some sources
Keep 7 days for some sources
3 4 5 6 7
Insurance Bank Fund
!!
50. Daily fact table partitioning
1 ... ...
2 ... ...
3 ... ...
4 ... ...
5 ... ...
6 ... ...
7 ... ...
Partition Key: source number
One partition per source
Helped with deadlock issues
Easy to switch data in and out
51. Monthly / Weekly / Daily fact table partitioning
12011 ... ...
12012 ... ...
12013 ... ...
12014 ... ...
19990 ... ...
19991 ... ...
19992 ... ...
Partition Key: Source number + Period type
Period type:
YYYY for monthly data
9991 for weekly data
9992 for daily data
Helped with deadlock issues, but…
52. Monthly / Weekly / Daily fact table partitioning
12011 ... ...
12012 ... ...
12013 ... ...
12014 ... ...
19990 ... ...
19991 ... ...
19992 ... ...
Partition key was difficult to remember
and was not used in queries
Data had to be inserted into and
deleted from partitions with existing
data, instead of switching partitions in
and out
53. What went wrong?
"The usual suspects": Conflicting priorities, changing
requirements and high pressure to deliver on time
Fact tables were expanded, not changed, when new
sources and requirements were implemented
Partitioning was implemented on fact tables with
multiple grains, instead of correcting the fact tables first
56. References and Resources: Table Partitioning
Brent Ozar Unlimited: SQL Server Table Partitioning Resources
brentozar.com/sql/table-partitioning-resources/
Bradley Ball: Partitioning in SQL Server 2012
pragmaticworks.com/Training/FreeTraining/ViewWebinar/WebinarID/541
57. References and Resources: Further Reading
Partial Backups in SQL Server 2014
msdn.microsoft.com/en-us/library/ms191539.aspx
Piecemeal Restores in SQL Server 2014
msdn.microsoft.com/en-us/library/ms177425.aspx
Benjamin Nevarez: SQL Server 2014 Incremental Statistics
benjaminnevarez.com/2015/02/2014-incremental-statistics/