1、分區(qū)表含義
分區(qū)表定義指根據(jù)可以設(shè)置為任意大小的規(guī)則,跨文件系統(tǒng)分配單個(gè)表的多個(gè)部分。實(shí)際上,表的不同部分在不同的位置被存儲(chǔ)為單獨(dú)的表。用戶所選擇的、實(shí)現(xiàn)數(shù)據(jù)分割的規(guī)則被稱為分區(qū)函數(shù),這在MySQL中它可以是模數(shù),或者是簡(jiǎn)單的匹配一個(gè)連續(xù)的數(shù)值區(qū)間或數(shù)值列表,或者是一個(gè)內(nèi)部HASH函數(shù),或一個(gè)線性HASH函數(shù)。
分表與分區(qū)的區(qū)別在于:分區(qū)從邏輯上來講只有一張表,而分表則是將一張表分解成多張表
2、分區(qū)表優(yōu)點(diǎn)
1)分區(qū)表更容易維護(hù)。對(duì)于那些已經(jīng)失去保存意義的數(shù)據(jù),通??梢酝ㄟ^刪除與那些數(shù)據(jù)有關(guān)的分區(qū),很容易地刪除那些數(shù)據(jù)。相反地,在某些情況下,添加新數(shù)據(jù)的過程又可以通過為那些新數(shù)據(jù)專門增加一個(gè)新的分區(qū),來很方便地實(shí)現(xiàn)。
2)一些查詢可以得到極大的優(yōu)化,這主要是借助于滿足一個(gè)給定WHERE語句的數(shù)據(jù)可以只保存在一個(gè)或多個(gè)分區(qū)內(nèi),這樣在查找時(shí)就不用查找其他剩余的分區(qū)。因?yàn)榉謪^(qū)可以在創(chuàng)建了分區(qū)表后進(jìn)行修改,所以在第一次配置分區(qū)方案時(shí)還不曾這么做時(shí),可以重新組織數(shù)據(jù),來提高那些常用查詢的效率。
3)優(yōu)化查詢。涉及到例如SUM()和COUNT(),可以在多個(gè)分區(qū)上并行處理,最終結(jié)果只需通過總計(jì)所有分區(qū)得到的結(jié)果。
4)通過跨多個(gè)磁盤來分散數(shù)據(jù)查詢,來獲得更大的查詢吞吐量。
3、分區(qū)表限制
1)一個(gè)表最多只能有1024個(gè)分區(qū);
2) MySQL5.1中,分區(qū)表達(dá)式必須是整數(shù),或者返回整數(shù)的表達(dá)式。在MySQL5.5中提供了非整數(shù)表達(dá)式分區(qū)的支持;
3)如果分區(qū)字段中有主鍵或者唯一索引的列,那么多有主鍵列和唯一索引列都必須包含進(jìn)來。即:分區(qū)字段要么不包含主鍵或者索引列,要么包含全部主鍵和索引列;
4)分區(qū)表中無法使用外鍵約束;
5)MySQL的分區(qū)適用于一個(gè)表的所有數(shù)據(jù)和索引,不能只對(duì)表數(shù)據(jù)分區(qū)而不對(duì)索引分區(qū),也不能只對(duì)索引分區(qū)而不對(duì)表分區(qū),也不能只對(duì)表的一部分?jǐn)?shù)據(jù)分區(qū)。
6)分區(qū)鍵必須是INT類型,或者通過表達(dá)式返回INT類型,可以為NULL。唯一的例外是當(dāng)分區(qū)類型為KEY分區(qū)的時(shí)候,可以使用其他類型的列作為分區(qū)鍵(BLOB or TEXT 列除外)
7)如果表中有主鍵和唯一索引,按主鍵字段進(jìn)行分區(qū)時(shí),唯一索引列應(yīng)該包含分區(qū)鍵。
8)目前mysql不支持空間類型和臨時(shí)表類型進(jìn)行分區(qū)。不支持全文索引。
9)對(duì)象限制(分區(qū)表達(dá)式不能出現(xiàn)Stored functions, stored procedures, UDFs, orplugins,Declared variables or user variables.)
10)運(yùn)算限制(支持加減乘等運(yùn)算出現(xiàn)在分區(qū)表達(dá)式,但是運(yùn)算后的結(jié)果必須是一個(gè)INT或者NULL。支持DIV,不支持/,|, , ^, , >>, and ~ 不允許出現(xiàn)在分區(qū)表達(dá)式中)
11)sql_mode限制(官方強(qiáng)烈建議你在創(chuàng)建分區(qū)表后,永遠(yuǎn)別改變mysql的sql_mode。因?yàn)樵诓煌哪J较?,某些函?shù)或者運(yùn)算返回的結(jié)果可能會(huì)不一樣)
12)不支持query_cache和INSERT DELAYED
13)分區(qū)鍵不能是一個(gè)子查詢(即使是子查詢返回的是int值或者null.)
14)子分區(qū)限制(只有RANG和LIST分區(qū)能進(jìn)行子分區(qū)。HASH和KEY分區(qū)不能進(jìn)行子分區(qū)并且子分區(qū)必須是HASH 或 KEY類型)
4、分區(qū)類型
1)水平分區(qū)(根據(jù)列屬性按行分)
如:一個(gè)包含十年發(fā)票記錄的表可以被分區(qū)為十個(gè)不同的分區(qū),每個(gè)分區(qū)包含的是其中一年的記錄。
水平分區(qū)的幾種模式:
* Range(范圍):這種模式允許DBA將數(shù)據(jù)劃分不同范圍。
如:可以將一個(gè)表通過年份劃分成三個(gè)分區(qū),80年代(1980's)的數(shù)據(jù),90年代(1990's)的數(shù)據(jù)以及任何在2000年(包括2000年)后的數(shù)據(jù)。
* Hash(哈希):這中模式允許DBA通過對(duì)表的一個(gè)或多個(gè)列的Hash Key進(jìn)行計(jì)算,最后通過這個(gè)Hash碼不同數(shù)值對(duì)應(yīng)的數(shù)據(jù)區(qū)域進(jìn)行分區(qū)。
如:可以建立一個(gè)對(duì)表主鍵進(jìn)行分區(qū)的表。
* Key(鍵值):上面Hash模式的一種延伸,這里的Hash Key是MySQL系統(tǒng)產(chǎn)生的。
* List(預(yù)定義列表):這種模式允許系統(tǒng)通過DBA定義的列表的值所對(duì)應(yīng)的行數(shù)據(jù)進(jìn)行分割。例如:DBA建立了一個(gè)橫跨三個(gè)分區(qū)的表,分別根據(jù)2004年2005年和2006年值所對(duì)應(yīng)的數(shù)據(jù)。
* Columns分區(qū)是對(duì)range,list分區(qū)的補(bǔ)充,彌補(bǔ)了后兩者只支持整型數(shù)分區(qū)(或者通過轉(zhuǎn)換為整型數(shù)),使得支持?jǐn)?shù)據(jù)類型增加很多(所有整數(shù)類型,日期時(shí)間類型,字符類型),還支持多列分區(qū)。
注:在多列分區(qū)表上插入數(shù)據(jù)時(shí),采用元組的比較,即多列排序,先根據(jù)field1排序,再根據(jù)field2排序,根據(jù)排序結(jié)果來來分區(qū)存儲(chǔ)數(shù)據(jù)。
* Composite(復(fù)合模式):以上模式的組合使用。
如:在初始化已經(jīng)進(jìn)行了Range范圍分區(qū)的表上,可以對(duì)其中一個(gè)分區(qū)再進(jìn)行hash哈希分區(qū)。
垂直分區(qū)(按列分):
如:一個(gè)包含了大text和BLOB列的表,這些text和BLOB列又不經(jīng)常被訪問,可以把這些不經(jīng)常使用的text和BLOB劃分到另一個(gè)分區(qū),在保證它們數(shù)據(jù)相關(guān)性的同時(shí)還能提高訪問速度。
注意:子分區(qū)(關(guān)鍵字subparttition):使用RANGE或LIST分區(qū)可以再次分割形成子分區(qū),子分區(qū)可以是HASH分區(qū)或者KEY分區(qū)。建議在多磁盤上使用。
查看是否有支持Partition分區(qū)表
mysql> SHOW PLUGINS ;
+----------------------------+----------+--------------------+---------+---------+
| Name | Status | Type | Library | License |
+----------------------------+----------+--------------------+---------+---------+
| partition | ACTIVE | STORAGE ENGINE | NULL | GPL |
+----------------------------+----------+--------------------+---------+---------+
或使用
mysql> SELECT PLUGIN_NAME as Name, PLUGIN_VERSION as Version, PLUGIN_STATUS as Status
-> FROM INFORMATION_SCHEMA.PLUGINS
-> WHERE PLUGIN_TYPE='STORAGE ENGINE';
注意:MySQL 5.6.1 之前的版本,可以下命令查看 have_partitioning 參數(shù),新的版本已移除該參數(shù)。
mysql> SHOW VARIABLES LIKE '%partition%';
5、實(shí)戰(zhàn)常用分區(qū)表幾種模式
1)使用RANGE分區(qū)模式
####創(chuàng)建測(cè)試表t1,并插入接近400萬行數(shù)據(jù),再?zèng)]有分區(qū)的情況下,對(duì)查詢某一條件耗時(shí)
mysql> CREATE TABLE `t1` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '表主鍵',
`pid` int(10) unsigned NOT NULL COMMENT '產(chǎn)品ID',
`price` decimal(15,2) NOT NULL COMMENT '單價(jià)',
`num` int(11) NOT NULL COMMENT '購(gòu)買數(shù)量',
`uid` int(10) unsigned NOT NULL COMMENT '客戶ID',
`atime` datetime NOT NULL COMMENT '下單時(shí)間',
`utime` int(10) unsigned NOT NULL DEFAULT 0 COMMENT '修改時(shí)間',
`isdel` tinyint(4) NOT NULL DEFAULT '0' COMMENT '軟刪除標(biāo)識(shí)',
PRIMARY KEY (`id`,`atime`)
)
INSERT INTO t1(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,CURRENT_TIMESTAMP());
INSERT INTO t1(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,'2016-05-01 00:00:00');
INSERT INTO t1(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,'2017-05-01 00:00:00');
INSERT INTO t1(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,'2018-05-01 00:00:00');
INSERT INTO t1(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89756,'2015-05-01 00:00:00');
INSERT INTO t1(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89756,'2016-05-01 00:00:00');
INSERT INTO t1(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89756,'2017-05-01 00:00:00');
INSERT INTO t1(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89756,'2018-05-01 00:00:00');
/**********************************主從復(fù)制大量數(shù)據(jù)******************************/
mysql> INSERT INTO `t1`(`pid`,`price`,`num`,`uid`,`atime`) SELECT `pid`,`price`,`num`,`uid`,`atime` FROM `t1`;
mysql> SELECT * FROM `t1` WHERE `uid`=89757 AND `atime` CURRENT_TIMESTAMP();
1048576 rows in set (5.62 sec) #沒有分區(qū)表情況耗時(shí)5.62s
如果是針對(duì)已有的表進(jìn)行表分區(qū),可以使用ALTER TABLE來進(jìn)行更改表為分區(qū)表,這個(gè)操作會(huì)創(chuàng)建一個(gè)分區(qū)表,然后自動(dòng)進(jìn)行數(shù)據(jù)copy然后刪除原表。
注: 這種會(huì)使服務(wù)器資源消耗比較大(400多萬數(shù)據(jù)要1分多鐘)
mysql> ALTER TABLE t1 PARTITION BY RANGE (YEAR(atime))
-> (
-> PARTITION p0 VALUES LESS THAN (2016),
-> PARTITION p1 VALUES LESS THAN (2017),
-> PARTITION p2 VALUES LESS THAN (2018),
-> PARTITION p3 VALUES LESS THAN MAXVALUE );
Query OK, 4194304 rows affected (1 min 8.32 sec)
mysql> EXPLAIN PARTITIONS SELECT * FROM `t1`; #查看分區(qū)情況
+----+-------------+-------+-------------+------+---------------+------+---------+------+---------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+-------------+------+---------------+------+---------+------+---------+----------+-------+
| 1 | SIMPLE | t1 | p0,p1,p2,p3 | ALL | NULL | NULL | NULL | NULL | 4180974 | 100.00 | NULL |
+----+-------------+-------+-------------+------+---------------+------+---------+------+---------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
同樣用上面的查詢測(cè)試結(jié)果
mysql> SELECT * FROM `t1` WHERE `uid`=89757 AND `atime` CURRENT_TIMESTAMP();
1048576 rows in set (4.46 sec) #與上面沒有分區(qū)查詢執(zhí)行的時(shí)間相比少了接近1s
mysql> EXPLAIN PARTITIONS SELECT * FROM `t1` WHERE `uid`=89757 AND `atime` CURRENT_TIMESTAMP(); #查看查詢使用的分區(qū)情況
+----+-------------+-------+------------+------+---------------+------+---------+------+---------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+---------+----------+-------------+
| 1 | SIMPLE | t1 | p0,p1,p2 | ALL | NULL | NULL | NULL | NULL | 3135804 | 3.33 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+---------+----------+-------------+
1 row in set, 2 warnings (0.00 sec)
同時(shí)也要注意,進(jìn)行表分區(qū)以后,mysql存放的數(shù)據(jù)文件夾中該表的存放文件也被拆分為多個(gè)
-rw-r----- 1 mysql mysql 8.7K 2月 14 14:49 t1.frm
-rw-r----- 1 mysql mysql 36M 2月 14 14:50 t1#P#p0.ibd
-rw-r----- 1 mysql mysql 64M 2月 14 14:50 t1#P#p1.ibd
-rw-r----- 1 mysql mysql 92M 2月 14 14:50 t1#P#p2.ibd
-rw-r----- 1 mysql mysql 64M 2月 14 14:50 t1#P#p3.ibd
實(shí)際生產(chǎn)環(huán)境中,大多是采用另外一種方式:新建一個(gè)和原來表一樣的分區(qū)表,然后把數(shù)據(jù)從原表導(dǎo)出,接著導(dǎo)入新表,最后建立普通索引。
mysql> CREATE TABLE `t2` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '表主鍵',
`pid` int(10) unsigned NOT NULL COMMENT '產(chǎn)品ID',
`price` decimal(15,2) NOT NULL COMMENT '單價(jià)',
`num` int(11) NOT NULL COMMENT '購(gòu)買數(shù)量',
`uid` int(10) unsigned NOT NULL COMMENT '客戶ID',
`atime` datetime NOT NULL COMMENT '下單時(shí)間',
`utime` int(10) unsigned NOT NULL DEFAULT 0 COMMENT '修改時(shí)間',
`isdel` tinyint(4) NOT NULL DEFAULT '0' COMMENT '軟刪除標(biāo)識(shí)',
PRIMARY KEY (`id`,`atime`)
)
PARTITION BY RANGE COLUMNS(atime) (
PARTITION p0 VALUES LESS THAN ('2016-01-01'),
PARTITION p1 VALUES LESS THAN ('2016-02-01'),
PARTITION p2 VALUES LESS THAN ('2016-03-01'),
PARTITION p3 VALUES LESS THAN ('2016-04-01'),
PARTITION p4 VALUES LESS THAN ('2016-05-01'),
PARTITION p5 VALUES LESS THAN ('2016-06-01'),
PARTITION p6 VALUES LESS THAN ('2016-07-01'),
PARTITION p7 VALUES LESS THAN ('2016-08-01'),
PARTITION p8 VALUES LESS THAN ('2016-09-01'),
PARTITION p9 VALUES LESS THAN ('2016-10-01'),
PARTITION p10 VALUES LESS THAN ('2016-11-01'),
PARTITION p11 VALUES LESS THAN ('2016-12-01'),
PARTITION p12 VALUES LESS THAN ('2017-01-01'),
PARTITION p13 VALUES LESS THAN ('2017-02-01'),
PARTITION p14 VALUES LESS THAN ('2017-03-01'),
PARTITION p15 VALUES LESS THAN ('2017-04-01'),
PARTITION p16 VALUES LESS THAN ('2017-05-01'),
PARTITION p17 VALUES LESS THAN ('2017-06-01'),
PARTITION p18 VALUES LESS THAN ('2017-07-01'),
PARTITION p19 VALUES LESS THAN ('2017-08-01'),
PARTITION p20 VALUES LESS THAN ('2017-09-01'),
PARTITION p21 VALUES LESS THAN ('2017-10-01'),
PARTITION p22 VALUES LESS THAN ('2017-11-01'),
PARTITION p23 VALUES LESS THAN ('2017-12-01'),
PARTITION p24 VALUES LESS THAN ('2018-01-01'),
PARTITION p25 VALUES LESS THAN ('2018-02-01'),
PARTITION p26 VALUES LESS THAN ('2018-03-01'),
PARTITION p27 VALUES LESS THAN ('2018-04-01'),
PARTITION p28 VALUES LESS THAN ('2018-05-01'),
PARTITION p29 VALUES LESS THAN ('2018-06-01'),
PARTITION p30 VALUES LESS THAN ('2018-07-01'),
PARTITION p31 VALUES LESS THAN ('2018-08-01'),
PARTITION p32 VALUES LESS THAN ('2018-09-01'),
PARTITION p33 VALUES LESS THAN ('2018-10-01'),
PARTITION p34 VALUES LESS THAN ('2018-11-01'),
PARTITION p35 VALUES LESS THAN ('2018-12-01'),
PARTITION p36 VALUES LESS THAN MAXVALUE
);
注:表主鍵只有id,而分區(qū)字段是atime, 這里主鍵要修改為 id,stsdate 聯(lián)合主鍵,分區(qū)表要求分區(qū)字段要是主鍵或者是主鍵的一部分!!!
mysql> EXPLAIN PARTITIONS SELECT * FROM `t2`\G;
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t2
partitions: p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20,p21,p22,p23,p24,p25,p26,p27,p28,p29,p30,p31,p32,p33,p34,p35,p36
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 1
filtered: 100.00
Extra: NULL
1 row in set, 2 warnings (0.00 sec)
*******************************************插入數(shù)據(jù)*************************************************
INSERT INTO `t2`(`pid`,`price`,`num`,`uid`,`atime`) SELECT `pid`,`price`,`num`,`uid`,`atime` FROM `t1`;
Query OK, 4194304 rows affected (1 min 18.54 sec)
Records: 4194304 Duplicates: 0 Warnings: 0
或采用導(dǎo)出數(shù)據(jù)再導(dǎo)入數(shù)據(jù),可再添加索引
mysqldump -u dbname -p --no-create-info dbname t2 > t2.sq
修改表名,導(dǎo)入數(shù)據(jù),測(cè)試下ok,刪除原來的表。
2)使用LIST分區(qū)模式(如果原表存在主鍵強(qiáng)烈創(chuàng)建新表時(shí),把原主鍵和要分區(qū)字段作為聯(lián)合主鍵一并創(chuàng)建)
mysql> CREATE TABLE `tb01` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '表主鍵',
`pid` int(10) unsigned NOT NULL COMMENT '產(chǎn)品ID',
`price` decimal(15,2) NOT NULL COMMENT '單價(jià)',
`num` int(11) NOT NULL COMMENT '購(gòu)買數(shù)量',
`uid` int(10) unsigned NOT NULL COMMENT '客戶ID',
`atime` datetime NOT NULL COMMENT '下單時(shí)間',
`utime` int(10) unsigned NOT NULL DEFAULT 0 COMMENT '修改時(shí)間',
`isdel` tinyint(4) NOT NULL DEFAULT '0' COMMENT '軟刪除標(biāo)識(shí)',
PRIMARY KEY (`id`,`num`)
);
*****************************插入測(cè)試數(shù)據(jù)******************************************************
INSERT INTO `tb01`(`pid`,`price`,`num`,`uid`,`atime`) SELECT `pid`,`price`,`num`,`uid`,`atime` FROM `tb`;
Query OK, 3145728 rows affected (46.26 sec)
Records: 3145728 Duplicates: 0 Warnings: 0
mysql> ALTER TABLE tb01 PARTITION BY LIST(num)
(
PARTITION pl01 VALUES IN (1,3),
PARTITION pl02 VALUES IN (2,4),
PARTITION pl03 VALUES IN (5,7),
PARTITION pl04 VALUES IN (6,8),
PARTITION pl05 VALUES IN (9,10)
);
Query OK, 3145728 rows affected (48.86 sec)
Records: 3145728 Duplicates: 0 Warnings: 0
存放mysql數(shù)據(jù)文件中生成,以下文件
-rw-r----- 1 mysql mysql 8.7K 2月 15 11:35 tb01.frm
-rw-r----- 1 mysql mysql 56M 2月 15 11:36 tb01#P#pl01.ibd
-rw-r----- 1 mysql mysql 32M 2月 15 11:36 tb01#P#pl02.ibd
-rw-r----- 1 mysql mysql 36M 2月 15 11:36 tb01#P#pl03.ibd
-rw-r----- 1 mysql mysql 36M 2月 15 11:36 tb01#P#pl04.ibd
-rw-r----- 1 mysql mysql 52M 2月 15 11:36 tb01#P#pl05.ibd
mysql> EXPLAIN PARTITIONS SELECT * FROM `tb01`;
+----+-------------+-------+--------------------------+------+---------------+------+---------+------+---------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+--------------------------+------+---------------+------+---------+------+---------+----------+-------+
| 1 | SIMPLE | tb01 | pl01,pl02,pl03,pl04,pl05 | ALL | NULL | NULL | NULL | NULL | 3136392 | 100.00 | NULL |
+----+-------------+-------+--------------------------+------+---------------+------+---------+------+---------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
3)COLUMNS分區(qū)
創(chuàng)建多列分區(qū)表tb02,這里兩列都不是聯(lián)合主鍵
mysql> CREATE TABLE tb02(
-> a int not null,
-> b int not null
-> )
-> PARTITION BY RANGE COLUMNS(a,b)(
-> partition p0 values less than(0,10),
-> partition p1 values less than(10,20),
-> partition p2 values less than(10,30),
-> partition p3 values less than(maxvalue,maxvalue)
-> );
mysql> EXPLAIN PARTITIONS SELECT * FROM `tb02`; #查看
+----+-------------+-------+-------------+------+---------------+------+---------+------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+-------------+------+---------------+------+---------+------+------+----------+-------+
| 1 | SIMPLE | tb02 | p0,p1,p2,p3 | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
+----+-------------+-------+-------------+------+---------------+------+---------+------+------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
mysql> insert into tb02 values (11,13); #手工插入測(cè)試數(shù)據(jù)
Query OK, 1 row affected (0.01 sec)
mysql> select PARTITION_NAME,PARTITION_EXPRESSION,TABLE_ROWS from information_schema.partitions where table_schema=schema() and table_name='tb02';
+----------------+----------------------+------------+
| PARTITION_NAME | PARTITION_EXPRESSION | TABLE_ROWS |
+----------------+----------------------+------------+
| p0 | `a`,`b` | 0 |
| p1 | `a`,`b` | 0 |
| p2 | `a`,`b` | 0 |
| p3 | `a`,`b` | 1 |
+----------------+----------------------+------------+
4 rows in set (0.03 sec)
4)Hase分區(qū)
HASH主要是為了讓數(shù)據(jù)在設(shè)定個(gè)數(shù)的分區(qū)中盡可能分布平均,執(zhí)行哈希分區(qū)時(shí),mysql會(huì)對(duì)分區(qū)鍵執(zhí)行哈希函數(shù),以確定數(shù)據(jù)放在哪個(gè)分區(qū)中。HASH分區(qū)分為常規(guī)HASH分區(qū)和線性HASH分區(qū),前者使用取模算法,后者使用線性2的冪的運(yùn)算規(guī)則。
CREATE TABLE `tb03` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '表主鍵',
`pid` int(10) unsigned NOT NULL COMMENT '產(chǎn)品ID',
`price` decimal(15,2) NOT NULL COMMENT '單價(jià)',
`num` int(11) NOT NULL COMMENT '購(gòu)買數(shù)量',
`uid` int(10) unsigned NOT NULL COMMENT '客戶ID',
`atime` datetime NOT NULL COMMENT '下單時(shí)間',
`utime` int(10) unsigned NOT NULL DEFAULT 0 COMMENT '修改時(shí)間',
`isdel` tinyint(4) NOT NULL DEFAULT '0' COMMENT '軟刪除標(biāo)識(shí)',
PRIMARY KEY (`id`)
)
PARTITION BY HASH(id) partitions 4;
插入2行數(shù)據(jù):
INSERT INTO tb03(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,CURRENT_TIMESTAMP());
INSERT INTO tb03(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,CURRENT_TIMESTAMP());
mysql> explain partitions select * from tb03 where id=1;
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| 1 | SIMPLE | tb03 | p1 | const | PRIMARY | PRIMARY | 4 | const | 1 | 100.00 | NULL |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
mysql> explain partitions select * from tb03 where id=2;
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| 1 | SIMPLE | tb03 | p2 | const | PRIMARY | PRIMARY | 4 | const | 1 | 100.00 | NULL |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
注意:HASH分區(qū)雖然盡可能讓數(shù)據(jù)平均地分布在每個(gè)分區(qū)上,提高了查詢效率,但增加了分區(qū)管理的代價(jià),比如以前有5個(gè)分區(qū),現(xiàn)在要加上一個(gè)分區(qū),算法有mod(expr,5)變成(expr,6),原5個(gè)分區(qū)的數(shù)據(jù)大部分要重新計(jì)算重新分區(qū)。雖然使用線性HASH分區(qū)會(huì)降低分區(qū)管理的代價(jià),但是數(shù)據(jù)卻沒有常規(guī)HASH分布得那么均勻。
5)KEY分區(qū)
KEY分區(qū)類似與HASH分區(qū),但是不能自定義表達(dá)式,不過支持分區(qū)鍵的類型很多,除Text,Blob等文本類型。
CREATE TABLE `tb04` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '表主鍵',
`pid` int(10) unsigned NOT NULL COMMENT '產(chǎn)品ID',
`price` decimal(15,2) NOT NULL COMMENT '單價(jià)',
`num` int(11) NOT NULL COMMENT '購(gòu)買數(shù)量',
`uid` int(10) unsigned NOT NULL COMMENT '客戶ID',
`atime` datetime NOT NULL COMMENT '下單時(shí)間',
`utime` int(10) unsigned NOT NULL DEFAULT 0 COMMENT '修改時(shí)間',
`isdel` tinyint(4) NOT NULL DEFAULT '0' COMMENT '軟刪除標(biāo)識(shí)',
PRIMARY KEY (`id`)
)
PARTITION BY KEY(id) partitions 4;
插入2行數(shù)據(jù):
INSERT INTO tb04(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,CURRENT_TIMESTAMP());
INSERT INTO tb04(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,CURRENT_TIMESTAMP());
#用執(zhí)行任務(wù)查看記錄落在分區(qū)情況
mysql> explain partitions select * from tb04 where id=1;
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| 1 | SIMPLE | tb04 | p0 | const | PRIMARY | PRIMARY | 4 | const | 1 | 100.00 | NULL |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
mysql> explain partitions select * from tb04 where id=2;
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| 1 | SIMPLE | tb04 | p3 | const | PRIMARY | PRIMARY | 4 | const | 1 | 100.00 | NULL |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
6)分區(qū)表管理
建議在生產(chǎn)環(huán)境中盡量不要修改分區(qū),alter會(huì)讀出存在舊表中的數(shù)據(jù),再存入新定義的表中,過程IO將很大,而且全表都會(huì)鎖住。
*1*刪除分區(qū):示例以上面tb01表
--未刪除p05分區(qū)查詢數(shù)據(jù),主要驗(yàn)證當(dāng)刪除分區(qū)數(shù)據(jù)是否被刪除
mysql> select count(1) from tb01 where num=10;
+----------+
| count(1) |
+----------+
| 524288 |
+----------+
1 row in set (0.37 sec)
mysql> alter table tb01 drop partition pl05; #刪除pl05分區(qū),如:一次性刪除多個(gè)分區(qū),alter table tb01 drop partition pl04,pl05;
Query OK, 0 rows affected (0.06 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> select count(1) from tb01 where num=10; #結(jié)果數(shù)據(jù)也被刪除,慎重操作
+----------+
| count(1) |
+----------+
| 0 |
+----------+
1 row in set (0.01 sec)
注意:刪除分區(qū)會(huì)刪除數(shù)據(jù),謹(jǐn)慎操作;不可以刪除hash或者key分區(qū)。
*2*增加分區(qū)
注:新分區(qū)的值不能包含任意一個(gè)現(xiàn)有分區(qū)中值列表中的值,否則報(bào)錯(cuò);新增分區(qū)會(huì)重新整理數(shù)據(jù),原有數(shù)據(jù)不會(huì)丟失。有MAXVALUE值后,直接不能直接加分區(qū),如示例以上面的t1表為例子。
mysql> ALTER TABLE t1 ADD PARTITION (PARTITION P4 VALUES LESS THAN (2018) ) ;
ERROR 1481 (HY000): MAXVALUE can only be used in last partition definition
示例:把tb01上面刪除的pl05分區(qū)添加
mysql> ALTER TABLE tb01 ADD PARTITION(PARTITION pl05 VALUES IN (9,10));
Query OK, 0 rows affected (0.05 sec)
Records: 0 Duplicates: 0 Warnings: 0
*3*分解分區(qū)
注:Reorganize partition關(guān)鍵字可以對(duì)表的部分分區(qū)或全部分區(qū)進(jìn)行修改,并且不會(huì)丟失數(shù)據(jù)。分解前后分區(qū)的整體范圍應(yīng)該一致。
示例:
mysql> create table tb05
-> (dep int,
-> birthdate date,
-> salary int
-> )
-> partition by range(salary)
-> (
-> partition p1 values less than (1000),
-> partition p2 values less than (2000),
-> partition p3 values less than maxvalue
-> );
Query OK, 0 rows affected (0.08 sec)
****插入一條測(cè)試數(shù)據(jù)
mysql> insert tb05 values(1,'2016-03-06',80);
Query OK, 1 row affected (0.01 sec)
mysql>alter table tb05 reorganize partition p1 into(
partition p01 values less than (100),
partition p02 values less than (1000)
); ----不會(huì)丟失數(shù)據(jù)
mysql> explain partitions select * from tb05 where salary=80; #查看已經(jīng)落在新的分區(qū)p01上
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| 1 | SIMPLE | tb05 | p01 | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 2 warnings (0.00 sec)
*4*合并分區(qū)
注:把2個(gè)分區(qū)合并為一個(gè)。
示例:把上面的tb05表中分解的p01和p02合并至p1上
mysql> alter table tb05 reorganize partition p01,p02 into(partition p1 values less than (1000)); --不會(huì)丟失數(shù)據(jù)
Query OK, 0 rows affected (0.05 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> explain partitions select * from tb05 where salary=80;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| 1 | SIMPLE | tb05 | p1 | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 2 warnings (0.00 sec)
*5*重新定義hash分區(qū)表:
RANGE和LIST分區(qū)在重新定義時(shí),只能重新定義相鄰的分區(qū),不可以跳過分區(qū),并且重新定義的分區(qū)區(qū)間必須和原分區(qū)區(qū)間一致,也不可以改變分區(qū)的類型。
示例:
mysql> EXPLAIN PARTITIONS SELECT * FROM `tb03`;
+----+-------------+-------+-------------+------+---------------+------+---------+------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+-------------+------+---------------+------+---------+------+------+----------+-------+
| 1 | SIMPLE | tb03 | p0,p1,p2,p3 | ALL | NULL | NULL | NULL | NULL | 4 | 100.00 | NULL |
+----+-------------+-------+-------------+------+---------------+------+---------+------+------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
mysql> Alter table tb03 partition by hash(id)partitions 8; #不會(huì)丟失數(shù)據(jù)
Query OK, 4 rows affected (0.13 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql> EXPLAIN PARTITIONS SELECT * FROM `tb03`;
+----+-------------+-------+-------------------------+------+---------------+------+---------+------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+-------------------------+------+---------------+------+---------+------+------+----------+-------+
| 1 | SIMPLE | tb03 | p0,p1,p2,p3,p4,p5,p6,p7 | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
+----+-------------+-------+-------------------------+------+---------------+------+---------+------+------+----------+-------+
1 row in set, 2 warnings (0.02 sec)
*6*刪除表的所有分區(qū):
示例:刪除tb03表所有分區(qū)
mysql> Alter table tb03 remove partitioning; #不會(huì)丟失數(shù)據(jù)
Query OK, 4 rows affected (0.07 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql> EXPLAIN PARTITIONS SELECT * FROM `tb03`;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
| 1 | SIMPLE | tb03 | NULL | ALL | NULL | NULL | NULL | NULL | 4 | 100.00 | NULL |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
1 row in set, 2 warnings (0.00 sec)
*7*整理分區(qū)碎片
注:如果從分區(qū)中刪除了大量的行,或者對(duì)一個(gè)帶有可變長(zhǎng)度的行(也就是說,有VARCHAR,BLOB,或TEXT類型的列)作了許多修改,可以使用“ALTER TABLE ... OPTIMIZE PARTITION”來收回沒有使用的空間,并整理分區(qū)數(shù)據(jù)文件的碎片。
ALTER TABLE tb03 optimize partition p1,p2;
*8*分析分區(qū):
讀取并保存分區(qū)的鍵分布。
mysql> ALTER TABLE tb04 CHECK partition p1,p2;
+--------------+-------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+--------------+-------+----------+----------+
| testsms.tb04 | check | status | OK |
+--------------+-------+----------+----------+
1 row in set (0.01 sec)
*9*檢查分區(qū):
可以使用幾乎與對(duì)非分區(qū)表使用CHECK TABLE 相同的方式檢查分區(qū)。這個(gè)命令可以告訴tb04表分區(qū)p1,p2中的數(shù)據(jù)或索引是否已經(jīng)被破壞。如果發(fā)生了這種情況,使用“ALTER TABLE ... REPAIR PARTITION”來修補(bǔ)該分區(qū)。
mysql> ALTER TABLE tb04 CHECK partition p1,p2;
+--------------+-------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+--------------+-------+----------+----------+
| testsms.tb04 | check | status | OK |
+--------------+-------+----------+----------+
1 row in set (0.01 sec)
6、實(shí)際生產(chǎn)簡(jiǎn)單應(yīng)用
場(chǎng)景:之前有個(gè)沒有分區(qū)的大數(shù)據(jù)量表SmsSend(例表,大概2800萬行記錄),統(tǒng)計(jì)過程非常的耗時(shí),考慮用年分區(qū),并且對(duì)歷史數(shù)據(jù)庫進(jìn)行備份,把過去2014年的數(shù)據(jù)轉(zhuǎn)移至新的備份表smssendbak。如在線重定義比較耗時(shí)間,可采用exchange處理!
1)查看當(dāng)前SmsSend表
mysql> SHOW CREATE TABLE SmsSend; #查看創(chuàng)建信息,未進(jìn)行分區(qū)
| SmsSend | CREATE TABLE `SmsSend` (
`Guid` char(36) NOT NULL COMMENT '唯一標(biāo)識(shí)',
`SID` varbinary(85) DEFAULT NULL COMMENT '商家唯一編號(hào)',
`Mobile` longtext NOT NULL COMMENT '接收手機(jī)號(hào)(以","分割)',
`SmsContent` varchar(500) NOT NULL COMMENT '短信內(nèi)容',
`SmsCount` int(11) NOT NULL DEFAULT '1' COMMENT '條數(shù)',
`Status` int(11) NOT NULL COMMENT '當(dāng)前狀態(tài)(0,等待發(fā)送;1,發(fā)送成功;-1,發(fā)送失敗)',
`SendChanelKeyName` varchar(20) DEFAULT NULL COMMENT '發(fā)送通道標(biāo)識(shí)',
`SendTime` datetime NOT NULL COMMENT '發(fā)送成功時(shí)間',
`SendType` int(11) NOT NULL DEFAULT '1' COMMENT '短信發(fā)送類型(1,單發(fā);2,群發(fā))',
`ReceiveTime` datetime DEFAULT NULL COMMENT '接收到回復(fù)報(bào)告的時(shí)間',
`Priority` int(11) NOT NULL DEFAULT '0' COMMENT '優(yōu)先級(jí)',
`UserAccount` varchar(50) DEFAULT NULL COMMENT '操作員',
`ChainStoreGuid` char(36) DEFAULT NULL COMMENT '操作店面唯一標(biāo)識(shí)',
`RelationKey` longtext COMMENT '回復(fù)報(bào)告關(guān)聯(lián)標(biāo)識(shí)',
`Meno` text COMMENT '備注',
`IsFree` bit(1) NOT NULL DEFAULT b'0' COMMENT '是否免費(fèi)'
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
mysql> SELECT COUNT(*) FROM SmsSend; #行記錄
+----------+
| COUNT(*) |
+----------+
| 28259803 |
+----------+
1 row in set (1 min 52.60 sec)
#可得知大數(shù)據(jù)表下在線分區(qū)比較慢并且耗性能
mysql> ALTER TABLE SmsSend PARTITION BY RANGE (YEAR(SendTime))
-> (
-> PARTITION py01 VALUES LESS THAN (2015),
-> PARTITION py02 VALUES LESS THAN (2016),
-> PARTITION py03 VALUES LESS THAN (2017) );
Query OK, 28259803 rows affected (20 min 36.05 sec)
Records: 28259803 Duplicates: 0 Warnings: 0
#查看分區(qū)記錄數(shù)
mysql> select count(1) from SmsSend partition(py01);
+----------+
| count(1) |
+----------+
| 10 |
+----------+
1 row in set (0.00 sec)
mysql> explain partitions select * from SmsSend where SendTime '2015-01-01'; #2014年的數(shù)據(jù)落在第一分區(qū)
+----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
| 1 | SIMPLE | SmsSend | py01 | ALL | NULL | NULL | NULL | NULL | 10 | 33.33 | Using where |
+----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 2 warnings (0.00 sec)
mysql> select count(1) from SmsSend partition(py02);
+----------+
| count(1) |
+----------+
| 10 |
+----------+
1 row in set (0.00 sec)
2)快速創(chuàng)建一個(gè)smssendbak備份表與原SmsSend表結(jié)構(gòu)一致,并刪除備份表所有分區(qū)
mysql> CREATE TABLE smssendbak LIKE SmsSend;
Query OK, 0 rows affected (0.14 sec)
mysql> ALTER TABLE smssendbak REMOVE PARTITIONING;
Query OK, 0 rows affected (0.19 sec)
Records: 0 Duplicates: 0 Warnings: 0
3)使用EXCHANGE PARTITION轉(zhuǎn)移分區(qū)數(shù)據(jù)至備份表,并查看原來表分區(qū)記錄以及新備份表
smssendbak記錄
mysql> ALTER TABLE SmsSend EXCHANGE PARTITION py01 WITH TABLE smssendbak;
Query OK, 0 rows affected (0.13 sec)
mysql> select count(1) from SmsSend partition(py01); #對(duì)比上面原SmsSend表分區(qū)的記錄
+----------+
| count(1) |
+----------+
| 0 |
+----------+
1 row in set (0.00 sec)
mysql> SELECT COUNT(1) FROM smssendbak; #查看新smssendbak備份表轉(zhuǎn)移記錄
+----------+
| COUNT(1) |
+----------+
| 10 |
+----------+
1 row in set (0.00 sec)
*****************測(cè)試使用的表***********************************************************************
創(chuàng)建一個(gè)基礎(chǔ)測(cè)試表:
CREATE TABLE `tb` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '表主鍵',
`pid` int(10) unsigned NOT NULL COMMENT '產(chǎn)品ID',
`price` decimal(15,2) NOT NULL COMMENT '單價(jià)',
`num` int(11) NOT NULL COMMENT '購(gòu)買數(shù)量',
`uid` int(10) unsigned NOT NULL COMMENT '客戶ID',
`atime` datetime NOT NULL COMMENT '下單時(shí)間',
`utime` int(10) unsigned NOT NULL DEFAULT 0 COMMENT '修改時(shí)間',
`isdel` tinyint(4) NOT NULL DEFAULT '0' COMMENT '軟刪除標(biāo)識(shí)',
) ;
插入數(shù)據(jù):
INSERT INTO tb(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,CURRENT_TIMESTAMP());
INSERT INTO tb(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,'2016-05-01 00:00:00');
INSERT INTO tb(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,'2017-05-01 00:00:00');
INSERT INTO tb(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89757,'2018-05-01 00:00:00');
INSERT INTO tb(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89756,'2015-05-01 00:00:00');
INSERT INTO tb(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89756,'2016-05-01 00:00:00');
INSERT INTO tb(`pid`,`price`,`num`,`uid`,`atime`) VALUES(1,12.23,1,89756,'2017-05-01 00:00:00');
************************************插入大量的數(shù)據(jù)(建議百萬以上)*************************************
INSERT INTO `tb`(`pid`,`price`,`num`,`uid`,`atime`) SELECT `pid`,`price`,`num`,`uid`,`atime` FROM `tb`;
****注意,如果要?jiǎng)h除自增長(zhǎng)的主鍵id(修改過程中,建議該庫改為只讀),如下操作:
Alter table tb change id id int(10); #先刪除自增長(zhǎng)
Alter table tb drop primary key;#刪除主建
Alter table tb change id id int not null auto_increment; #如果想重新設(shè)置為自增字段
Alter table tb auto_increment=1; #自增起始
以上就是本文的全部?jī)?nèi)容,希望對(duì)大家的學(xué)習(xí)有所幫助,也希望大家多多支持腳本之家。
您可能感興趣的文章:- MySQL最佳實(shí)踐之分區(qū)表基本類型
- MySQL分區(qū)表的基本入門教程
- MySQL分區(qū)表的最佳實(shí)踐指南
- MySQL分區(qū)表的正確使用方法
- MySQL分區(qū)表的局限和限制詳解
- Mysql分區(qū)表的管理與維護(hù)
- MySQL優(yōu)化之分區(qū)表
- Rails中使用MySQL分區(qū)表一個(gè)提升性能的方法
- mysql使用教程之分區(qū)表的使用方法(刪除分區(qū)表)
- 詳解MySQL分區(qū)表