美文网首页
mysql 字段压缩和表压缩还有其它的

mysql 字段压缩和表压缩还有其它的

作者: 尹楷楷 | 来源:发表于2021-01-11 15:34 被阅读0次

压缩的实质就是降低网络带宽的占用和磁盘存储空间的占用,或者由于主机上存在大数据量的sql操作,而导致主从同步延迟时也可使用压缩,看看能不能缓解延迟。

代价就是要用一定的cpu资源来交换。如果数据库机器的CPU负载已经很高或者网络带宽够用且磁盘空间也充足,就不太建议再开压缩了

mysql可以在 连接、字段、表三个维度上进行压缩

使用压缩协议连接

mysql压缩协议适合的场景是mysql的服务器端和客户端之间传输的数据量很大,或者可用带宽不高的情况,典型的场景有如下两个:
a、查询大量的数据,带宽不够(比如导出数据的时候)

##普通连接
mysql  -hlocalhost -P666 -uroot -p123456 --compress
## 导出数据时
mysqldump -hlocalhost -P666 -uroot -p123456 -default-character-set=utf8  --compress --single-transaction test test > test.sql

b、复制的时候binlog量太大导致主从同步延迟。可以看到,开启slave_compressed_protocol=ON 后,带宽会得到了很大的压缩(节省了2/3的带宽),在跨机房同步的时候,可以避免专线的过高占用。

压缩目标列

写入的时候调用COMPRESS函数对那个列的内容进行压缩,然后存放到对应的列。读取的时候,使用UNCOMPRESSED函数对压缩的内容进行解压缩。适用针对mysql中某个列或者某几个列数据量特别大,一般都是varchar、text、char等数据类型。

mysql的压缩函数COMPRESS压缩一个字符串,然后返回一个二进制串。使用该函数需要mysql服务端支持压缩,否则会返回NULL,压缩字段最好采用varbinary或者blob字段类型保存。使用UNCOMPRESSED函数对压缩过的数据进行解压。注意,采用这种方式需要在业务侧做少量改造。压缩后的内容存储方式如下:a、空字符串就以空字符串存储 b、非空字符串存储方式为前4个bype保存未压缩的字符串,紧接着保存压缩的字符串

压缩
SELECT COMPRESS(REPEAT('a',1000))
解压
select UNCOMPRESS( COMPRESS(REPEAT('a',1000)))
查询压缩情况,判断压缩效率
SELECT UNCOMPRESSED_LENGTH( COMPRESS(REPEAT('a',1000))) '原字符串长度', LENGTH( COMPRESS(REPEAT('a',1000))) '压缩后字符长度', UNCOMPRESS( COMPRESS(REPEAT('a',1000))), COMPRESS(REPEAT('a',1000))

测试下,创建表如下。其中signed_data和timestamp两个字段需要进行压缩

CREATE TABLE `test`.`test`  (
  `id` bigint(0) NOT NULL,
  `base64_source_data` longtext CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  `source_data` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `cert_id` varchar(36) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  `signed_data` longtext CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  `timestamp` longtext CHARACTER SET latin1 COLLATE latin1_swedish_ci NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

执行insert

INSERT INTO `test`(`id`, `base64_source_data`, `source_data`, `cert_id`, `signed_data`, `timestamp`) VALUES (222, 'dnZ2dg==', 'vvvv','16a20401-cb8d-411f-82f1-890c819200b1',  COMPRESS('MIIBiQYKKoEcz1UGAQQCAqCCAXkwggF1AgEDMQwwCgYIKoEcz1UBgxEwFAYKKoEcz1UGAQQCAaAGBAR2dnZ2MYIBSjCCAUYCAQEwcTBiMQswCQYDVQQGEwJDTjEkMCIGA1UECgwbTkVUQ0EgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MS0wKwYDVQQDDCRORVRDQSBTTTIgVEVTVDAxIGFuZCBFdmFsdWF0aW9uIENBMDECCxD1fwFc4urjNhs9MAoGCCqBHM9VAYMRoGowGQYJKoZIhvcNAQkDMQwGCiqBHM9VBgEEAgEwHAYJKoZIhvcNAQkFMQ8XDTIwMTIyNTA4MDMyMlowLwYJKoZIhvcNAQkEMSIEIMldsBXsFf91Z/Wy7ykrKrUutsd/RtiGOn0MAGqO6QhdMA0GCSqBHM9VAYItAQUABEcwRQIhAIiFZZhN9JoFO2n9+0mLU2jPEuUlkdgTgMLw1PsCCodNAiB1qWesqwUkp2DIMETnelEHQq4aC1w4itBkyUJdBOmb3g=='), COMPRESS('MIIO+gYJKoZIhvcNAQcCoIIO6zCCDucCAQMxDzANBglghkgBZQMEAgEFADCBmQYLKoZIhvcNAQkQAQSggYkEgYYwgYMCAQEGCysGAQQBgZJIAQgDMDEwDQYJYIZIAWUDBAIBBQAEIGT4BoD535ygvGqcVsRTvi+g6CHlVx5r6jxxTVmgpqU+AhMHNuhWcIqZHgheG26UEXRnL9YDGBMyMDIwMTIyNTA4MDMyMi4zNzFaAhQ42b6AD5tD6/nKIn9KYHq4sOiwsKCCC+UwggN8MIICZKADAgECAgEBMA0GCSqGSIb3DQEBBQUAME8xCzAJBgNVBAYTAkNOMSQwIgYDVQQKExtORVRDQSBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxGjAYBgNVBAMTEU5FVENBIFJvb3QgQ2xhc3NBMB4XDTAzMDUwODAwMDAwMFoXDTMzMDUwODAwMDAwMFowTzELMAkGA1UEBhMCQ04xJDAiBgNVBAoTG05FVENBIENlcnRpZmljYXRlIEF1dGhvcml0eTEaMBgGA1UEAxMRTkVUQ0EgUm9vdCBDbGFzc0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDLZa8KKj9hF3CK1eSgkat86ZVUfUUdxxaXXH31HSUxFxB6upilPmOWCpkbdYEbWvH/luVCGk90kN6M5IjA6DqW29F1+qdHOMdlhAvO28M9DI8tTqIH/pEQA9CeBN7EGRHrM4JK+fh7afa0KszftO8cgXhOUiQ/9RjDGEPP3SB6H2oSYSlYlQv7xdEHLuJ3ZGqlNPWxIegWmjRRWQPMrr8fe6LJa3UoOvPywdM2QyLb0pa1ZktyOqwjKzYFJMWxL9H+q5tfRterP6+Z8In4tBz9+FLIAzdjOcMcpRlFKSsMVh1Fj9kDvu1W/rDHKWyjJRj0bCrQBXJa7qlP/IARTv+9AgMBAAGjYzBhMB8GA1UdIwQYMBaAFAtK3Dg/vtDWBQ575YdtQn5/jp8rMB0GA1UdDgQWBBQLStw4P77Q1gUOe+WHbUJ+f46fKzAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQUFAAOCAQEAu/zJ5jdN1rP34ynssm46xO9Cfu00G0zX9liXi6gnX3n7PL9aGjqY4lUJBs7J19+T3VC8V9namT7UK3vKDH6++llNQahuGZ8wU7zjVZrJLKtgNfkaLgB5M0k2OuDaEZR8ZDHcykbJGXIUq9JJ6vmV5Dxp2R84Watj4tT+ARpXGIGI2RjvltbEc7+wTPAxiYAe1p74AKF1O6ToSLE1Ypqf7Q1G3qPcexifxJixLgbIOTnP5q5fETZ+YKCFKwGHfZ47FKNiXRzW+zWDA6+UdK9hdSPpy0JWeKBoNUTmYCjrhm8Zr3bAzMJcLO/mKvBVWYGn6eT5u8Up8jllNs/pzT58sjCCBGYwggNOoAMCAQICEHgC8qs+fBVsFVWsUkfPZJUwDQYJKoZIhvcNAQEFBQAwbzELMAkGA1UEBhMCQ04xJDAiBgNVBAoTG05FVENBIENlcnRpZmljYXRlIEF1dGhvcml0eTEZMBcGA1UECxMQU2VydmVyIENsYXNzQSBDQTEfMB0GA1UEAxMWTkVUQ0EgU2VydmVyIENsYXNzQSBDQTAeFw0yMDA5MjEwNjQwMzRaFw0yMTA5MjEwNjQwMzRaMFsxCzAJBgNVBAYTAkNOMSQwIgYDVQQKExtORVRDQSBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxJjAkBgNVBAMTHU5FVENBIFRpbWUgU3RhbXBpbmcgQXV0aG9yaXR5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA22+oHnBK3AQg66rSsqOsG2DBYzuhCEg834nYl0gksYeNUJpEbWjLrsWj71mnFxlg2e164v0D5ondFeMx7T6TUQoABEDu0qbSzhHsPpfAXiwmcvpIbt6KZBaoj/xhRsL8lVEub2dcV1WjBVd1P47e/jGBSs9XWxVVujOGNbgh9/44Gtw2T0tDIaytiDQ+/jVG+X8khKje5p0YKYTs0dc2zHV4M642uN0B4TLv98qqa0LQlhOvAkLTxB40hw3YoZtOnQ72MpiesdEsgljOsYzRU0S3GXWumOWBpX0qHZdkBrxvo6ELXtwUFT/UXbJ/tLH+7H92bQz9uJmA6lw2gtx8LwIDAQABo4IBEDCCAQwwHwYDVR0jBBgwFoAUuvNKBSTm+CTI5lfaeI0MWeRDZMowHQYDVR0OBBYEFOccKHQAJeJZiP/DgdXXN/fwnvYPMFcGA1UdIARQME4wTAYKKwYBBAGBkkgBCjA+MDwGCCsGAQUFBwIBFjBodHRwOi8vd3d3LmNuY2EubmV0L2NzL2tub3dsZWRnZS93aGl0ZXBhcGVyL2Nwcy8wDAYDVR0TAQH/BAIwADAOBgNVHQ8BAf8EBAMCBsAwFgYDVR0lAQH/BAwwCgYIKwYBBQUHAwgwOwYDVR0fBDQwMjAwoC6gLIYqaHR0cDovL2NsYXNzYWNhMS5jbmNhLm5ldC9jcmwvU2VydmVyQ0EuY3JsMA0GCSqGSIb3DQEBBQUAA4IBAQDMC1I6rvoD+0UTv65R0CVxQ2X7RTmcbo5K7FuRReS4cXDF2yLMYsuqn0YMagwixrUyByjG+PxOudGuRWn/nJpI0COZgbG7CZIJZm5LL1Q76kG/GfB/kQuA5APrRkan84hSeCYOaYsYKty0Q2REcaZ3A2FswCVoNVJG90cz57fPy0P/sRlLlYoixseVr0iZ9uuQbfrNzDhcuEm+7lhVFmO7VlomG67eI4MKTB2WBnf9au6lKc8VwHbHwp1/F536SG3xtl8M44olSs9q9XipNTa5lsaZ2eqvOgcIK4CrRUdDUZfJ3ts87g7iyqg9Rv8SFSGMFpD7SHX5qnDqDW+n9+x4MIID9zCCAt+gAwIBAgIBBDANBgkqhkiG9w0BAQUFADBPMQswCQYDVQQGEwJDTjEkMCIGA1UEChMbTkVUQ0EgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MRowGAYDVQQDExFORVRDQSBSb290IENsYXNzQTAeFw0wMzA1MTgwMDAwMDBaFw0yMzA1MTgwMDAwMDBaMG8xCzAJBgNVBAYTAkNOMSQwIgYDVQQKExtORVRDQSBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxGTAXBgNVBAsTEFNlcnZlciBDbGFzc0EgQ0ExHzAdBgNVBAMTFk5FVENBIFNlcnZlciBDbGFzc0EgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDhIkzceY83bbS0FB6tEWM2Zq6QW7fWPJrlDDJeZ0ljA/3Fr8bLn3CzHnrFIZ7SA4C1KNn3qqetpFubv8rdYMI8HcWgiMk6US5HMeJb93IT4VLavW5CNwHbCbApHZodwkt8a5c7J6cYkCFY868U92JrDI3M9o6h0P31jjPtxCFQFX/+DcqQPaidh0aehN2dYsVH05AP+3qUZvhi8CacGN9uDaI2Pu8Wf/pHR+geKDBrnyfnUJx6ZjHCO5jzY2afKIpEqYvpWit/ECOlWp/IH4M8egNWjXX35FYrv9tsvM4ll5frNW96C6aLrmdUb9h8MyNRksUdiMB30GbD58KaUsN7AgMBAAGjgb0wgbowHwYDVR0jBBgwFoAUC0rcOD++0NYFDnvlh21Cfn+OnyswHQYDVR0OBBYEFLrzSgUk5vgkyOZX2niNDFnkQ2TKMA4GA1UdDwEB/wQEAwIBBjBXBgNVHSAEUDBOMEwGCisGAQQBgZJIAQowPjA8BggrBgEFBQcCARYwaHR0cDovL3d3dy5jbmNhLm5ldC9jcy9rbm93bGVkZ2Uvd2hpdGVwYXBlci9jcHMvMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEBAJaea86s48jYSdLutXq5SzUg+7sTrTKttozXipleE9hhy5Y0PzJOeMf1IKajwIuBxD5PGQC29wrsnKmfC7C1I+kkRXSYNguYkkNwzGjpWftvIxYQUjgf9b5zmLgaME1vcidgKE8d91MTFDPh+wTV/bI78c1sWvJGCu/R3sQduVP8bcebUDV5MuW6rGDZSenQdz5Nu2HiKWzxWlmyRt2dUYQwYPkEc9lQWDQXr4yNtSdTEuHNGSw6GjKLYuKvclQFFEOm4Cy6rjwbQMTsvrM/W2mvNOY5Ruzi+ai0m9rzCmLd720dLO5hg0s87C9AkP0u4GWeYVoTAUarQFMuYAWv7nQxggJKMIICRgIBATCBgzBvMQswCQYDVQQGEwJDTjEkMCIGA1UEChMbTkVUQ0EgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MRkwFwYDVQQLExBTZXJ2ZXIgQ2xhc3NBIENBMR8wHQYDVQQDExZORVRDQSBTZXJ2ZXIgQ2xhc3NBIENBAhB4AvKrPnwVbBVVrFJHz2SVMA0GCWCGSAFlAwQCAQUAoIGYMBoGCSqGSIb3DQEJAzENBgsqhkiG9w0BCRABBDAcBgkqhkiG9w0BCQUxDxcNMjAxMjI1MDgwMzIyWjArBgsqhkiG9w0BCRACDDEcMBowGDAWBBSJ1B0Q3CdAu8ANX5//5TnuDaWkyTAvBgkqhkiG9w0BCQQxIgQgqLDNWHRjXCyXRGIZas96ltgxARgT7LssmGC9oEi44+YwDQYJKoZIhvcNAQEBBQAEggEAZNfux6/MiQk2Nzrw+Xn+YOG9ALqpcUEc+qxboje7f57fdcv8tl9WAayr/nhyi87TU7/z/9Cq/kyGtJzpau99eNIPFtXGYbbBv6ahPlFno0PdM6kZvPTn80nmFGBDWGBUwD0NLjXLhpx3nel6wA3eyfGHR6e08DCg6j4dcgO1N5mSq+/pq+I6zt9kqLcNpP3PIZOX4upAZyjuFaPKR5i5VLukOL/Y8ujl8OR3INPpDaTErnP4esYUE0RMdq8zPzsAy33dtPtk//PuuGd4vg/A+AgRgjdCyy2mJmaJB+jTkL8mEDfFk7IOOjoxwtzxuv7yw9agubRLm4P1X4uvyczm1g=='));

进行查询

SELECT UNCOMPRESSED_LENGTH(timestamp) '原字符串长度', LENGTH(timestamp) '压缩后字符串长度',UNCOMPRESS(timestamp)'压缩后大小' from test;
| 原字符串长度 | 压缩后字符串长度 | 压缩后大小
| 5120 | 3334 | MIIO+gYJKoZIhvcNAQcCoIIO6zCCDucCAQMxD...

压缩率0.65,比较高了。针对text、char、varchr、blob等,如果里面重复的数据越多压缩效果就越好。

创建压缩表

创建表时指定ROW_FORMAT=COMPRESSED

CREATE TABLE `book` (
  `id` int NOT NULL AUTO_INCREMENT,
  `user_id` bigint DEFAULT NULL,
  `book_name` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
  `zz` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
  `gg` varchar(255) DEFAULT NULL,
  `mj` enum('one','two','three','4') CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  KEY `index_user_id` (`user_id`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=762475 DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED 

相关文章

  • mysql 字段压缩和表压缩还有其它的

    压缩的实质就是降低网络带宽的占用和磁盘存储空间的占用,或者由于主机上存在大数据量的sql操作,而导致主从同步延迟时...

  • 常用的Shell命令(三)

    文件的压缩和解压缩命令 tar (这是最常用的,其它的还有zip,unzip,rar,unrar,gzip ......

  • MySQL的安装

    MySQL的安装 首先下载MySQL的的压缩包(MySQL社区版) 安装 如果以前安装过其它版本的MySQL那么需...

  • MySQL InnoDB表压缩

    适用于I/O为系统主要瓶颈的应用系统,表压缩可以增加I/O吞吐,但是同时解压缩会耗费cpu资源。所以本质上是以cp...

  • 第11期:压缩表

    一、概念 压缩表从名字上来看,简单理解为压缩后的表,也就是把原始表根据一定的压缩算法按照一定的压缩比率压缩后生成的...

  • MySQL的存储引擎

    MySQL的存储引擎 1.查看所有引擎 将结果列成表格 InnoDB和MyISAM对比 *(MylSAM的压缩表必...

  • 在 windows 10 上安装 解压缩版 MySql

    在 windows 10 上安装 解压缩版 MySql mysql win10 解压缩 下载 解压 Mysql ...

  • 7z压缩

    什么是7z 一种文件压缩格式,具有高压缩比率,进行数据压缩有多种压缩算法可以选择。与其它压缩格式相比,得到的压缩文...

  • 最优秀好用的 BandiZip 中文版 – 免费文件压缩/解压缩

    Bandizip 是一款可靠和快速的压缩软件,它支持WinZip、7-Zip 和 WinRAR 以及其它压缩格式。...

  • Bandizip v6.24 文件解压缩工具软件

    Bandizip 是一款可靠和快速的压缩软件,它支持WinZip、7-Zip 和 WinRAR 以及其它压缩格式。...

网友评论

      本文标题:mysql 字段压缩和表压缩还有其它的

      本文链接:https://www.haomeiwen.com/subject/puhdaktx.html