三天三夜的三更半夜! StoneDB on Rocky 9

TL;DR

之前体验过如何安装 StoneDB，并发文演示了几个新特性。关于什么是 StoneDB，请先参阅前文。

StoneDB 初体验 | StoneDB-5.7-v1.0.4 安装及新特性速览

本文记录了这几天在 Rocky Linux 9.2 上编译安装 StoneDB 5.7 最新源码的“扎心”经历。
对 StoneDB、MySQL、RocksDB 或 Rocky 9 感兴趣的同学请继续往下阅读。

环境信息

操作系统为 Rocky Linux 9.2，编译过程中没有出现特别消耗资源的情况，这点不错，我的 VM 是 2c4g，可以编译成功。

[shawnyan@rocky9 ~]$ cat /etc/redhat-release
Rocky Linux release 9.2 (Blue Onyx)
[shawnyan@rocky9 ~]$ uname -a
Linux rocky9.shawnyan.cn 5.14.0-284.30.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Sep 16 09:55:41 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
[shawnyan@rocky9 ~]$
[shawnyan@rocky9 ~]$ gcc --version
gcc (GCC) 11.3.1 20221121 (Red Hat 11.3.1-4)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[shawnyan@rocky9 ~]$ cmake --version
cmake version 3.20.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
[shawnyan@rocky9 ~]$ top
top - 15:28:57 up 12 days,  1:23,  2 users,  load average: 0.01, 0.03, 0.00
Tasks: 272 total,   1 running, 271 sleeping,   0 stopped,   0 zombie
%Cpu0  :  0.0 us,  0.3 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.3 hi,  0.3 si,  0.0 st
%Cpu1  :  0.0 us,  0.3 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   3623.0 total,    818.1 free,    947.4 used,   2131.4 buff/cache
MiB Swap:   4020.0 total,   3158.1 free,    861.9 used.   2675.6 avail Mem

查看 StoneDB 的安装路径和版本号

先上成果物，看看编译安装后的 StoneDB 目录下都有哪些文件。

安装路径是默认值，安装在了 /usr/local/mysql 目录下，查看版本号为 StoneDB-5.7.36。

本地环境的 ntpd 没有同步，所以编译时间有点错乱，快了 8 小时。

创建 Tianmu 测试表

StoneDB 启动后，通过客户端进行连接，\s 查看状态，服务器版本看着有点“凌乱”，不用在意这种细节。

查看默认数据库，与前文描述一致。创建一个测试库，并创建引擎为 TIANMU 的测试表，功能使用正常。

编译配置

相对于前文的丝滑安装，这里的编译过程极其痛苦，犹如工兵步步排雷，好在最终扛到军旗。

源码仓库

StoneDB 提供了 GitHub 和 Gitee 两个代码仓，只是并不同步，如果网路条件允许，还是从 GH 下载比较好。

官网地址解析

官网只能识别 https，无法识别 http，好在vx群里有好人帮忙确认官网状态正常，感谢。

Rocksdb 下载链接

Tianmu 引擎依赖 Rocksdb，如果配置时没有指定包，则会自动下载，只是下载的版本有些“老舅”，且“门牌号”不对。

RocksDB 6.12.6 发行于 2020-10-13，目前，最新版本为 RocksDB 8.6.7 发行于 09/26/2023。

-- Could not find (the correct version of) rocksdb.
-- MySQL currently requires vrocksdb-6.12.6

CMake Error at cmake/rocksdb.cmake:81 (MESSAGE):
  You can download it with -DDOWNLOAD_ROCKSDB=1 -DWITH_ROCKSDB=<directory>

  This CMake script will look for rocksdb in <directory>.  If it is not
  there, it will download and unpack it (in that directory) for you.

  If you are inside a firewall, you may need to use an http proxy:

  export http_proxy=http://example.com:80

Call Stack (most recent call first):
  cmake/rocksdb.cmake:239 (COULD_NOT_FIND_ROCKSDB)
  CMakeLists.txt:315 (INCLUDE)

下载 rocksdb 源码包后，配置正常。

-- Local rocksdb dir /data/stonedb-stonedb-5.7-dev/release/rocksdb-6.12.6
found ROCKSDB_INCLUDE_DIR:/data/stonedb-stonedb-5.7-dev/release/rocksdb-6.12.6
-- Found /data/stonedb-stonedb-5.7-dev/release/rocksdb-6.12.6/include/rocksdb/version.h
-- current ROCKSDB_VERSION_NUMBER is #define ROCKSDB_MAJOR 6.#define ROCKSDB_MINOR 12
-- current ROCKSDB_MAJOR_VERSION is: 6, ROCKSDB_MAJOR_VERSION:12
-- ROCKSDB_INCLUDE_DIR /data/stonedb-stonedb-5.7-dev/release/rocksdb-6.12.6

marisa-trie 下载链接

StoneDB 依赖“玛丽莎”，同上，需要提前下载源码包。

found MARISA_INCLUDE_DIR:MARISA_INCLUDE_DIR-NOTFOUND
-- Looked for include/marisa.h in  and
-- MARISA_INCLUDE_DIR MARISA_INCLUDE_DIR-NOTFOUND
-- LOCAL_MARISA_DIR
-- Could not find (the correct version of) marisa.
-- MySQL currently requires marisa-trie

CMake Error at cmake/marisa.cmake:59 (MESSAGE):
  You can download it, install it, then specify the marisa path with
  -DWITH_MARISA=<director>

  This CMake script will look for marisa in <directory>.
Call Stack (most recent call first):
  cmake/marisa.cmake:123 (COULD_NOT_FIND_MARISA)
  CMakeLists.txt:323 (INCLUDE)

也可以编译时自动下载，只是这里还有个小包袱，git变成了gi。

完整地址为：

https://github.com/s-yata/marisa-trie/archive/refs/tags/v0.2.6.tar.gz

boost 版本升级

StoneDB 5.7 依赖 boost 1.66.0，但编译时报错。

[shawnyan@rocky9 boost_1_66_0]$ ./bootstrap.sh
[shawnyan@rocky9 boost_1_66_0]$ ./b2

...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-gnu-11/release/link-static/threadapi-pthread/threading-multi/converter/builtin_converters.o...
...skipped <pbin.v2/libs/python/build/gcc-gnu-11/release/link-static/threadapi-pthread/threading-multi>libboost_python3.a(clean) for lack of <pbin.v2/libs/python/build/gcc-gnu-11/release/link-static/threadapi-pthread/threading-multi>converter/builtin_converters.o...
...skipped <pbin.v2/libs/python/build/gcc-gnu-11/release/link-static/threadapi-pthread/threading-multi>libboost_python3.a for lack of <pbin.v2/libs/python/build/gcc-gnu-11/release/link-static/threadapi-pthread/threading-multi>converter/builtin_converters.o...
...skipped <pstage/lib>libboost_python3.a for lack of <pbin.v2/libs/python/build/gcc-gnu-11/release/link-static/threadapi-pthread/threading-multi>libboost_python3.a...
...failed updating 2 targets...
...skipped 6 targets...

所以，我就升级到了 boost 1.77.0，并修改了基础标识。

./cmake/boost.cmake:SET(BOOST_PACKAGE_NAME "boost_1_66_0")
./cmake/boost.cmake:SET(BOOST_PATCHES_DIR "${CMAKE_SOURCE_DIR}/include/boost_1_66_0/patches")
./cmake/boost.cmake:SET(BOOST_SOURCES_DIR "${CMAKE_SOURCE_DIR}/include/boost_1_66_0")
./codecov.yml:- "include/boost_1_66_0"

升级后，可以正常进行编译。

...updated 1745 targets...

The Boost C++ Libraries were successfully built!

The following directory should be added to compiler include paths:

    /data/stonedb-stonedb-5.7-dev/release/boost_1_77_0

The following directory should be added to linker library paths:

    /data/stonedb-stonedb-5.7-dev/release/boost_1_77_0/stage/lib

[shawnyan@rocky9 boost_1_77_0]$

openssl 1.1.1

Rocky 9 中通过 dnf 安装的 openssl 为 3.x 版本，版本太新，配置无法识别，而且和 1.1.1 系列代码差异很大，这里下载了 openssl 1.1.1 系列的最新版本 openssl-1.1.1w。

Cannot find appropriate system libraries for WITH_SSL=system.
Make sure you have specified a supported SSL version.
Valid options are :
system (use the OS openssl library),
yes (synonym for system),
</path/to/custom/openssl/installation>

CMake Error at cmake/ssl.cmake:63 (MESSAGE):
  Please install the appropriate openssl developer package.

Call Stack (most recent call first):
  cmake/ssl.cmake:280 (FATAL_SSL_NOT_FOUND_ERROR)
  CMakeLists.txt:630 (MYSQL_CHECK_SSL)

下载后，配置安装 openssl。

./config
make
make install

完整的编译日志

编译终于通关，来看下完整的日志。

[shawnyan@rocky9 release]$ DISABLE_WARNING_AS_ERROR=1 DEBUG_LEVEL=0 make
[  0%] Built target abi_check
[  0%] Built target lz4_lib
[  0%] Built target INFO_SRC
[  0%] Built target INFO_BIN
[  0%] Built target zlib
[  0%] Built target libedit_emacs
[  0%] Built target libedit_vi
[  0%] Built target libedit_common
[  0%] Built target libedit_fcns
[  0%] Built target libedit_help
[  1%] Built target libedit_func
[  3%] Built target edit
[  3%] Built target event_pthreads
[  4%] Built target event_core
[  4%] Built target event_openssl
[  4%] Built target event_extra
[  7%] Built target strings
[  7%] Built target gen_dtrace_header
[ 11%] Built target mysys
[ 11%] Built target dbug
[ 12%] Built target vio
[ 12%] Built target regex
[ 12%] Built target base64_test
[ 12%] Built target queues
[ 13%] Built target thr_lock
[ 14%] Built target mysys_ssl
[ 14%] Built target comp_err
[ 14%] Built target GenError
[ 15%] Built target clientlib
[ 15%] Built target libmysql
[ 15%] Built target libmysql_api_test
[ 15%] Built target mysqlclient
[ 15%] Built target authentication_ldap_sasl_client
[ 15%] Built target binlogevents_static
[ 15%] Built target binlogstandalone_shared
[ 16%] Built target binlogstandalone_static
[ 16%] Built target archive
[ 16%] Built target blackhole
[ 17%] Built target csv
[ 18%] Built target mysqlservices
[ 18%] Built target example
[ 19%] Built target federated
[ 21%] Built target heap
[ 27%] Built target innobase
[ 29%] Built target myisam
[ 31%] Built target myisam_nopfs
[ 32%] Built target myisam_ftdump
[ 32%] Built target myisampack
[ 32%] Built target myisamlog
[ 33%] Built target myisamchk
[ 34%] Built target myisammrg
[ 34%] Built target partition
[ 40%] Built target perfschema
[ 50%] Built target tianmu
[ 51%] Built target test_prog
[ 51%] Built target test_security_context
[ 51%] Built target audit_null
[ 51%] Built target auth
[ 52%] Built target qa_auth_interface
[ 52%] Built target auth_test_plugin
[ 52%] Built target qa_auth_client
[ 52%] Built target qa_auth_server
[ 52%] Built target mysql_no_login
[ 52%] Built target auth_socket
[ 52%] Built target connection_control
[ 52%] Built target daemon_example
[ 52%] Built target ngram_parser
[ 52%] Built target ftexample
[ 52%] Built target keyring_file
[ 52%] Built target validate_password
[ 52%] Built target replication_observers_example
[ 52%] Built target rewrite_example
[ 52%] Built target rewriter
[ 53%] Built target semisync_slave
[ 53%] Built target semisync_master
[ 53%] Built target test_sql_replication
[ 53%] Built target test_sql_sqlmode
[ 54%] Built target test_sql_lock
[ 54%] Built target test_sql_commit
[ 54%] Built target test_x_sessions_init
[ 54%] Built target test_sql_errors
[ 54%] Built target test_x_sessions_deinit
[ 55%] Built target test_session_detach
[ 55%] Built target test_session_info
[ 55%] Built target test_sql_complex
[ 55%] Built target test_sql_all_col_types
[ 55%] Built target test_sql_2_sessions
[ 55%] Built target test_sql_cmds_1
[ 55%] Built target test_sql_shutdown
[ 55%] Built target test_session_in_thd
[ 55%] Built target test_sql_processlist
[ 55%] Built target test_sql_stored_procedures_functions
[ 55%] Built target test_sql_views_triggers
[ 55%] Built target test_services_thread
[ 55%] Built target test_services
[ 55%] Built target test_framework
[ 55%] Built target test_udf_services
[ 55%] Built target version_token
[ 60%] Built target group_replication
[ 60%] Built target keyring_udf
[ 62%] Built target protobuf
[ 64%] Built target protoclib
[ 64%] Built target protoc
[ 66%] Built target mysqlxtest
[ 66%] Built target protobuf-lite
[ 66%] Built target GenLiteProtos
[ 71%] Built target mysqlx
[ 72%] Built target my_print_defaults
[ 73%] Built target perror
[ 73%] Built target lz4_decompress
[ 74%] Built target resolveip
[ 74%] Built target replace
[ 75%] Built target innochecksum
[ 76%] Built target zlib_decompress
[ 76%] Built target resolve_stack_dump
[ 76%] Built target mysql_ssl_rsa_setup
[ 76%] Built target comp_sql
[ 76%] Built target GenSysSchema
[ 76%] Built target GenBootstrapPriv
[ 76%] Built target mysql_install_db
[ 76%] Built target mysql_secure_installation
[ 76%] Built target mysql
[ 76%] Built target mysqltest
[ 76%] Built target GenFixPrivs
[ 76%] Built target mysqlcheck_core
[ 77%] Built target client_base
[ 78%] Built target mysql_upgrade
[ 78%] Built target mysqlcheck
[ 78%] Built target mysqldump
[ 78%] Built target mysqlimport
[ 78%] Built target mysqlshow
[ 78%] Built target mysql_plugin
[ 78%] Built target mysqlbinlog
[ 78%] Built target mysqladmin
[ 79%] Built target mysql_config_editor
[ 79%] Built target mysqlslap
[ 79%] Built target boost_lib
[ 83%] Built target mysqlpump_lib
[ 84%] Built target mysqlpump
[ 84%] Built target mysql_client_test
[ 84%] Built target locking_service
[ 84%] Built target udf_example
[ 86%] Built target binlog
[ 86%] Built target gen_lex_hash
[ 87%] Built target GenServerSource
[ 87%] Built target gen_lex_token
[ 87%] Built target GenDigestServerSource
[ 99%] Built target sql
[ 99%] Built target rpl
[ 99%] Built target master
[100%] Built target slave
[100%] Built target mysqld
[100%] Built target sqlgunitlib
[100%] Built target mysql_tzinfo_to_sql
[shawnyan@rocky9 release]$

安装日志

编译完成，直接进行安装。日志中，可以看到安装了几个二进制命令，以及一些配置文件和脚本。

[shawnyan@rocky9 release]$ sudo make install
[  0%] Built target abi_check
[  0%] Built target lz4_lib
[  0%] Built target INFO_SRC
[  0%] Built target INFO_BIN
[  0%] Built target zlib
...
[100%] Built target mysqld
[100%] Built target sqlgunitlib
[100%] Built target mysql_tzinfo_to_sql
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Installing: /usr/local/mysql/./README
...
-- Installing: /usr/local/mysql/bin/mysql_config
-- Installing: /usr/local/mysql/bin/mysqldumpslow
-- Installing: /usr/local/mysql/bin/mysqld_multi
-- Installing: /usr/local/mysql/bin/mysqld_safe
-- Installing: /usr/local/mysql/my.cnf
-- Installing: /usr/local/mysql/mysql_server
-- Installing: /usr/local/mysql/install.sh
-- Installing: /usr/local/mysql/reinstall.sh
-- Installing: /usr/local/mysql/support-files/mysqld_multi.server
-- Installing: /usr/local/mysql/support-files/mysql-log-rotate
-- Installing: /usr/local/mysql/support-files/magic
-- Installing: /usr/local/mysql/share/aclocal/mysql.m4
-- Installing: /usr/local/mysql/support-files/mysql.server

启动日志

安装完成，对 StoneDB 进行数据初始化，并启动数据库。

[mysql@rocky9 mysql]$ ./bin/mysqld --initialize
2023-11-12T00:29:25.724443Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2023-11-12T00:29:25.936358Z 0 [Warning] InnoDB: New log files created, LSN=45790
2023-11-12T00:29:25.968770Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2023-11-12T00:29:26.055356Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 88cba90a-80f2-11ee-bdca-000c2919b58f.
2023-11-12T00:29:26.056284Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2023-11-12T00:29:26.277207Z 0 [Warning] A deprecated TLS version TLSv1 is enabled. Please use TLSv1.2 or higher.
2023-11-12T00:29:26.277310Z 0 [Warning] A deprecated TLS version TLSv1.1 is enabled. Please use TLSv1.2 or higher.
2023-11-12T00:29:26.277919Z 0 [Warning] CA certificate ca.pem is self signed.
2023-11-12T00:29:26.291107Z 1 [Note] A temporary password is generated for root@localhost: r6K_oTsETP<q
[mysql@rocky9 mysql]$

[mysql@rocky9 mysql]$ ./bin/mysqld
2023-11-12T00:29:54.068726Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2023-11-12T00:29:54.068785Z 0 [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled
2023-11-12T00:29:54.068806Z 0 [Note] ./bin/mysqld (mysqld 5.7.36-StoneDB-) starting as process 290517 ...
2023-11-12T00:29:54.076153Z 0 [Note] InnoDB: PUNCH HOLE support available
2023-11-12T00:29:54.076348Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2023-11-12T00:29:54.076503Z 0 [Note] InnoDB: Uses event mutexes
2023-11-12T00:29:54.076697Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2023-11-12T00:29:54.076756Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2023-11-12T00:29:54.076805Z 0 [Note] InnoDB: Using Linux native AIO
2023-11-12T00:29:54.077154Z 0 [Note] InnoDB: Number of pools: 1
2023-11-12T00:29:54.077581Z 0 [Note] InnoDB: Using CPU crc32 instructions
2023-11-12T00:29:54.080562Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2023-11-12T00:29:54.092507Z 0 [Note] InnoDB: Completed initialization of buffer pool
2023-11-12T00:29:54.095925Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2023-11-12T00:29:54.110502Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2023-11-12T00:29:54.118113Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2023-11-12T00:29:54.118465Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2023-11-12T00:29:54.140182Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2023-11-12T00:29:54.141586Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2023-11-12T00:29:54.141913Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2023-11-12T00:29:54.142898Z 0 [Note] InnoDB: 5.7.36 started; log sequence number 2750173
2023-11-12T00:29:54.143317Z 0 [Note] InnoDB: Loading buffer pool(s) from /usr/local/mysql/data/ib_buffer_pool
2023-11-12T00:29:54.143846Z 0 [Note] Plugin 'FEDERATED' is disabled.
2023-11-12T00:29:54.144584Z 0 [Note] InnoDB: Buffer pool(s) load completed at 231112  8:29:54
2023-11-12T00:29:54.187904Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2023-11-12T00:29:54.188261Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
2023-11-12T00:29:54.188456Z 0 [Warning] A deprecated TLS version TLSv1 is enabled. Please use TLSv1.2 or higher.
2023-11-12T00:29:54.188639Z 0 [Warning] A deprecated TLS version TLSv1.1 is enabled. Please use TLSv1.2 or higher.
2023-11-12T00:29:54.189276Z 0 [Warning] CA certificate ca.pem is self signed.
2023-11-12T00:29:54.189519Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
2023-11-12T00:29:54.190187Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
2023-11-12T00:29:54.190764Z 0 [Note] IPv6 is available.
2023-11-12T00:29:54.190992Z 0 [Note]   - '::' resolves to '::';
2023-11-12T00:29:54.191189Z 0 [Note] Server socket created on IP: '::'.
2023-11-12T00:29:54.202723Z 0 [Note] Event Scheduler: Loaded 1 event
2023-11-12T00:29:54.203592Z 0 [Note] ./bin/mysqld: ready for connections.
Version: '5.7.36-StoneDB-'  socket: '/tmp/mysql.sock'  port: 3306  build-

Title: 三天三夜的三更半夜! StoneDB on Rocky 9

Author: 严少安

Created at: 2023-11-12 02:34:53

Updated at: 2023-11-12 02:34:53

Link: https://shawnyan.cn/2023/stonedb/stonedb-compile-on-rocky-9/

License: This work is licensed under CC BY-NC-SA 4.0.