I am encountering a compilation error when trying to build the pg_bulkload plugin with PolarDB 11. Here’s a detailed description of the steps I followed and the errors encountered
Reproduction Steps:
- Set Up PolarDB Development Environment:
- I set up a PolarDB development environment by compiling and installing PolarDB 11 from the source.Pulled a Docker image for PolarDB development on a familiar OS, such as
Ubuntu 22.04:
docker pull registry.cn-hangzhou.aliyuncs.com/polardb_pg/polardb_pg_devel:ubuntu22.04
- Created and ran a container:
docker run -d -it -P --shm-size=1g --cap-add=SYS_PTRACE --cap-add SYS_ADMIN --privileged=true --name polardb_pg_devel registry.cn-hangzhou.aliyuncs.com/polardb_pg/polardb_pg_devel:ubuntu22.04 bash
- Entered the container:
# 进入容器
docker exec -ti polardb_pg_devel bash
- Cloned PolarDB 11 source code and deployed the PolarDB-PG instance:
# 例如这里拉取 POLARDB_11_STABLE 分支;
# PS: 截止2024.9.24 PolarDB开源的最新分支为: POLARDB_15_STABLE
cd /tmp
git clone -c core.symlinks=true --depth 1 -b POLARDB_11_STABLE https://github.com/ApsaraDB/PolarDB-for-PostgreSQL
# 编译PolarDB 11并初始化实例
cd /tmp/PolarDB-for-PostgreSQL
./polardb_build.sh --without-fbl --debug=off
# 验证PolarDB-PG
psql -c 'SELECT version();'
version
--------------------------------
PostgreSQL 11.9 (POLARDB 11.9)
(1 row)
# 在容器内关闭、启动PolarDB数据库方法如下:
pg_ctl stop -m fast -D ~/tmp_master_dir_polardb_pg_1100_bld
pg_ctl start -D ~/tmp_master_dir_polardb_pg_1100_bld
2.Inside the container, downloaded the pg_bulkload source code:
cd /tmp
git clone -c core.symlinks=true --depth 1 https://github.com/ossc-db/pg_bulkload
- Compile
pg_bulkload:
Attempted to compilepg_bulkload:
cd /tmp/pg_bulkload
USE_PGXS=1 make install
Error Messages:
In file included from /home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/storage/pg_shmem.h:30,
from recovery.c:29:
/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/utils/hsearch.h:78:9: error: unknown type name ‘MemoryContext’
78 | MemoryContext hcxt; /* memory context to use for allocations */
| ^~~~~~~~~~~~~
make[1]: *** [<builtin>: recovery.o] Error 1
make[1]: Leaving directory '/tmp/pg_bulkload/bin'
make: *** [Makefile:27: all] Error 2
The error message should be due to the lack of a header file reference containing memoryContext. After searching the code, I found this definition in utils/palloc.h.
/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/utils/palloc.h
... ...
/*
* Type MemoryContextData is declared in nodes/memnodes.h. Most users
* of memory allocation should just treat it as an abstract type, so we
* do not provide the struct contents here.
*/
typedef struct MemoryContextData *MemoryContext;
... ...
Modified recovery.c to include the missing header:
vi bin/recovery.c
... ...
#include "storage/bufpage.h"
// 新增一个 include :
#include "utils/palloc.h"
#include "storage/pg_shmem.h"
... ...
After fixing the first error, a new error appeared:
cd /tmp/pg_bulkload
USE_PGXS=1 make install
# 报错如下:
writer_binary.c: In function ‘open_output_file’:
writer_binary.c:430:14: error: too few arguments to function ‘BasicOpenFilePerm’
430 | fd = BasicOpenFilePerm(fname, O_WRONLY | O_CREAT | O_EXCL | PG_BINARY,
| ^~~~~~~~~~~~~~~~~
In file included from /home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/utils/sharedtuplestore.h:17,
from /home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/nodes/execnodes.h:27,
from ../include/reader.h:18,
from ../include/binary.h:15,
from writer_binary.c:19:
/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/storage/fd.h:115:17: note: declared here
115 | extern int BasicOpenFilePerm(const char *fileName, int fileFlags, mode_t fileMode, bool polar_vfs);
| ^~~~~~~~~~~~~~~~~
make[1]: *** [<builtin>: writer_binary.o] Error 1
make[1]: Leaving directory '/tmp/pg_bulkload/lib'
make: *** [Makefile:27: all] Error 2
It seems that the BasicOpenFilePerm called by pg_bulkload has been modified by PolarDB and is incompatible with PostgreSQL 11.
How to solve this error?