presto适配arm环境

安装jdk1.8

Presto在java17及更高版本中编译会出现如下报错，需要安装jdk1.8

链接: jdk1.8

[ERROR] Failed to execute goal com.github.spotbugs:spotbugs-maven-plugin:3.1.10:spotbugs (spotbugs) on project hadoop-apache2: Execution spotbugs of goal com.github.spotbugs:spotbugs-maven-plugin:3.1.10:spotbugs failed: Unable to load the mojo 'spotbugs' in the plugin 'com.github.spotbugs:spotbugs-maven-plugin:3.1.10'. A required class is missing: Could not initialize class org.codehaus.groovy.vmplugin.v7.Java7

cd /usr/local
tar -zxvf jdk-8u361-linux-aarch64.tar.gz

#vim /etc/profileexport JAVA_HOME=/usr/local/jdk1.8
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$PATHsource /etc/profile

java -version

安装maven3

链接: apache-maven-3.8.6-bin.tar.gz

cd /usr/local
tar -zxvf apache-maven-3.8.6-bin.tar.gz

vim /etc/profileexport MAVEN_HOME=/usr/local/apache-maven-3.8.6
export MAVEN_OPTS="-Xss64m -Xmx2g -XX:ReservedCodeCacheSize=1g"
export PATH=$MAVEN_HOME/bin:$PATHsource /etc/profile

mvn -version

安装protobuf 2.5

presto适配arm环境需要presto-hadoop-apache2，presto-hadoop-apache2的编译需要在protobuf 2.5版本中执行。protobuf2.5版本官方没有适配arm环境，需要通过第三方patch打补丁后安装。

参考: ARM64 系统中编译安装Hadoop · Issue #12 · liusheng/liusheng.github.io · GitHub

protobuf 2.5
ARM64 系统中编译安装Hadoop · Issue #12 · liusheng/liusheng.github.io · GitHub
yum install -y build-essential autoconf automake libtool cmake zlib1g-dev pkg-config libssl-dev libsasl2-dev
wget -O protobuf-2.5.0-arm64.patch https://git.io/JeObp
wget https://github.com/protocolbuffers/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.bz2#因为缺少bzip包
yum install -y bzip2
tar xvfj protobuf-2.5.0.tar.bz2#因为缺少patch包
cd protobuf-2.5.0
yum -y install patch
patch -p1 < ../protobuf-2.5.0-arm64.patch./autogen.sh
./configure --prefix=/usr
#报错configure: error: C++ preprocessor "/lib/cpp" fails sanity check
yum install gcc-c++
make
make check
make install
protoc --version

编译presto-hadoop-apache2

git clone https://github.com/prestodb/presto-hadoop-apache2.git

需要最新版：presto-hadoop-apache2:2.7.4-12-SNAPSHOT

mvn install

编译报错需要检查jdk版本是否过高 或者跳过测试：

mvn install -DskipTests

编译后可以在本地仓库找到对应的包：

#pwd
/root/.m2/repository/com/facebook/presto/hadoop/hadoop-apache2

编译presto

参考: Support ARM in Presto by ChunxuTang · Pull Request #18960 · prestodb/presto (github.com)
下载presto源代码压缩包解压:

#https://github.com/prestodb/presto
tar -zxvf presto-0.280.tar.gz 

pom.xml添加

            <dependency><groupId>org.fusesource.jansi</groupId><artifactId>jansi</artifactId><version>1.8</version></dependency>

            <plugin><groupId>pl.project13.maven</groupId><artifactId>git-commit-id-plugin</artifactId><configuration><failOnNoGitDirectory>false</failOnNoGitDirectory><runOnlyOnce>true</runOnlyOnce><injectAllReactorProjects>true</injectAllReactorProjects><offline>true</offline><useNativeGit>true</useNativeGit></configuration></plugin>

修改：版本与presto-hadoop-apache2对应

            <dependency><groupId>com.facebook.presto.hadoop</groupId><artifactId>hadoop-apache2</artifactId><version>2.7.4-12-SNAPSHOT</version></dependency>

presto-cli/pom.xml添加

        <dependency><groupId>org.fusesource.jansi</groupId><artifactId>jansi</artifactId></dependency>

            <plugin><groupId>org.basepom.maven</groupId><artifactId>duplicate-finder-maven-plugin</artifactId><configuration><ignoredDependencies><dependency><groupId>org.fusesource.jansi</groupId><artifactId>jansi</artifactId></dependency></ignoredDependencies></configuration></plugin>

修改：presto-main/src/main/java/com/facebook/presto/server/PrestoSystemRequirements.java

            if (!"amd64".equals(osArch) && !"ppc64le".equals(osArch) && !"aarch64".equals(osArch)) {failRequirement("Presto requires amd64 or ppc64le or aarch64 on Linux (found %s)", osArch);

           if (!"x86_64".equals(osArch) && !"aarch64".equals(osArch)) {failRequirement("Presto requires x86_64 or aarch64 on Mac OS X (found %s)", osArch);

编译：

mvn clean package -DskipTests -Dmaven.javadoc.skip=true  -pl '!:presto-docs'