> 文章列表 > Java GenericObjectPool 对象池化技术--SpringBoot sftp 连接池工具类

Java GenericObjectPool 对象池化技术--SpringBoot sftp 连接池工具类

Java GenericObjectPool 对象池化技术--SpringBoot sftp 连接池工具类

通常一个对象创建、销毁非常耗时的时候,我们不会频繁的创建和销毁它,而是考虑复用。复用对象的一种做法就是对象池,将创建好的对象放入池中维护起来,下次再用的时候直接拿池中已经创建好的对象继续用,这就是池化的思想。

Apache Commons Pool是一个对象池的框架,他提供了一整套用于实现对象池化的API。它提供了三种对象池:GenericKeyedObjectPool,SoftReferenceObjectPool和GenericObjectPool,其中GenericObjectPool是我们最常用的对象池,内部实现也最复杂。

GenericObjectPool

GenericObjectPool 是一个通用对象池框架,我们可以借助它实现一个健壮的对象池,UML图如下所示:

GenericObjectPool 实现了ObjectPool接口,而ObjectPool就是对象池的核心接口,它定义了一个对象池应该实现的行为。

public interface ObjectPool<T> extends Closeable {/* 从池中借走到一个对象*/T borrowObject() throws Exception, NoSuchElementException, IllegalStateException;/* 把对象归还给对象池*/void returnObject(T var1) throws Exception;/* 验证对象的有效性*/void invalidateObject(T var1) throws Exception;/* 往池中添加一个对象*/void addObject() throws Exception, IllegalStateException, UnsupportedOperationException;/* 返回对象池中有多少对象是空闲的,也就是能够被借走的对象的数量。*/int getNumIdle();/* 返回对象池中有对象对象是活跃的,也就是已经被借走的,在使用中的对象的数量。*/int getNumActive();/* 清理对象池。注意是清理不是清空,该方法要求的是,清理所有空闲对象,释放相关资源。*/void clear() throws Exception, UnsupportedOperationException;/* 关闭对象池。这个方法可以达到清空的效果,清理所有对象以及相关资源。*/void close();
}

BasePooledObjectFactory

Java BasePooledObjectFactory 对象池化技术

使用GenericObjectPool只需要创建一个对象工厂类,继承BasePooledObjectFactory并重写它的create()destroyObject()
如下文中的:SftpPool.java

public interface PooledObjectFactory<T> {/* 创建一个可由池提供服务的实例,并将其封装在由池管理的PooledObject中。*/PooledObject<T> makeObject() throws Exception;/*  销毁池不再需要的实例*/void destroyObject(PooledObject<T> var1) throws Exception;/* 确保实例可以安全地由池返回*/boolean validateObject(PooledObject<T> var1);/* 重新初始化池返回的实例*/void activateObject(PooledObject<T> var1) throws Exception;/* 取消初始化要返回到空闲对象池的实例*/void passivateObject(PooledObject<T> var1) throws Exception;
}

配置类GenericObjectPoolConfig

GenericObjectPoolConfig是封装GenericObject池配置的简单“结构”,此类不是线程安全的;它仅用于提供创建池时使用的属性。大多数情况,可以使用GenericObjectPoolConfig提供的默认参数就可以满足日常的需求。

工作原理流程

  1. 构造方法
    当我们执行构造方法时,主要工作就是创建了一个存储对象的LinkedList类型容器,也就是概念意义上的“池”
  2. 从对象池中获取对象
    获取池中的对象是通过borrowObject()命令,源码比较复杂,简单而言就是去LinkedList中获取一个对象,如果不存在的话,要调用构造方法中第一个参数Factory工厂类的makeObject()方法去创建一个对象再获取,获取到对象后要调用validateObject方法判断该对象是否是可用的,如果是可用的才拿去使用。LinkedList容器减一
  3. 归还对象到线程池
    简单而言就是先调用validateObject方法判断该对象是否是可用的,如果可用则归还到池中,LinkedList容器加一,如果是不可以的则调用destroyObject方法进行销毁

上面三步就是最简单的流程,由于取和还的流程步骤都在borrowObject和returnObject方法中固定的,所以我们只要重写Factory工厂类的makeObject()和validateObject以及destroyObject方法即可实现最简单的池的管理控制,通过构造方法传入该Factory工厂类对象则可以创建最简单的对象池管理类。这算是比较好的解耦设计模式,借和还的流程如下图所示:

使用Demo

<dependency><groupId>org.apache.commons</groupId><artifactId>commons-pool2</artifactId><version>2.7.0</version>
</dependency><!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
<dependency><groupId>com.jcraft</groupId><artifactId>jsch</artifactId><version>0.1.55</version>
</dependency>

点击查看代码

application.yaml

server:port: 8088application:name: sftp Demosftp:host: 172.16.3.88 # 服务器ipport: 22 # ssh端口username: root # 用户名password: root # 密码# 连接池参数pool:max-total: 10max-idle: 10min-idle: 5

SftpPoolException.java

package com.vipsoft.sftp.exception;/* sftp连接池异常*/
public class SftpPoolException extends RuntimeException {private static final long serialVersionUID = 1L;/* Constructs a new runtime exception with {@code null} as its* detail message.  The cause is not initialized, and may subsequently be* initialized by a call to {@link #initCause}.*/public SftpPoolException() {}/* Constructs a new runtime exception with the specified detail message.* The cause is not initialized, and may subsequently be initialized by a* call to {@link #initCause}. @param message the detail message. The detail message is saved for*                later retrieval by the {@link #getMessage()} method.*/public SftpPoolException(String message) {super(message);}/* Constructs a new runtime exception with the specified detail message and* cause.  <p>Note that the detail message associated with* {@code cause} is <i>not</i> automatically incorporated in* this runtime exception's detail message. @param message the detail message (which is saved for later retrieval*                by the {@link #getMessage()} method).* @param cause   the cause (which is saved for later retrieval by the*                {@link #getCause()} method).  (A <tt>null</tt> value is*                permitted, and indicates that the cause is nonexistent or*                unknown.)* @since 1.4*/public SftpPoolException(String message, Throwable cause) {super(message, cause);}/* Constructs a new runtime exception with the specified cause and a* detail message of <tt>(cause==null ? null : cause.toString())</tt>* (which typically contains the class and detail message of* <tt>cause</tt>).  This constructor is useful for runtime exceptions* that are little more than wrappers for other throwables. @param cause the cause (which is saved for later retrieval by the*              {@link #getCause()} method).  (A <tt>null</tt> value is*              permitted, and indicates that the cause is nonexistent or*              unknown.)* @since 1.4*/public SftpPoolException(Throwable cause) {super(cause);}/* Constructs a new runtime exception with the specified detail* message, cause, suppression enabled or disabled, and writable* stack trace enabled or disabled. @param message            the detail message.* @param cause              the cause.  (A {@code null} value is permitted,*                           and indicates that the cause is nonexistent or unknown.)* @param enableSuppression  whether or not suppression is enabled*                           or disabled* @param writableStackTrace whether or not the stack trace should*                           be writable* @since 1.7*/public SftpPoolException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {super(message, cause, enableSuppression, writableStackTrace);}}

config

SftpConfig.java

package com.vipsoft.sftp.config;import com.vipsoft.sftp.pool.SftpFactory;
import com.vipsoft.sftp.pool.SftpPool;
import com.vipsoft.sftp.utils.SftpUtil;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;@Configuration
@EnableConfigurationProperties(SftpProperties.class)
public class SftpConfig {// 工厂@Beanpublic SftpFactory sftpFactory(SftpProperties properties) {return new SftpFactory(properties);}// 连接池@Beanpublic SftpPool sftpPool(SftpFactory sftpFactory) {return new SftpPool(sftpFactory);}// 辅助类@Beanpublic SftpUtil sftpUtil(SftpPool sftpPool) {return new SftpUtil(sftpPool);}
}

SftpProperties.java

package com.vipsoft.sftp.config;import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import org.springframework.boot.context.properties.ConfigurationProperties;@ConfigurationProperties(prefix = "sftp")
public class SftpProperties {private String host;private int port = 22;private String username = "root";private String password = "root";private Pool pool = new Pool();public String getHost() {return host;}public void setHost(String host) {this.host = host;}public int getPort() {return port;}public void setPort(int port) {this.port = port;}public String getUsername() {return username;}public void setUsername(String username) {this.username = username;}public String getPassword() {return password;}public void setPassword(String password) {this.password = password;}public Pool getPool() {return pool;}public void setPool(Pool pool) {this.pool = pool;}public static class Pool extends GenericObjectPoolConfig<ChannelSftp> {private int maxTotal = DEFAULT_MAX_TOTAL;private int maxIdle = DEFAULT_MAX_IDLE;private int minIdle = DEFAULT_MIN_IDLE;public Pool() {super();}@Overridepublic int getMaxTotal() {return maxTotal;}@Overridepublic void setMaxTotal(int maxTotal) {this.maxTotal = maxTotal;}@Overridepublic int getMaxIdle() {return maxIdle;}@Overridepublic void setMaxIdle(int maxIdle) {this.maxIdle = maxIdle;}@Overridepublic int getMinIdle() {return minIdle;}@Overridepublic void setMinIdle(int minIdle) {this.minIdle = minIdle;}}
}

Pool

SftpFactory.java

package com.vipsoft.sftp.pool;import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.vipsoft.sftp.config.SftpProperties;
import com.vipsoft.sftp.exception.SftpPoolException;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;import java.util.Properties;public class SftpFactory extends BasePooledObjectFactory<ChannelSftp> {private  final Logger logger = LoggerFactory.getLogger(this.getClass());private SftpProperties properties;public SftpProperties getProperties() {return properties;}public void setProperties(SftpProperties properties) {this.properties = properties;}public SftpFactory(SftpProperties properties) {this.properties = properties;}@Overridepublic ChannelSftp create() {try {JSch jsch = new JSch();Session sshSession = jsch.getSession(properties.getUsername(), properties.getHost(), properties.getPort());sshSession.setPassword(properties.getPassword());Properties sshConfig = new Properties();sshConfig.put("StrictHostKeyChecking", "no");sshSession.setConfig(sshConfig);sshSession.connect();ChannelSftp channel = (ChannelSftp) sshSession.openChannel("sftp");channel.connect();return channel;} catch (JSchException e) {throw new SftpPoolException("连接sfpt失败", e);}}@Overridepublic PooledObject<ChannelSftp> wrap(ChannelSftp channelSftp) {return new DefaultPooledObject<>(channelSftp);}// 销毁对象@Overridepublic void destroyObject(PooledObject<ChannelSftp> p) {ChannelSftp channelSftp = p.getObject();channelSftp.disconnect();}}

SftpPool.java

package com.vipsoft.sftp.pool;import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPool;public class SftpPool<T> extends GenericObjectPool<ChannelSftp> {public SftpPool(SftpFactory factory) {super(factory,factory.getProperties().getPool());}/* 获取一个sftp连接对象* @return sftp连接对象*/@Overridepublic ChannelSftp borrowObject() throws Exception {return super.borrowObject();}/* 归还一个sftp连接对象* @param channelSftp sftp连接对象*/@Overridepublic void returnObject(ChannelSftp channelSftp) {if (channelSftp!=null) {super.returnObject(channelSftp);}}}

Utils

ByteUtil.java

package com.vipsoft.sftp.utils;import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.SftpException;
import com.vipsoft.sftp.exception.SftpPoolException;
import com.vipsoft.sftp.pool.SftpPool;import java.io.InputStream;public class SftpUtil {private SftpPool pool;public SftpUtil(SftpPool pool) {this.pool = pool;}/* 下载文件 @param dir  远程目录* @param name 远程文件名* @return 文件字节数组*/public byte[] download(String dir, String name) {ChannelSftp sftp = null;try {sftp = pool.borrowObject();sftp.cd(dir);InputStream in = sftp.get(name);return ByteUtil.inputStreamToByteArray(in);} catch (Exception e) {throw new SftpPoolException("sftp下载文件出错", e);} finally {pool.returnObject(sftp);}}/* 上传文件 @param dir  远程目录* @param name 远程文件名* @param in   输入流*/public void upload(String dir, String name, InputStream in) {ChannelSftp sftp = null;try {sftp = pool.borrowObject();mkdirs(sftp, dir);sftp.cd(dir);sftp.put(in, name);} catch (Exception e) {throw new SftpPoolException("sftp上传文件出错", e);} finally {pool.returnObject(sftp);}}/* 删除文件 @param dir  远程目录* @param name 远程文件名*/public void delete(String dir, String name) {ChannelSftp sftp = null;try {sftp = pool.borrowObject();sftp.cd(dir);sftp.rm(name);} catch (Exception e) {throw new SftpPoolException("sftp删除文件出错", e);} finally {pool.returnObject(sftp);}}/* 递归创建多级目录 @param dir 多级目录*/private void mkdirs(ChannelSftp sftp, String dir) {String[] folders = dir.split("/");try {sftp.cd("/");for (String folder : folders) {if (folder.length() > 0) {try {sftp.cd(folder);} catch (Exception e) {sftp.mkdir(folder);sftp.cd(folder);}}}} catch (SftpException e) {throw new SftpPoolException("sftp创建目录出错", e);}}}

Test

SftpTest.java

package com.vipsoft.sftp;import com.vipsoft.sftp.utils.SftpUtil;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;@SpringBootTest
public class SftpTest {@Autowiredprivate SftpUtil sftpUtil;@Testvoid downloadTest() {byte[] dockerfiles = sftpUtil.download("/opt/demo/", "Dockerfile");System.out.println("FileSize =>" + dockerfiles.length);}}