网站导航
> 文章列表 > 尚硅谷大数据技术Zookeeper教程-笔记01【Zookeeper(入门、本地安装、集群操作)】

尚硅谷大数据技术Zookeeper教程-笔记01【Zookeeper(入门、本地安装、集群操作)】

网友: 文章列表 2024-03-21 19:09:57

尚硅谷大数据技术Zookeeper教程-笔记01【Zookeeper(入门、本地安装、集群操作)】

视频地址：【尚硅谷】大数据技术之Zookeeper 3.5.7版本教程_哔哩哔哩_bilibili

  1. 尚硅谷大数据技术Zookeeper教程-笔记01【Zookeeper(入门、本地安装、集群操作)】
  2. 尚硅谷大数据技术Zookeeper教程-笔记02【服务器动态上下线监听案例、ZooKeeper分布式锁案例】
  3. 尚硅谷大数据技术Zookeeper教程-笔记03【源码解析-算法基础】
  4. 尚硅谷大数据技术Zookeeper教程-笔记04【源码解析-源码详解】

目录

第01章-Zookeeper入门

P001【001_尚硅谷_zk_开篇_课程介绍】08:08

P002【002_尚硅谷_zk_入门_概述】06:38

P003【003_尚硅谷_zk_入门_特点】05:58

P004【004_尚硅谷_zk_入门_数据结构】01:38

P005【005_尚硅谷_zk_入门_应用场景】08:53

P006【006_尚硅谷_zk_入门_下载地址】01:47

第02章-Zookeeper本地安装

P007【007_尚硅谷_zk_本地_安装】10:58

P008【008_尚硅谷_zk_本地_配置参数解读】02:56

第03章-Zookeeper集群操作

P009【009_尚硅谷_zk_集群_安装】09:33

P010【010_尚硅谷_zk_集群_第一次启动选举机制】07:40

P011【011_尚硅谷_zk_集群_非第一次启动选举机制】06:18

P012【012_尚硅谷_zk_集群_启动停止脚本】06:46

P013【013_尚硅谷_zk_客户端命令行_节点信息】06:04

P014【014_尚硅谷_zk_客户端命令行_节点类型】11:23

P015【015_尚硅谷_zk_客户端命令行_监听器及节点删除】10:11

P016【016_尚硅谷_zk_客户端API_创建节点】12:38

P017【017_尚硅谷_zk_客户端API_监听节点变化】06:26

P018【018_尚硅谷_zk_客户端API_判断节点是否存在】02:30

P019【019_尚硅谷_zk_写数据原理】04:39

第01章-Zookeeper入门

P001【001_尚硅谷_zk_开篇_课程介绍】08:08

大家记得一键三连呀【点赞、投币、收藏】感谢支持~

尚硅谷Zookeeper视频教程再次全方面大升级！在对Zookeeper的集群安装、选举机制、基本操作等内容讲解之外，增加了Zookeeper的分布式锁案例、对ZAB协议的精细讲解，更加细致地讲解了Paxos算法，深入剖析了Zookeeper源码。

本套教程的特点是“新、全、细”。“新”：基于稳定版Zookeeper 3.5.7讲解。

一、课程升级的重点内容
    1、Zookeeper分布式锁案例
    2、Paxos算法
    3、ZAB协议
    4、CAP
    5、源码（zk服务端初始化源码、服务器端加载数据源码、选举算法、状态同步算法、Leader启动源码、Follower启动源码、客户端启动源码）
二、课程特色
    1、新：Zookeeper3.5.7
    2、细：注释详细，文档中代码复制粘贴就可以
    3、全：几乎涵盖了所有关于ZK相关讲解
    4、生动PPT动画
三、资料获取方式
    1、关注尚硅谷教育公众号，回复：java、大数据
    2、谷粒学院
    3、b站
四、技术基础要求
    javase + maven + idea + linux常用命令

P002【002_尚硅谷_zk_入门_概述】06:38

第1章 Zookeeper入门

1.1 Zookeeper概述：Zookeeper是一个开源的分布式的，为分布式框架提供协调服务的Apache项目。

Zookeeper工作机制

P003【003_尚硅谷_zk_入门_特点】05:58

1.2 Zookeeper特点

  • 1）Zookeeper：一个领导者（Leader），多个跟随者（Follower）组成的集群。
  • 2）集群中只要有半数以上节点存活，Zookeeper集群就能正常服务。所 以Zookeeper适合安装奇数台服务器。
  • 3）全局数据一致：每个Server保存一份相同的数据副本，Client无论连接到哪个Server，数据都是一致的。
  • 4）更新请求顺序执行，来自同一个Client的更新请求按其发送顺序依次执行。
  • 5）数据更新原子性，一次数据更新要么成功，要么失败。
  • 6）实时性，在一定时间范围内，Client能读到最新数据。

P004【004_尚硅谷_zk_入门_数据结构】01:38

1.3 数据结构

ZooKeeper 数据模型的结构与 Unix 文件系统很类似，整体上可以看作是一棵树，每个节点称做一个 ZNode。每一个ZNode默认能够存储 1MB 的数据，每个 ZNode 都可以通过其路径唯一标识。

P005【005_尚硅谷_zk_入门_应用场景】08:53

1.4 应用场景

提供的服务包括：统一命名服务、统一配置管理、统一集群管理、服务器节点动态上下线、软负载均衡等。

面试官：Zookeeper是什么？我：zk是向client介绍技师的老鸨。。。

P006【006_尚硅谷_zk_入门_下载地址】01:47

1.5 下载地址

官网首页： Apache ZooKeeper

老版本：Index of /dist/zookeeper

第02章-Zookeeper本地安装

P007【007_尚硅谷_zk_本地_安装】10:58

第2章 Zookeeper本地安装

2.1 本地模式安装

[atguigu@node1 software]$ tar -zxvf apache-zookeeper-3.5.7-bin.tar.gz -C /opt/module/
[atguigu@node1 software]$ cd ../
[atguigu@node1 opt]$ cd module/
[atguigu@node1 module]$ mv apache-zookeeper-3.5.7-bin/ zookeeper-3.5.7

P008【008_尚硅谷_zk_本地_配置参数解读】02:56

2.2 配置参数解读

/opt/module/zookeeper-3.5.7/conf/zoo.cfg配置文件

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/opt/module/zookeeper-3.5.7/zkData
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

第03章-Zookeeper集群操作

P009【009_尚硅谷_zk_集群_安装】09:33

第3章 Zookeeper集群操作

3.1 集群操作

3.1.1 集群安装

启动zk服务器，必须得启动半数以上的服务器。

  1. cd /opt/module/zookeeper-3.5.7/
  2. bin/zkServer.sh start
  3. bin/zkServer.sh status

P010【010_尚硅谷_zk_集群_第一次启动选举机制】07:40

3.1.2 选举机制（面试重点）

Zookeeper选举机制——第一次启动

P011【011_尚硅谷_zk_集群_非第一次启动选举机制】06:18

Zookeeper选举机制——非第一次启动

P012【012_尚硅谷_zk_集群_启动停止脚本】06:46

3.1.3 ZK集群启动停止脚本

1 zookeeper 3.5.7安装部署_andyonlines的博客-CSDN博客

 

#!/bin/bash
case $1 in
"start") {for i in hadoop102 hadoop103 hadoop104doecho ---------- zookeeper $i 启动 ----------ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh start"done
}
;;
"stop") {for i in hadoop102 hadoop103 hadoop104doecho ---------- zookeeper $i 停止 ----------ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh stop"done
}
;;
"status") {for i in hadoop102 hadoop103 hadoop104doecho ---------- zookeeper $i 状态 ----------ssh $i "/opt/module/zookeeper-3.5.7/bin/zkServer.sh status"done
}
;;
esac

P013【013_尚硅谷_zk_客户端命令行_节点信息】06:04

3.2 客户端命令行操作

3.2.1 命令行语法

P014【014_尚硅谷_zk_客户端命令行_节点类型】11:23

3.2.3 节点类型（持久/短暂/有序号/无序号）

节点类型

[zk: node1:2181(CONNECTED) 3] create /sanguo "diaochan" # 给sanguo一个描述“diaochan”
Created /sanguo
[zk: node1:2181(CONNECTED) 4] ls /
[sanguo, zookeeper]
[zk: node1:2181(CONNECTED) 5] [zk: node1:2181(CONNECTED) 0] create /sanguo/shuguo "liubei"
Created /sanguo/shuguo
[zk: node1:2181(CONNECTED) 1] ls /sanguo
[shuguo]
[zk: node1:2181(CONNECTED) 2] get -s /sanguo 
diaochan
cZxid = 0x400000004
ctime = Wed Apr 12 22:22:22 CST 2023
mZxid = 0x400000004
mtime = Wed Apr 12 22:22:22 CST 2023
pZxid = 0x400000007
cversion = 1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 8
numChildren = 1
[zk: node1:2181(CONNECTED) 3] get -s /sanguo/shuguo
liubei
cZxid = 0x400000007
ctime = Wed Apr 12 22:29:30 CST 2023
mZxid = 0x400000007
mtime = Wed Apr 12 22:29:30 CST 2023
pZxid = 0x400000007
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0
[zk: node1:2181(CONNECTED) 4] create /sanguo/weiguo "caocao"
Created /sanguo/weiguo
[zk: node1:2181(CONNECTED) 5] ls /sanguo
[shuguo, weiguo]
[zk: node1:2181(CONNECTED) 6] create -s  /sanguo/weiguo/zhangliao "zhangliao"
Created /sanguo/weiguo/zhangliao0000000000
[zk: node1:2181(CONNECTED) 7] ls /sanguo/weiguo
[zhangliao0000000000]
[zk: node1:2181(CONNECTED) 8] create -s  /sanguo/weiguo/zhangliao "zhangliao"
Created /sanguo/weiguo/zhangliao0000000001
[zk: node1:2181(CONNECTED) 9] create /sanguo/weiguo "caocao"
Node already exists: /sanguo/weiguo
[zk: node1:2181(CONNECTED) 10] quitWATCHER::WatchedEvent state:Closed type:None path:null
2023-04-12 22:34:40,693 [myid:] - INFO  [main:ZooKeeper@1422] - Session: 0x100007eb3420002 closed
2023-04-12 22:34:40,693 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@524] - EventThread shut down for session: 0x100007eb3420002
[root@node1 zookeeper-3.5.7]# bin/zkCli.sh -server node1:2181
[zk: node1:2181(CONNECTED) 0] ls /sanguo
[shuguo, weiguo]
[zk: node1:2181(CONNECTED) 1] ls /sanguo/weiguo
[zhangliao0000000000, zhangliao0000000001]
[zk: node1:2181(CONNECTED) 2] create -e  /sanguo/wuguo "zhouyu"
Created /sanguo/wuguo
[zk: node1:2181(CONNECTED) 3] ls /sanguo
[shuguo, weiguo, wuguo]
[zk: node1:2181(CONNECTED) 4] create -e -s /sanguo/wuguo "zhouyu"
Created /sanguo/wuguo0000000003
[zk: node1:2181(CONNECTED) 5] ls /sanguo/wuguo
[]
[zk: node1:2181(CONNECTED) 6] ls /sanguo
[shuguo, weiguo, wuguo, wuguo0000000003]
[zk: node1:2181(CONNECTED) 7] quit
[root@node1 zookeeper-3.5.7]# bin/zkCli.sh -server node1:2181[zk: node1:2181(CONNECTED) 0] ls /sanguo
[shuguo, weiguo]
[zk: node1:2181(CONNECTED) 1] set /sanguo/weiguo "simayi"
[zk: node1:2181(CONNECTED) 2] get -s /sanguo/weiguo
simayi
cZxid = 0x400000008
ctime = Wed Apr 12 22:31:49 CST 2023
mZxid = 0x400000012
mtime = Wed Apr 12 22:45:39 CST 2023
pZxid = 0x40000000a
cversion = 2
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 2
[zk: node1:2181(CONNECTED) 3] set /sanguo/weiguo "sima"
[zk: node1:2181(CONNECTED) 4] get -s /sanguo/weiguo
sima
cZxid = 0x400000008
ctime = Wed Apr 12 22:31:49 CST 2023
mZxid = 0x400000013
mtime = Wed Apr 12 22:46:13 CST 2023
pZxid = 0x40000000a
cversion = 2
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 4
numChildren = 2
[zk: node1:2181(CONNECTED) 5]

P015【015_尚硅谷_zk_客户端命令行_监听器及节点删除】10:11

3.2.4 监听器原理

客户端注册监听它关心的目录节点，当目录节点发生变化（数据改变、节点删除、子目录节点增加删除）时，ZooKeeper会通知客户端。监听机制保证 ZooKeeper 保存的任何的数据的任何改变都能快速的响应到监听了该节点的应用程序。

监听器原理

-s是see查看，-w是wath监视。

[zk: localhost:2181(CONNECTED) 0] set /sanguo "sanguoyanyi000"
[zk: localhost:2181(CONNECTED) 1] 
addauth         close           config          connect         create          delete          deleteall       delquota        get             getAcl          
history         listquota       ls              ls2             printwatches    quit            reconfig        redo            removewatches   rmr             
set             setAcl          setquota        stat            sync            
[zk: localhost:2181(CONNECTED) 1] set /sanguo "sanguoyanyi001"
[zk: localhost:2181(CONNECTED) 2] set /sanguo "sanguoyanyi002"
[zk: localhost:2181(CONNECTED) 3] create /sanguo/jin "sima"
Node already exists: /sanguo/jin
[zk: localhost:2181(CONNECTED) 4] create /sanguo/jin "sima"
Node already exists: /sanguo/jin
[zk: localhost:2181(CONNECTED) 5] delete /sanguo/jin
[zk: localhost:2181(CONNECTED) 6] create /sanguo/jin "sima"
Created /sanguo/jin
[zk: localhost:2181(CONNECTED) 7] stat /sanguo
cZxid = 0x400000004
ctime = Wed Apr 12 22:22:22 CST 2023
mZxid = 0x500000007
mtime = Thu Apr 13 10:05:04 CST 2023
pZxid = 0x50000000c
cversion = 9
dataVersion = 4
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 14
numChildren = 3
[zk: localhost:2181(CONNECTED) 8] 
[zk: localhost:2181(CONNECTED) 0] get -s /sanguo
diaochan
cZxid = 0x400000004
ctime = Wed Apr 12 22:22:22 CST 2023
mZxid = 0x400000004
mtime = Wed Apr 12 22:22:22 CST 2023
pZxid = 0x400000010
cversion = 6
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 8
numChildren = 2
[zk: localhost:2181(CONNECTED) 1] set /sanguo "sanguoyanyi"
[zk: localhost:2181(CONNECTED) 2] get -s /sanguo
sanguoyanyi
cZxid = 0x400000004
ctime = Wed Apr 12 22:22:22 CST 2023
mZxid = 0x500000004
mtime = Thu Apr 13 10:01:49 CST 2023
pZxid = 0x400000010
cversion = 6
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 11
numChildren = 2
[zk: localhost:2181(CONNECTED) 3] get -w /sanguo
sanguoyanyi
[zk: localhost:2181(CONNECTED) 4] 
WATCHER::WatchedEvent state:SyncConnected type:NodeDataChanged path:/sanguo
get -w /sanguo
sanguoyanyi001
[zk: localhost:2181(CONNECTED) 5] get -w /sanguo
sanguoyanyi001
[zk: localhost:2181(CONNECTED) 6] 
WATCHER::WatchedEvent state:SyncConnected type:NodeDataChanged path:/sanguo[zk: localhost:2181(CONNECTED) 6] create /sanguo/jin "sima"
Created /sanguo/jin
[zk: localhost:2181(CONNECTED) 7] ls -w /sanguo
[jin, shuguo, weiguo]
[zk: localhost:2181(CONNECTED) 8] 
WATCHER::WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/sanguo[zk: localhost:2181(CONNECTED) 8]

P016【016_尚硅谷_zk_客户端API_创建节点】12:38

3.3 客户端API操作

前提：保证 hadoop102、hadoop103、hadoop104 服务器上 Zookeeper 集群服务端启动。

3.3.1 IDEA环境搭建

3.3.2 创建ZooKeeper客户端

package com.atguigu.zk;import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;import java.io.IOException;
import java.util.List;public class zkClient {private String connectString = "node1:2181,node2:2181,node3:2181"; //注意：逗号左右不能有空格private int sessionTimeout = 2000;@Testpublic void init() throws IOException {ZooKeeper zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher() {@Overridepublic void process(WatchedEvent watchedEvent) {}});}
}

3.3.3 创建子节点

package com.atguigu.zk;import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;import java.io.IOException;
import java.util.List;public class zkClient {private String connectString = "node1:2181,node2:2181,node3:2181";//注意：逗号左右不能有空格private int sessionTimeout = 2000;private ZooKeeper zkClient;@Beforepublic void init() throws IOException {zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher() {@Overridepublic void process(WatchedEvent watchedEvent) {}});}@Testpublic void create() throws KeeperException, InterruptedException {String nodeCreated = zkClient.create("/atguigu", "ss.avi".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);}
}

P017【017_尚硅谷_zk_客户端API_监听节点变化】06:26

3.3.4 获取子节点并监听节点变化

package com.atguigu.zk;import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;import java.io.IOException;
import java.util.List;public class zkClient {private String connectString = "node1:2181,node2:2181,node3:2181";//注意：逗号左右不能有空格private int sessionTimeout = 2000;private ZooKeeper zkClient;@Beforepublic void init() throws IOException {zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher() {@Overridepublic void process(WatchedEvent watchedEvent) {System.out.println("-------------------------------");List<String> children = null;try {children = zkClient.getChildren("/", true);for (String child : children) {System.out.println(child);}System.out.println("-------------------------------");} catch (KeeperException e) {e.printStackTrace();} catch (InterruptedException e) {e.printStackTrace();}}});}@Testpublic void create() throws KeeperException, InterruptedException {String nodeCreated = zkClient.create("/atguigu", "ss.avi".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);}@Testpublic void getChildren() throws KeeperException, InterruptedException {List<String> children = zkClient.getChildren("/", true);for (String child : children) {System.out.println(child);}Thread.sleep(Long.MAX_VALUE);//延时}
}

P018【018_尚硅谷_zk_客户端API_判断节点是否存在】02:30

3.3.5 判断Znode是否存在

package com.atguigu.zk;import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;import java.io.IOException;
import java.util.List;public class zkClient {private String connectString = "node1:2181,node2:2181,node3:2181";//注意：逗号左右不能有空格private int sessionTimeout = 2000;private ZooKeeper zkClient;@Beforepublic void init() throws IOException {zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher() {@Overridepublic void process(WatchedEvent watchedEvent) {
//                System.out.println("-------------------------------");
//                List<String> children = null;
//                try {
//                    children = zkClient.getChildren("/", true);
//                    for (String child : children) {
//                        System.out.println(child);
//                    }
//                    System.out.println("-------------------------------");
//                } catch (KeeperException e) {
//                    e.printStackTrace();
//                } catch (InterruptedException e) {
//                    e.printStackTrace();
//                }}});}@Testpublic void create() throws KeeperException, InterruptedException {String nodeCreated = zkClient.create("/atguigu", "ss.avi".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);}@Testpublic void getChildren() throws KeeperException, InterruptedException {List<String> children = zkClient.getChildren("/", true);for (String child : children) {System.out.println(child);}Thread.sleep(Long.MAX_VALUE);//延时}@Testpublic void exist() throws KeeperException, InterruptedException {Stat stat = zkClient.exists("/atguigu", false);System.out.println(stat == null ? "not exist " : "exist");}
}

P019【019_尚硅谷_zk_写数据原理】04:39

3.4 客户端向服务端写数据流程

写流程之写入请求直接发送给Leader节点

写流程之写入请求发送给follower节点

网络标签：

相关问题