为什么 hbase 无法运行在 docker 中？

  • Z时代
  • 2024-02-15
  • 分类：IT

我需要让 hbase 跑在 arm 上

而且想用 docker

所以我写了下面的 Dockerfile

FROM python:3.10-buster 
RUN . /etc/os-release && cat > /etc/apt/sources.list <<EOF
    deb http://mirrors.tuna.tsinghua.edu.cn/debian/ ${VERSION_CODENAME} main contrib non-free
    deb http://mirrors.tuna.tsinghua.edu.cn/debian/ ${VERSION_CODENAME}-updates main contrib non-free
    deb http://mirrors.tuna.tsinghua.edu.cn/debian/ ${VERSION_CODENAME}-backports main contrib non-free
    deb http://mirrors.tuna.tsinghua.edu.cn/debian-security buster/updates main contrib non-free
EOF
RUN apt-get update
RUN apt-get install -y vim
RUN apt-get install -y openjdk-11-jdk
RUN apt-get install -y wget
WORKDIR /root
RUN wget -P /root https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/2.4.17/hbase-2.4.17-src.tar.gz
RUN tar xvf /root/hbase-2.4.17-src.tar.gz
我先跑在 x86 上，看看 ok 不 ok ，所以上面的 java home 是 amd64 的

但是不行

╰─➤  docker run --rm -it ponponon/hbase-arm bash               
root@97bc49430241:~# cd /root/hbase-2.4.17
root@97bc49430241:~/hbase-2.4.17# 
root@97bc49430241:~/hbase-2.4.17# 
root@97bc49430241:~/hbase-2.4.17# HBASE_HOME=/root/hbase-2.4.17 JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar bash bin/hbase-daemon.sh start master 
running master, logging to /root/hbase-2.4.17/logs/hbase--master-97bc49430241.out
/root/hbase-2.4.17/bin/hbase-daemon.sh: line 92: kill: (29) - No such process
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.master.HMaster

在我的虚拟机上，直接 JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 bash bin/hbase-daemon.sh start master 就行了！

我的虚拟机是 ubuntu18.04，可以直接 sudo apt install openjdk-8-jdk，所以有 jdk8，但是用的 debian10 已经没有 debian8 了，我就用 jdk11 了

https://hbase.apache.org/book.html#java

不知道是不是因为 jdk11 引起的

应该不是 jdk 的问题，因为我换成 ubuntu20 的镜像之后，安装 jdk8 之后也是不行

FROM ubuntu:focal
RUN . /etc/os-release && cat > /etc/apt/sources.list <<EOF
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal main restricted universe multiverse
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-updates main restricted universe multiverse
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-backports main restricted universe multiverse
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-security main restricted universe multiverse
EOF
RUN apt-get update
RUN apt-get install -y vim
RUN apt-get install -y openjdk-8-jdk
RUN apt-get install -y wget
WORKDIR /root
RUN wget -P /root https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/2.4.17/hbase-2.4.17-src.tar.gz
RUN tar xvf /root/hbase-2.4.17-src.tar.gz
RUN echo "export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64" | tee -a /etc/profile >/dev/null
注意，这是 arm 版本

docker run --rm -it ponponon/hbase-arm-focal bash

root@d36c7ca4a53d:~/hbase-2.4.17# JAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64 bash bin/hbase-daemon.sh start master
running master, logging to /root/hbase-2.4.17/bin/../logs/hbase--master-d36c7ca4a53d.out
/root/hbase-2.4.17/bin/hbase-daemon.sh: line 92: kill: (267) - No such process

不是用 harisekhon/hbase 是因为在 apple arm mac 上会报错 TTransportException: TTransportException(type=4, message='TSocket read 0 bytes')

version: "3"
services:
  hbase:
    container_name: hbase
    image: harisekhon/hbase:2.1
    restart: always
    ports:
      - "2181:2181"
      - "8080:8080"
      - "8085:8085"
      - "9090:9090"
      - "9095:9095"
      - "16000:16000"
      - "16010:16010"
      - "16201:16201"
      - "16020:16020"
      - "16030:16030"
      - "16301:16301"

启动容器

─➤  docker-compose up -d                             
[+] Building 0.0s (0/0)                                                                                                                           
[+] Running 1/0
 ⠿ Container hbase                                                                                                                                [+] Running 1/2.1s 
 ⠿ Container hbase                                                                                                                                [+] Running 1/2.2s 
 ⠿ Container hbase                                                                                                                                [+] Running 2/2.3s 
 ✔ Container hbase                                                                                                                                      Started0.3s 
 ! hbase The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s

使用下面的 python 脚本连接

import happybase
# 定义HBase连接信息
host = '192.168.38.191'  # HBase主机名或IP地址
host = '127.0.0.1'  # HBase主机名或IP地址
port = 9090  # HBase端口，默认为9090
# 建立HBase连接
connection = happybase.Connection(host=host, port=port)
# 获取所有表名
table_names = connection.tables()
# 删除所有表
print('删除所有表')
for table_name in table_names:
    connection.delete_table(table_name, disable=True)
table_name = 'my_table'  # 表名
column_families = {
    'cf1': dict(),  # 第一个列族
    'cf2': dict()   # 第二个列族
}
# 建立HBase连接
connection = happybase.Connection(host=host, port=port)
# 创建表
print('创建表')
connection.create_table(table_name, column_families)
# 获取表对象
table = connection.table(table_name)
# 插入数据
print('插入数据')
row_key = 'row1'
data = {
    'cf1:column1': 'value1',
    'cf1:column2': 'value2',
    'cf2:column3': 'value3'
}
table.put(row_key, data)
# 获取数据
row = table.row(row_key)
print(row)
# 扫描数据
print('读取数据')
scan_result = table.scan(row_start=row_key, row_stop=row_key)
for key, value in scan_result:
    print(key, value)
# 删除数据
table.delete(row_key)
# 关闭连接
connection.close()

会报错

─➤  python -u "/Users/ponponon/Desktop/code/me/ideaboom/5.py"
Traceback (most recent call last):
  File "/Users/ponponon/Desktop/code/me/ideaboom/5.py", line 14, in <module>
    table_names = connection.tables()
  File "/Users/ponponon/.local/share/virtualenvs/ideaboom-B0dr_aXc/lib/python3.10/site-packages/happybase/connection.py", line 242, in tables
    names = self.client.getTableNames()
  File "/Users/ponponon/.local/share/virtualenvs/ideaboom-B0dr_aXc/lib/python3.10/site-packages/thriftpy2/thrift.py", line 219, in _req
    return self._recv(_api)
  File "/Users/ponponon/.local/share/virtualenvs/ideaboom-B0dr_aXc/lib/python3.10/site-packages/thriftpy2/thrift.py", line 231, in _recv
    fname, mtype, rseqid = self._iprot.read_message_begin()
  File "thriftpy2/protocol/cybin/cybin.pyx", line 463, in cybin.TCyBinaryProtocol.read_message_begin
  File "thriftpy2/protocol/cybin/cybin.pyx", line 68, in cybin.read_i32
  File "thriftpy2/transport/buffered/cybuffered.pyx", line 65, in thriftpy2.transport.buffered.cybuffered.TCyBufferedTransport.c_read
  File "thriftpy2/transport/buffered/cybuffered.pyx", line 69, in thriftpy2.transport.buffered.cybuffered.TCyBufferedTransport.read_trans
  File "thriftpy2/transport/cybase.pyx", line 61, in thriftpy2.transport.cybase.TCyBuffer.read_trans
  File "/Users/ponponon/.local/share/virtualenvs/ideaboom-B0dr_aXc/lib/python3.10/site-packages/thriftpy2/transport/socket.py", line 131, in read
    raise TTransportException(type=TTransportException.END_OF_FILE,
thriftpy2.transport.base.TTransportException: TTransportException(type=4, message='TSocket read 0 bytes')

回答：

解决了，可看: 借助 Docker 在 Arm 设备上运行 Hbase

回答：

你直接使用hbase的基础镜像不就行了

docker run -d -h 127.0.0.1 -p 2181:2181 -p 8080:8080 -p 8085:8085 -p 9090:9090 -p 9000:9000  -p 9095:9095  -p 16000:16000  -p 16010:16010  -p 16201:16201  -p 16301:16301 -p 16020:16020 --name hbase  harisekhon/hbase

如果想自己改造就
Dockerfile

FROM harisekhon/hbase
xxxxxxx
.......

以上是 为什么 hbase 无法运行在 docker 中？ 的全部内容， 来源链接： utcz.com/p/945254.html

回到顶部