我刚刚开始使用hadoop,并根据本文安装了一个带有docker compose的hadoop环境。
不幸的是,看起来hue服务崩溃了,我不能访问任何ui(甚至不能访问namenodeui)。我在wsl2内部的ubuntu 20.04中运行以下命令:
git clone https://github.com/m-semnani/bd-infra.git
cd bd-infra
docker-compose up -d
然后,在安装完所有内容并启动并运行服务之后,我做了一个简单的docker ps并看到以下结果:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d1cac433a6ee bde2020/hive:2.3.2-postgresql-metastore "entrypoint.sh /bin/…" 6 seconds ago Up 4 seconds 0.0.0.0:10000->10000/tcp, 10002/tcp hive-server
603604a7ce37 bde2020/hive:2.3.2-postgresql-metastore "entrypoint.sh /opt/…" 7 seconds ago Up 5 seconds 10000/tcp, 0.0.0.0:9083->9083/tcp, 10002/tcp hive-metastore
9964d2f49f5b bde2020/hive-metastore-postgresql:2.3.0 "/docker-entrypoint.…" 8 seconds ago Up 6 seconds 5432/tcp hive-metastore-postgresql
494f5edaed7e bde2020/spark-worker:2.4.0-hadoop2.7 "/bin/bash /worker.sh" 8 seconds ago Up 6 seconds 0.0.0.0:8081->8081/tcp spark-worker
3d42461686fb gethue/hue:20191107-135001 "./startup.sh" 8 seconds ago Up 7 seconds 0.0.0.0:8888->8888/tcp hue
ef5213648cee bde2020/hadoop-datanode:2.0.0-hadoop2.7.4-java8 "/entrypoint.sh /run…" 9 seconds ago Up 7 seconds (health: starting) 0.0.0.0:50075->50075/tcp datanode
82dc9e4ab2ab bde2020/hadoop-namenode:2.0.0-hadoop2.7.4-java8 "/entrypoint.sh /run…" 10 seconds ago Up 8 seconds (health: starting) 0.0.0.0:50070->50070/tcp namenode
51271761043f wurstmeister/kafka:2.12-2.3.0 "start-kafka.sh" 10 seconds ago Up 8 seconds 0.0.0.0:9092->9092/tcp bd-infra_kafka_1
8c6f51379a48 bde2020/spark-master:2.4.0-hadoop2.7 "/bin/bash /master.sh" 10 seconds ago Up 8 seconds 0.0.0.0:7077->7077/tcp, 6066/tcp, 0.0.0.0:8080->8080/tcp spark-master
b8ca46f10106 wurstmeister/zookeeper:3.4.6 "/bin/sh -c '/usr/sb…" 10 seconds ago Up 7 seconds 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp bd-infra_zookeeper_1
如我们所见,hue服务正在运行,容器id为3d42461686fb。但是,在我等待几秒钟并再次执行docker ps之后,我看到色调容器不再运行:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d1cac433a6ee bde2020/hive:2.3.2-postgresql-metastore "entrypoint.sh /bin/…" 29 seconds ago Up 28 seconds 0.0.0.0:10000->10000/tcp, 10002/tcp hive-server
603604a7ce37 bde2020/hive:2.3.2-postgresql-metastore "entrypoint.sh /opt/…" 30 seconds ago Up 29 seconds 10000/tcp, 0.0.0.0:9083->9083/tcp, 10002/tcp hive-metastore
9964d2f49f5b bde2020/hive-metastore-postgresql:2.3.0 "/docker-entrypoint.…" 31 seconds ago Up 30 seconds 5432/tcp hive-metastore-postgresql
494f5edaed7e bde2020/spark-worker:2.4.0-hadoop2.7 "/bin/bash /worker.sh" 31 seconds ago Up 30 seconds 0.0.0.0:8081->8081/tcp spark-worker
ef5213648cee bde2020/hadoop-datanode:2.0.0-hadoop2.7.4-java8 "/entrypoint.sh /run…" 32 seconds ago Up 31 seconds (healthy) 0.0.0.0:50075->50075/tcp datanode
82dc9e4ab2ab bde2020/hadoop-namenode:2.0.0-hadoop2.7.4-java8 "/entrypoint.sh /run…" 33 seconds ago Up 32 seconds (healthy) 0.0.0.0:50070->50070/tcp namenode
51271761043f wurstmeister/kafka:2.12-2.3.0 "start-kafka.sh" 33 seconds ago Up 32 seconds 0.0.0.0:9092->9092/tcp bd-infra_kafka_1
8c6f51379a48 bde2020/spark-master:2.4.0-hadoop2.7 "/bin/bash /master.sh" 33 seconds ago Up 31 seconds 0.0.0.0:7077->7077/tcp, 6066/tcp, 0.0.0.0:8080->8080/tcp spark-master
b8ca46f10106 wurstmeister/zookeeper:3.4.6 "/bin/sh -c '/usr/sb…" 33 seconds ago Up 31 seconds 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp bd-infra_zookeeper_1
另外,当我尝试访问上namenode的ui时localhost:50070 and 色调的用户界面localhost:8888,我得到了一个错误,这个站点无法访问(对于hue服务来说显然是这样,但是对于namenode来说也是如此)
与repo的docker compose文件相比,我所做的唯一更改是在docker compose文件中将卷路径/tmp更改为./tmp,但在这两种情况下我都遇到了相同的问题。你对如何解决这个问题有什么建议吗?
更新1:下面是运行几秒钟后的色调容器日志 docker-compose up --build -d
然后 docker-compose logs hue
:
Attaching to hue
hue | [21/Mar/2021 10:38:00 ] settings INFO Welcome to Hue 4.5.0
hue | [21/Mar/2021 03:38:04 -0700] decorators INFO AXES: BEGIN LOG
hue | [21/Mar/2021 03:38:04 -0700] decorators INFO Using django-axes 2.2.0
hue | Traceback (most recent call last):
hue | File "./build/env/bin/hue", line 11, in <module>
hue | load_entry_point('desktop', 'console_scripts', 'hue')()
hue | File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
hue | execute_from_command_line(sys.argv)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
hue | utility.execute()
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 356, in execute
hue | self.fetch_command(subcommand).run_from_argv(self.argv)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/base.py", line 283, in run_from_argv
hue | self.execute(*args,**cmd_options)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/base.py", line 327, in execute
hue | self.check()
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/base.py", line 359, in check
hue | include_deployment_checks=include_deployment_checks,
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/base.py", line 346, in _run_checks
hue | return checks.run_checks(**kwargs)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/checks/registry.py", line 81, in run_checks
hue | new_errors = check(app_configs=app_configs)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/checks/model_checks.py", line 30, in check_all_models
hue | errors.extend(model.check(**kwargs))
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/models/base.py", line 1284, in check
hue | errors.extend(cls._check_fields(**kwargs))
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/models/base.py", line 1359, in _check_fields
hue | errors.extend(field.check(**kwargs))
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/models/fields/__init__.py", line 913, in check
hue | errors = super(AutoField, self).check(**kwargs)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/models/fields/__init__.py", line 219, in check
hue | errors.extend(self._check_backend_specific_checks(**kwargs))
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/models/fields/__init__.py", line 322, in _check_backend_specific_checks
hue | return connections[db].validation.check_field(self,**kwargs)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/mysql/validation.py", line 49, in check_field
hue | field_type = field.db_type(self.connection)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/models/fields/__init__.py", line 644, in db_type
hue | return connection.data_types[self.get_internal_type()] % data
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/utils/functional.py", line 35, in __get__
hue | res = instance.__dict__[self.name] = self.func(instance)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/mysql/base.py", line 174, in data_types
hue | if self.features.supports_microsecond_precision:
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/utils/functional.py", line 35, in __get__
hue | res = instance.__dict__[self.name] = self.func(instance)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/mysql/features.py", line 53, in supports_microsecond_precision
hue | return self.connection.mysql_version >= (5, 6, 4) and Database.version_info >= (1, 2, 5)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/utils/functional.py", line 35, in __get__
hue | res = instance.__dict__[self.name] = self.func(instance)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/mysql/base.py", line 385, in mysql_version
hue | with self.temporary_connection() as cursor:
hue | File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
hue | return self.gen.next()
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/base/base.py", line 591, in temporary_connection
hue | cursor = self.cursor()
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/base/base.py", line 254, in cursor
hue | return self._cursor()
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/base/base.py", line 229, in _cursor
hue | self.ensure_connection()
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/base/base.py", line 213, in ensure_connection
hue | self.connect()
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/utils.py", line 94, in __exit__
hue | six.reraise(dj_exc_type, dj_exc_value, traceback)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/base/base.py", line 213, in ensure_connection
hue | self.connect()
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/base/base.py", line 189, in connect
hue | self.connection = self.get_new_connection(conn_params)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/db/backends/mysql/base.py", line 274, in get_new_connection
hue | conn = Database.connect(**conn_params)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/MySQL_python-1.2.5-py2.7-linux-x86_64.egg/MySQLdb/__init__.py", line 81, in Connect
hue | return Connection(*args,**kwargs)
hue | File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/MySQL_python-1.2.5-py2.7-linux-x86_64.egg/MySQLdb/connections.py", line 193, in __init__
hue | super(Connection, self).__init__(*args,**kwargs2)
hue | django.db.utils.OperationalError: (2005, "Unknown MySQL server host 'database' (0)")
实际上有更多的日志,但它们只是上面的一个代表。正如注解中所建议的,看起来确实存在数据库连接错误。我最初的想法是,也许这件事出了问题 database
hue-overrides.ini文件的部分,但主机名和端口名对我来说似乎有意义。如果有人能分享更多关于这方面的见解,我将不胜感激。
谢谢,凯文
暂无答案!
目前还没有任何答案,快来回答吧!