ElasticSearch Logstash未连接“连接被拒绝”- Docker

mdfafbf1  于 2023-01-19  发布在  Logstash
关注(0)|答案(1)|浏览(590)

我需要帮助!(谁会想到呢,对吗?哈哈)
几天后我有一个工作面试,对我来说,做好充分的准备,并有一些工作示例,这将意味着世界。
我正在尝试建立一个ELK管道,从Kafka流数据,通过logstash,elasticsearch,最后从Kibana读取它。通常。我正在使用容器,但logstash - elasticsearch这对组合让我大吃一惊。
其他的一切都运行得很好,我已经检查了Kafka的日志,它运行得很好,Kibana被收集到elasticsearch中也很好,但是logstash和es真的不想匹配。
设置如下
docker-compose.yml

version: '3.6'
services:
  elasticsearch:
    image: elasticsearch:8.6.0
    container_name: elasticsearch
    #restart: always
    volumes:
    - elastic_data:/usr/share/elasticsearch/data/
    environment:
      cluster.name: elf-kafka-cluster
      ES_JAVA_OPTS: "-Xmx256m -Xms256m"
      discovery.type: single-node    
      xpack.security.enabled: false
    ports:
    - '9200:9200'
    - '9300:9300'
    networks:
      - elk

  kibana:
    image: kibana:8.6.0
    container_name: kibana
    #restart: always       
    ports:
    - '5601:5601'
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200  
    depends_on:
      - elasticsearch  
    networks:
      - elk

  logstash:
    image: logstash:8.6.0
    container_name: logstash
    #restart: always
    volumes:
    - type: bind
      source: ./logstash_pipeline/
      target: /usr/share/logstash/pipeline
      read_only: true
    command: logstash -f /home/ettore/Documenti/Portfolio/ELK/logstash/logstash.conf 
    depends_on:
      - elasticsearch
    ports:
    - '9600:9600'
    environment:
      xpack.monitoring.enabled: true
    #   LS_JAVA_OPTS: "-Xmx256m -Xms256m"    
    links:
      - elasticsearch
    networks:
      - elk

volumes:
  elastic_data: {}

networks:
  elk:
    driver: bridge

logstash.conf

input {
    kafka {
        bootstrap_servers => "localhost:9092"
        topics => ["topic"]
    }
}

output {
    elasitcsearch {
        hosts => ["http://localhost:9200"]
        index => "topic"
        workers => 1
    }
}

这些是我编写时的logstash错误日志:

logstash       | [2023-01-17T13:59:02,680][WARN ][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.
logstash       | Please configure Metricbeat to monitor Logstash. Documentation can be found at: 
logstash       | https://www.elastic.co/guide/en/logstash/current/monitoring-with-metricbeat.html
logstash       | [2023-01-17T13:59:04,711][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
logstash       | [2023-01-17T13:59:05,373][INFO ][logstash.licensechecker.licensereader] Failed to perform request {:message=>"Connect to elasticsearch:9200 [elasticsearch/172.20.0.2] failed: Connection refused", :exception=>Manticore::SocketException, :cause=>#<Java::OrgApacheHttpConn::HttpHostConnectException: Connect to elasticsearch:9200 [elasticsearch/172.20.0.2] failed: Connection refused>}
logstash       | [2023-01-17T13:59:05,379][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::SocketException] Connect to elasticsearch:9200 [elasticsearch/172.20.0.2] failed: Connection refused"}
logstash       | [2023-01-17T13:59:05,436][INFO ][logstash.licensechecker.licensereader] Failed to perform request {:message=>"Connect to elasticsearch:9200 [elasticsearch/172.20.0.2] failed: Connection refused", :exception=>Manticore::SocketException, :cause=>#<Java::OrgApacheHttpConn::HttpHostConnectException: Connect to elasticsearch:9200 [elasticsearch/172.20.0.2] failed: Connection refused>}
logstash       | [2023-01-17T13:59:05,444][WARN ][logstash.licensechecker.licensereader] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://elasticsearch:9200/_xpack][Manticore::SocketException] Connect to elasticsearch:9200 [elasticsearch/172.20.0.2] failed: Connection refused {:url=>http://elasticsearch:9200/, :error_message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/_xpack][Manticore::SocketException] Connect to elasticsearch:9200 [elasticsearch/172.20.0.2] failed: Connection refused", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
logstash       | [2023-01-17T13:59:05,449][WARN ][logstash.licensechecker.licensereader] Attempt to validate Elasticsearch license failed. Sleeping for 0.02 {:fail_count=>1, :exception=>"Elasticsearch Unreachable: [http://elasticsearch:9200/_xpack][Manticore::SocketException] Connect to elasticsearch:9200 [elasticsearch/172.20.0.2] failed: Connection refused"}
logstash       | [2023-01-17T13:59:05,477][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"}
logstash       | [2023-01-17T13:59:05,567][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
logstash       | [2023-01-17T13:59:05,661][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/home/ettore/Documenti/Portfolio/ELK/logstash/logstash.conf"}
logstash       | [2023-01-17T13:59:05,664][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
logstash       | [2023-01-17T13:59:06,333][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
logstash       | [2023-01-17T13:59:06,411][INFO ][logstash.runner          ] Logstash shut down.
logstash       | [2023-01-17T13:59:06,419][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
logstash       | org.jruby.exceptions.SystemExit: (SystemExit) exit
logstash       |    at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
logstash       |    at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
logstash       |    at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

而这是为了证明一切都在按照ES的预期工作(或者看起来是这样)

netstat -an | grep 9200

tcp        0      0 0.0.0.0:9200            0.0.0.0:*               LISTEN     
tcp6       0      0 :::9200                 :::*                    LISTEN     
unix  3      [ ]         STREAM     CONNECTED     49200

我已经看过所有的东西,这是100%没有重复,因为我已经尝试了所有。我真的想不出来。希望任何人可以帮助。
谢谢你抽出时间。

pxq42qpu

pxq42qpu1#

您应该设置logstash.yml
1.使用以下值创建logstash.yml:
http.host:“0.0.0.0“
xpack.monitoring.elasticsearch.hosts:[“http://本地主机:9200”]
1.在docker-compose.yml中,在Logstash容器中添加另一个卷,如下所示:
日志文件:/usr/共享/日志文件/配置/日志文件
此外,它的良好运行与重新启动条件。

相关问题