在Kibana中找不到Logstash索引

csga3l58  于 2022-12-09  发布在  Logstash
关注(0)|答案(1)|浏览(308)

I am trying to setup a central Logging solution for Syslog-UDP Logging data with ELK stack in the docker environment.
I tried to setup a docker-compose.yml with the oss versions of the ELK-Stack. After firing docker-compose up, I expected to see the Logstash Index on the kibana interface under "Stack Management" => "Index Patterns", but there is no Index to be found.

Problem:

After starting all three containers (elastic, logstash, kibana) my Logstash Index doesnt seem to appear in the kibana web-interface.
Software:

  • Host-System: Windows 10
  • Docker: Docker-Desktop 4.5.1 (74721)
    docker-compose.yml:
version: '3.7'
services:
  # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/docker.html
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
    container_name: ambulance_kibana_elasticsearch
    ports:
      - "9200:9200"
      - "9300:9300"
    networks:
      - amb_kibana
    environment:
      discovery.type: single-node
      ES_JAVA_OPTS: "-Xmx512m -Xms512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    deploy:
      resources:
        limits:
          memory: 1g
  logstash:
    image: docker.elastic.co/logstash/logstash-oss:7.10.2
    container_name: ambulance_kibana_logstash
    volumes:
      - ./logstash/:/logstash_dir
    command: logstash -f /logstash_dir/logstash.conf
    environment:
      LS_JAVA_OPTS: "-Xmx256m -Xms256m"
    depends_on:
      - elasticsearch
    ports:
      - "9600:9600"
      - "514:514/udp"
    networks:
      - amb_kibana
  # Kibana: https://www.elastic.co/guide/en/kibana/current/docker.html
  kibana:
    image: docker.elastic.co/kibana/kibana-oss:7.10.2
    container_name: ambulance_kibana
    environment:
      - "ELASTICSEARCH_HOSTS=http://host.docker.internal:9200"
    depends_on:
      - elasticsearch
    ports:
      - "5601:5601"
    networks:
      - amb_kibana
networks:
  amb_kibana:
    driver: bridge

logstash.conf:

input {
    udp {
        host => "127.0.0.1"
        port => 514
    }
}
output {
    elasticsearch {
        hosts => "elasticsearch:9200"
    }
}

Console-Output after docker-compose up:

Creating ambulance_kibana_elasticsearch ... done
Creating ambulance_kibana               ... done
Creating ambulance_kibana_logstash      ... done
Attaching to ambulance_kibana_elasticsearch, ambulance_kibana_logstash, ambulance_kibana
ambulance_kibana_logstash | Using bundled JDK: /usr/share/logstash/jdk
ambulance_kibana_logstash | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:22,827Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "version[7.10.2], pid[8], build[oss/docker/747e1cc71def077253878a59143c1f785afa92b9/2021-01-13T00:42:
12.435326Z], OS[Linux/4.19.128-microsoft-standard/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:22,831Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "JVM home [/usr/share/elasticsearch/jdk], using bundled JDK [true]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:22,832Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.tt
l=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirec
tArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-12772762765856285752, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=d
ata, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=268435456, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/confi
g, -Des.distribution.flavor=oss, -Des.distribution.type=docker, -Des.bundled_jdk=true]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,268Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [aggs-matrix-stats]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,268Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [analysis-common]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,268Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [geo]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,269Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [ingest-common]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,269Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [ingest-geoip]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,271Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [ingest-user-agent]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,272Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [kibana]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,272Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [lang-expression]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,272Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [lang-mustache]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,272Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [lang-painless]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,273Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [mapper-extras]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,274Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [parent-join]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,274Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [percolator]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,274Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [rank-eval]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,274Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [reindex]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,275Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [repository-url]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,276Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "loaded module [transport-netty4]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,283Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "no plugins loaded" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,358Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "using [1] data paths, mounts [[/ (overlay)]], net usable_space [218.2gb], net total_space
 [250.9gb], types [overlay]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,359Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "heap size [512mb], compressed ordinary object pointers [true]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:24,452Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "node name [1bd1af51a22e], node ID [vjcbyWJMTn2Xcopkbx_HXw], cluster name [docker-cluster], roles [ma
ster, remote_cluster_client, data, ingest]" }
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:26Z","tags":["info","plugins-service"],"pid":7,"message":"Plugin \"visTypeXy\" is disabled."}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:26Z","tags":["info","plugins-system"],"pid":7,"message":"Setting up [40] plugins: [usageCollection,telemetryCollectionManager,telemetry,kibanaUsageCollection,securityOss,newsfeed,mapsLegacy,kibanaLegacy,share,legacyExport,embeddable,expressions,dat
a,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeVega,visTypeTimelion,timelion,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,esUiShared,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,sa
vedObjectsManagement,bfetch]"}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:27Z","tags":["info","savedobjects-service"],"pid":7,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:27Z","tags":["error","elasticsearch","data"],"pid":7,"message":"[ConnectionError]: socket hang up"}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:27Z","tags":["error","savedobjects-service"],"pid":7,"message":"Unable to retrieve version information from Elasticsearch nodes."}
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:29,633Z", "level": "INFO", "component": "o.e.t.NettyAllocator", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocati
on_size=256kb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=512mb}]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:29,749Z", "level": "INFO", "component": "o.e.d.DiscoveryModule", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "using discovery type [single-node] and seed hosts providers [settings]" }
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:29Z","tags":["error","elasticsearch","data"],"pid":7,"message":"[ConnectionError]: socket hang up"}
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:30,206Z", "level": "WARN", "component": "o.e.g.DanglingIndicesState", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "gateway.auto_import_dangling_indices is disabled, dangling indices will not be autom
atically detected or imported and must be managed manually" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:30,507Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "initialized" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:30,508Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "starting ..." }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:30,692Z", "level": "INFO", "component": "o.e.t.TransportService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "publish_address {192.168.96.2:9300}, bound_addresses {0.0.0.0:9300}" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:31,018Z", "level": "INFO", "component": "o.e.c.c.Coordinator", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "setting initial configuration to VotingConfiguration{vjcbyWJMTn2Xcopkbx_HXw}" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:31,171Z", "level": "INFO", "component": "o.e.c.s.MasterService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "elected-as-master ([1] nodes joined)[{1bd1af51a22e}{vjcbyWJMTn2Xcopkbx_HXw}{uaKJzBE9Qhqst
6ENFzsJ0A}{192.168.96.2}{192.168.96.2:9300}{dimr} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 1, version: 1, delta: master node changed {previous [], current [{1bd1af51a22e}{vjcbyWJMTn2Xcopkbx_HXw}{uaKJzBE9Qhqst6ENFzsJ0A}{192.168.96.2}{192.168.96.2:9300}{dimr}]}" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:31,243Z", "level": "INFO", "component": "o.e.c.c.CoordinationState", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "cluster UUID set to [xHujkBpFSeiR_6uHyalNdg]" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:31,314Z", "level": "INFO", "component": "o.e.c.s.ClusterApplierService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "master node changed {previous [], current [{1bd1af51a22e}{vjcbyWJMTn2Xcopkbx_HXw}
{uaKJzBE9Qhqst6ENFzsJ0A}{192.168.96.2}{192.168.96.2:9300}{dimr}]}, term: 1, version: 1, reason: Publication{term=1, version=1}" }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:31,340Z", "level": "INFO", "component": "o.e.h.AbstractHttpServerTransport", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "publish_address {192.168.96.2:9200}, bound_addresses {0.0.0.0:9200}", "cluste
r.uuid": "xHujkBpFSeiR_6uHyalNdg", "node.id": "vjcbyWJMTn2Xcopkbx_HXw"  }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:31,340Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "started", "cluster.uuid": "xHujkBpFSeiR_6uHyalNdg", "node.id": "vjcbyWJMTn2Xcopkbx_HXw"  }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:31,368Z", "level": "INFO", "component": "o.e.g.GatewayService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "recovered [0] indices into cluster_state", "cluster.uuid": "xHujkBpFSeiR_6uHyalNdg", "node
.id": "vjcbyWJMTn2Xcopkbx_HXw"  }
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:32Z","tags":["info","savedobjects-service"],"pid":7,"message":"Starting saved objects migrations"}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:32Z","tags":["info","savedobjects-service"],"pid":7,"message":"Creating index .kibana_1."}
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:33,090Z", "level": "INFO", "component": "o.e.c.m.MetadataCreateIndexService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "[.kibana_1] creating index, cause [api], templates [], shards [1]/[1]", "clu
ster.uuid": "xHujkBpFSeiR_6uHyalNdg", "node.id": "vjcbyWJMTn2Xcopkbx_HXw"  }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:33,107Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "updating number_of_replicas to [0] for indices [.kibana_1]", "cluster.uuid": "xHujk
BpFSeiR_6uHyalNdg", "node.id": "vjcbyWJMTn2Xcopkbx_HXw"  }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:33,680Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.
kibana_1][0]]]).", "cluster.uuid": "xHujkBpFSeiR_6uHyalNdg", "node.id": "vjcbyWJMTn2Xcopkbx_HXw"  }
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:33Z","tags":["info","savedobjects-service"],"pid":7,"message":"Pointing alias .kibana to .kibana_1."}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:33Z","tags":["info","savedobjects-service"],"pid":7,"message":"Finished in 1376ms."}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:34Z","tags":["info","plugins-system"],"pid":7,"message":"Starting [40] plugins: [usageCollection,telemetryCollectionManager,telemetry,kibanaUsageCollection,securityOss,newsfeed,mapsLegacy,kibanaLegacy,share,legacyExport,embeddable,expressions,data,
home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeVega,visTypeTimelion,timelion,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,esUiShared,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,save
dObjectsManagement,bfetch]"}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:34Z","tags":["listening","info"],"pid":7,"message":"Server running at http://0:5601"}
ambulance_kibana | {"type":"log","@timestamp":"2022-04-04T14:01:34Z","tags":["info","http","server","Kibana"],"pid":7,"message":"http server running at http://0:5601"}
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:34,698Z", "level": "INFO", "component": "o.e.c.m.MetadataMappingService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "[.kibana_1/8CKId7l_T0iddC74lgNj-w] update_mapping [_doc]", "cluster.uuid": "xHuj
kBpFSeiR_6uHyalNdg", "node.id": "vjcbyWJMTn2Xcopkbx_HXw"  }
ambulance_kibana_logstash | WARNING: An illegal reflective access operation has occurred
ambulance_kibana_logstash | WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/tmp/jruby-1/jruby2086728291139417056jopenssl.jar) to field java.security.MessageDigest.provider
ambulance_kibana_logstash | WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
ambulance_kibana_logstash | WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
ambulance_kibana_logstash | WARNING: All illegal access operations will be denied in a future release
ambulance_kibana_logstash | Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
ambulance_kibana_logstash | [2022-04-04T14:01:40,318][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.10.2", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.8+10 on 11.0.8+10 +indy +jit [linux-x86_64]"}
ambulance_kibana_logstash | [2022-04-04T14:01:40,362][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
ambulance_kibana_logstash | [2022-04-04T14:01:40,375][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
ambulance_kibana_logstash | [2022-04-04T14:01:40,661][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
ambulance_kibana_logstash | [2022-04-04T14:01:40,687][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"cd04a6bf-9615-45e3-be8d-172faa7274bd", :path=>"/usr/share/logstash/data/uuid"}
ambulance_kibana_logstash | [2022-04-04T14:01:42,016][INFO ][org.reflections.Reflections] Reflections took 32 ms to scan 1 urls, producing 23 keys and 47 values 
ambulance_kibana_logstash | [2022-04-04T14:01:42,410][WARN ][deprecation.logstash.outputs.elasticsearch] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired E
CS Compatibility mode.
ambulance_kibana_logstash | [2022-04-04T14:01:42,815][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
ambulance_kibana_logstash | [2022-04-04T14:01:42,997][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://elasticsearch:9200/"}
ambulance_kibana_logstash | [2022-04-04T14:01:43,054][INFO ][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
ambulance_kibana_logstash | [2022-04-04T14:01:43,058][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
ambulance_kibana_logstash | [2022-04-04T14:01:43,083][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//elasticsearch:9200"]}
ambulance_kibana_logstash | [2022-04-04T14:01:43,145][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
ambulance_kibana_logstash | [2022-04-04T14:01:43,174][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["/logstash_dir/logstash.conf"], :thread=>"#<Th
read:0x3c182f72 run>"}
ambulance_kibana_logstash | [2022-04-04T14:01:43,200][INFO ][logstash.outputs.elasticsearch][main] Index Lifecycle Management is set to 'auto', but will be disabled - Index Lifecycle management is not installed on your Elasticsearch cluster
ambulance_kibana_logstash | [2022-04-04T14:01:43,202][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"messag
e_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@ti
mestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
ambulance_kibana_logstash | [2022-04-04T14:01:43,234][INFO ][logstash.outputs.elasticsearch][main] Installing elasticsearch template to _template/logstash
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-04T14:01:43,318Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1bd1af51a22e", "message": "adding template [logstash] for index patterns [logstash-*]", "cluster.uuid
": "xHujkBpFSeiR_6uHyalNdg", "node.id": "vjcbyWJMTn2Xcopkbx_HXw"  }
ambulance_kibana_logstash | [2022-04-04T14:01:43,925][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.75}
ambulance_kibana_logstash | [2022-04-04T14:01:43,948][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
ambulance_kibana_logstash | [2022-04-04T14:01:44,008][INFO ][logstash.inputs.udp      ][main][62a945d698b51a6132bd05e47e528d6c402f8383ef39403f40bf57020a0a2dfc] Starting UDP listener {:address=>"127.0.0.1:514"}
ambulance_kibana_logstash | [2022-04-04T14:01:44,012][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
ambulance_kibana_logstash | [2022-04-04T14:01:44,053][INFO ][logstash.inputs.udp      ][main][62a945d698b51a6132bd05e47e528d6c402f8383ef39403f40bf57020a0a2dfc] UDP listener started {:address=>"127.0.0.1:514", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
ambulance_kibana_logstash | [2022-04-04T14:01:44,225][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

I would appreciate if someone can give me clue whats going wrong. Also this is my first ever stackoverflow question. Feedback is highly welcome

logstash

1条答案

按热度按时间
px9o7tmv

px9o7tmv1#

我设法通过简单地将日志发送到logstash来解决这个问题。第一批日志一到达，elasticsearch就自动创建logstash索引。

ambulance_kibana_elasticsearch | {"type": "deprecation", "timestamp": "2022-04-05T09:28:46,990Z", "level": "DEPRECATION", "component": "o.e.d.a.b.BulkRequestParser", "cluster.name": "docker-cluster", "node.name": "b1f1e50666ff", "message": "[types removal] Specifying types in bulk requests is deprecated.", "clu
ster.uuid": "sm_xQ8ZwTFG3JKWfsxkZuQ", "node.id": "XSvHBxFaRSWLercYGL1x-g"  }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-05T09:28:47,010Z", "level": "INFO", "component": "o.e.c.m.MetadataCreateIndexService", "cluster.name": "docker-cluster", "node.name": "b1f1e50666ff", "message": "[logstash-2022.04.05] creating index, cause [auto(bulk api)], templates [log
stash], shards [1]/[1]", "cluster.uuid": "sm_xQ8ZwTFG3JKWfsxkZuQ", "node.id": "XSvHBxFaRSWLercYGL1x-g"  }
ambulance_kibana_elasticsearch | {"type": "server", "timestamp": "2022-04-05T09:28:47,363Z", "level": "INFO", "component": "o.e.c.m.MetadataMappingService", "cluster.name": "docker-cluster", "node.name": "b1f1e50666ff", "message": "[logstash-2022.04.05/bCLws1k0Sziu-2wwPJoxqw] update_mapping [_doc]", "cluster.uu
id": "sm_xQ8ZwTFG3JKWfsxkZuQ", "node.id": "XSvHBxFaRSWLercYGL1x-g"  }

logstash index
感谢评论中的帮助。

赞(0）分享  回复(0）举报  2022-12-09
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com