参考链接

filebat:
https://www.elastic.co/cn/downloads/beats/filebeat
https://www.elastic.co/cn/products/beats/filebeat

Elastic Stack :
https://www.elastic.co/cn/solutions/logging

ELK 架构和 Filebeat 工作原理详解:
https://www.ibm.com/developerworks/cn/opensource/os-cn-elk-filebeat/index.html

Filebeat的架构分析、配置解释与示例:
https://zhuanlan.zhihu.com/p/23049700

Elastic 技术栈之 Filebeat:
https://www.cnblogs.com/jingmoxukong/p/8185321.html

elasticsearch docker compose:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

docker compose:
https://github.com/maxyermayank/docker-compose-elasticsearch-kibana/blob/master/docker-compose.yml
https://gist.github.com/shreyu86/90377c42a78f1ff2d0e17a9ad502ac31

docker es demo:
https://github.com/rusher81572/elasticsearch-demo/blob/master/docker-compose.yml

elk 设置密码:
https://blog.csdn.net/qq_41980563/article/details/88725584

install elasticsearch with docker:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

kibana compose:
https://discuss.elastic.co/t/kibana-error-connecting-to-es-on-docker-cannot-revive-connection/133335/4

docker-elk:
https://github.com/deviantony/docker-elk/wiki/External-applications

logstash:
https://www.elastic.co/guide/en/logstash/7.1/input-plugins.html

es+kib+logtash+filebeat实践
https://blog.csdn.net/ma15732625261/article/details/79821160
https://cloud.tencent.com/developer/article/1353068

docker-compose配置

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.1.0
    container_name: es01
    environment:
      - node.name=es01
      - discovery.seed_hosts=es02
      - cluster.initial_master_nodes=es01,es02
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - esnet
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.1.0
    container_name: es02
    environment:
      - node.name=es02
      - discovery.seed_hosts=es01
      - cluster.initial_master_nodes=es01,es02
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata02:/usr/share/elasticsearch/data
    networks:
      - esnet
  kibana:
    image: 'docker.elastic.co/kibana/kibana:7.1.0'
    container_name: kibana
    environment:
      SERVER_NAME: localhost
      ELASTICSEARCH_HOSTS: http://es01:9200
    ports:
      - '5601:5601'
    networks:
      - esnet
  headPlugin:
    image: 'mobz/elasticsearch-head:5'
    container_name: head
    ports:
      - '9002:9100'
    networks:
      - esnet
  filebeat:
    image: docker.elastic.co/beats/filebeat:7.1.0
    command: --strict.perms=false
    environment:
      - setup.kibana.host=kibana:5601
      - output.elasticsearch.hosts=["http://127.0.0.1:9200"]
    ports:
      - 9001:9000
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - esnet
volumes:
  esdata01:
    driver: local
  esdata02:
    driver: local

networks:
  esnet:

使用php hdfs 读取大文件

最近用hdfs上传了一个2.8g的视频,发现用原始的read方式会把整个视频加载到内存,会非常慢。所以使用hdfs的web api里面的按文件偏移量加载的方式。 参考:htt...

阅读全文

php获取字符串中的数字

$str = 'aasd222ssdd32'; $str = preg_replace("/[^0-9]/","",$str); echo $str;

阅读全文

Apache Hadoop Hdfs 集群安装(2台),并使用php操作hdfs上传、读取文件

Hadoop Java版本 https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions Apache Hadoop 3.x现在仅支持Java 8 从2.7.x到2.x的Apache Had...

阅读全文

欢迎留言