docker-compose logstash pipeline_id:main 오류

안녕하세요.

 

환경에 구애받지 않고, 유연한 이동을 위해 익숙한 docker-compose로 containerizer하던 중..

만난 오류 해결에 대한 글입니다.

 

업무 PC는 로컬에 docker 환경이 구축되지 않아 windows에 설치하여 데모를 만들었습니다.

 

이 기억을 토대로 docker-compose yml을 만들면서 logstash만 shutdown 되는 현상을 파악하는 과정입니다.

 

발생한 오류

logstash_1 | [2021-01-15T16:14:12,164][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 13, column 1 (byte 528) after ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPip elineExt.java:184:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/l ogstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:365:in `block in converge_state'"]}

logstash_1  | [2021-01-15T16:14:12,164][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [
 \\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 13, column 1 (byte 528) after ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPip
elineExt.java:184:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/l
ogstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:365:in `block in converge_state'"]}

• pipeline_id:main 에 대한 오류

 

1. 정상적으로 구동되는 대상과 비교

docker-elk에서 제공하고 있는 기본 포멧이 있지만 제가 발생시킨 오류와는 달라서 문제점이 여긴 아니었습니다.

 

2. docker 실행할 때 directory 및 file 위치 확인

 

ARG ELK_VERSION

# https://www.docker.elastic.co/
FROM docker.elastic.co/logstash/logstash:${ELK_VERSION}

# Add your logstash plugins setup here
# Example: RUN logstash-plugin install logstash-filter-json

RUN ls -al
RUN pwd

 

build 결과

Building logstash
Step 1/4 : ARG ELK_VERSION
Step 2/4 : FROM docker.elastic.co/logstash/logstash:${ELK_VERSION}
 ---> e7ae9aeccc0e
Step 3/4 : RUN ls -al
 ---> Running in acdf52cf890c
total 88
drwxrwsr-x 1 logstash root  4096 Dec  5 03:43 .
drwxr-xr-x 1 root     root  4096 Dec  5 03:42 ..
drwxrwsr-x 2 logstash root  4096 Dec  5 03:42 bin
drwxrwsr-x 1 logstash root  4096 Dec  5 03:43 config
-rw-rw-r-- 1 logstash root  2276 Dec  5 03:21 CONTRIBUTORS
drwxrwsr-x 2 logstash root  4096 Dec  5 03:21 data
-rw-rw-r-- 1 logstash root  4041 Dec  5 03:22 Gemfile
-rw-rw-r-- 1 logstash root 22917 Dec  5 03:22 Gemfile.lock
drwxrwsr-x 9 logstash root  4096 Dec  5 03:42 jdk
drwxrwsr-x 6 logstash root  4096 Dec  5 03:42 lib
drwxrwsr-x 4 logstash root  4096 Dec  5 03:42 logstash-core
drwxrwsr-x 3 logstash root  4096 Dec  5 03:42 logstash-core-plugin-api
drwxrwsr-x 4 logstash root  4096 Dec  5 03:42 modules
drwxr-sr-x 1 logstash root  4096 Dec  5 03:43 pipeline
drwxrwsr-x 3 logstash root  4096 Dec  5 03:42 tools
drwxrwsr-x 4 logstash root  4096 Dec  5 03:42 vendor
drwxrwsr-x 9 logstash root  4096 Dec  5 03:42 x-pack
Removing intermediate container acdf52cf890c
 ---> d54b3df63cd3
Step 4/4 : RUN pwd
 ---> Running in 40bc8850a3d6
/usr/share/logstash
Removing intermediate container 40bc8850a3d6
 ---> c9169cbb1bde

Successfully built c9169cbb1bde
Successfully tagged my-cluster03_logstash:latest

• 각 directory 구조와 위치는 정상으로 판단했습니다.

 

docker-compose.yml 내 설정

services:

# ...
  logstash:
    build:
      context: logstash/
      args:
        ELK_VERSION: $ELK_VERSION
    environment:
      LS_JAVA_OPTS: "-Xmx256m -Xms256m"
    volumes:
      - type: bind
        source: ./logstash/config/logstash.yml
        target: /usr/share/logstash/config/logstash.yml
        read_only: true
      - type: bind
        source: ./logstash/pipeline
        target: /usr/share/logstash/pipeline
        read_only: true
    ports:
      - 5044:5044  # beat
      - 5000-5012  # logback, fluent, other ...
      - 9600-9700  # default ports
    depends_on:
      - es01
      - es02
      - es03
    networks:
      - elk
      
# ...

• logstash.yml: host 설정 및 port, pipeline order 설정
• pipeline: logstash.conf가 있는 directory

pipeline directory 안에 pipeline config가 있을 거라는 착각을 여기서 했던 것 같네요.

 

- 처음 생각한 config 위치

pipelines.yml 위치 이동 전

• pipelines.yml 포함
• conf 파일 묶음

 

- 실제 pipeline directory

logstash container pipeline directory

 

- 실제 config directory

logstash container config directory

 

3. pipelines.yml 위치 이동

pipelines.yml 위치 이동 후

 

4. pipelines.yml 추가

services:

# ...
  logstash:
    build:
      context: logstash/
      args:
        ELK_VERSION: $ELK_VERSION
    environment:
      LS_JAVA_OPTS: "-Xmx256m -Xms256m"
    volumes:
      - type: bind
        source: ./logstash/config/logstash.yml
        target: /usr/share/logstash/config/logstash.yml
        read_only: true
      - type: bind
        source: ./logstash/pipeline
        target: /usr/share/logstash/pipeline
        read_only: true
        
     # pipelines 추가
      - type: bind
        source: ./logstash/config/pipelines.yml
        target: /usr/share/logstash/config/pipelines.yml
        read_only: true
     # pipelines 추가
     
    ports:
      - 5044:5044  # beat
      - 5000-5012  # logback, fluent, other ...
      - 9600-9700  # default ports
    depends_on:
      - es01
      - es02
      - es03
    networks:
      - elk

      
# ...

 

 

기본 구조는 참 많이 봤지만 인지하지 못했던 증거가 아닐까 생각합니다.

 

감사합니다!