PaaS-TA-5.5.0
Search…
Monasca Server 가이드

Monasca Server 설치 가이드

1. 개요

1.1. 문서 목적

본 문서(설치가이드)는, IaaS(Infrastructure as a Service) 중 하나인 Openstack 기반의 Cloud 서비스 상태 및 자원 정보, 그리고 VM Instance의 시스템 정보를 수집 및 관리하고, 사전에 정의한 Alarm 규칙에 따라 실시간으로 모니터링하여 관리자에게 관련 정보를 제공하기 위한 서버를 설치하는데 그 목적이 있다.

1.2. 범위

본 문서의 범위는 Openstack 모니터링을 위한 오픈소스인 Monasca 제품군의 설치 및 관련 S/W(Kafka, Zookeeper, InfluxDB, MariaDB) 설치하기 위한 내용으로 한정되어 있다.

1.3. 참고자료

2. Pre-Requisite(전제조건)

  • Monasca Server를 설치하기 위해서는 Bare Metal 서버 또는 Openstack 에서 생성한 Instance(Ubuntu 기준, Flavor - x1.large 이상)가 준비되어 있어야 한다.
  • Openstack Cross-tenant 설정이 되어 있어야 한다.
    Reference : Cross-Project(Tenant) 사용자 추가 및 권한 부여 (openstack 기준)
  • Monasca Server 설치에 필요한 프로그램 리스트 및 버전은 아래 사항을 참조한다.
  • Monasca Server 를 설치하기에 필요한 프로그램을 사전에 설치한다.
  • 설치 환경은 Ubuntu 18.04 , OpenStack Stein 기준으로 작성하였다.
※ 설치 프로그램 리스트 및 버전 참조 (순서)
  • repo branch 정책에 따라 버전이 변경될 수 있음.
    • INFLUXDB_VERSION=1.3.3-alpine
    • INFLUXDB_INIT_VERSION=1.0.1
    • MYSQL_VERSION=5.7
    • MYSQL_INIT_VERSION=1.5.4
    • MEMCACHED_VERSION=1.5.0-alpine
    • CADVISOR_VERSION=v0.27.1
    • ZOOKEEPER_VERSION=3.4
※ 설치 전 사전에 설치되어 있어야 하는 프로그램
  • install git
    1
    sudo apt-get update
    2
    sudo apt-get install -y git
    Copied!
  • install python
    1
    sudo apt-get install python-keystoneclient
    Copied!

3. docker 설치

  • Docker Key 등록
    1
    $ sudo apt update
    2
    $ sudo apt install apt-transport-https ca-certificates curl software-properties-common
    3
    $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add –
    Copied!
  • Docker repository 정보 등록
    1
    $ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable
    2
    $ sudo apt update
    3
    $ apt-cache policy docker-ce
    Copied!
  • Docker 설치
    1
    $ sudo systemctl status docker
    Copied!
  • Docker 설치 확인
1
$ sudo apt install docker-ce
2
3
docker.service - Docker Application Container Engine
4
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
5
Active: active (running) since Mon 2019-06-17 01:40:41 UTC; 11s ago
6
Docs: https://docs.docker.com
7
Main PID: 3821 (dockerd)
8
Tasks: 10
9
CGroup: /system.slice/docker.service
10
└─3821 /usr/bin/dockerd -H fd:/ --containerd=/run/containerd/containerd.sock
Copied!
  • Docker-Compose 설치
1
$ sudo apt install docker-compose
Copied!

4. Monasca-Docker 설치

  • Openstack Keyston network route open
1
$ sudo route add -net 172.31.30.0/24 gw 10.0.201.254
Copied!
  • Monasa-Docker 설치파일 다운로드
1
$ mkdir workspace & cd workspace
2
$ git clone https://github.com/monasca/monasca-docker.git
Copied!
  • Monasa-Docker docker-compose.yml 파일 변경
1
$ cd monasca-docker
2
$ vi docker-compose.yml
3
4
version: '3'
5
services:
6
7
memcached:
8
image: memcached:${MEMCACHED_VERSION}
9
environment:
10
LOGSTASH_FIELDS: "service=memcached"
11
12
influxdb:
13
image: influxdb:${INFLUXDB_VERSION}
14
environment:
15
LOGSTASH_FIELDS: "service=influxdb"
16
ports:
17
- "8086:8086"
18
influxdb-init:
19
image: monasca/influxdb-init:${INFLUXDB_INIT_VERSION}
20
environment:
21
LOGSTASH_FIELDS: "service=influxdb-init"
22
depends_on:
23
- influxdb
24
25
# cadvisor will allow host metrics to be collected, but requires significant
26
# access to the host system
27
# if this is not desired, the following can be commented out, and the CADVISOR
28
# environment variable should be set to "false" in the `agent-collector`
29
# block - however no metrics will be collected
30
cadvisor:
31
image: google/cadvisor:${CADVISOR_VERSION}
32
environment:
33
LOGSTASH_FIELDS: "service=cadvisor"
34
volumes:
35
- "/:/rootfs:ro"
36
- "/var/run:/var/run:rw"
37
- "/sys:/sys:ro"
38
- "/var/lib/docker:/var/lib/docker:ro"
39
40
agent-forwarder:
41
image: monasca/agent-forwarder:${MON_AGENT_FORWARDER_VERSION}
42
environment:
43
NON_LOCAL_TRAFFIC: "true"
44
LOGSTASH_FIELDS: "service=monasca-agent-forwarder"
45
OS_AUTH_URL: http://{keystone api ip}:{keystone port}/v3 # openstack keystone(identity) api ip, port
46
OS_USERNAME: admin # openstack admin account
47
OS_PASSWORD: password # openstack admin password
48
OS_PROJECT_NAME: admin # openstack admin project
49
extra_hosts:
50
- "monasca:192.168.0.103" # monasca-api host ip
51
- "control:192.168.56.103" # openstack control node host:ip
52
- "compute:192.168.56.102" # openstack compute node host:ip
53
- "compute2:192.168.56.101" # openstack compute node host:ip
54
- "compute3:192.168.56.104" # openstack compute node host:ip
55
56
agent-collector:
57
image: monasca/agent-collector:${MON_AGENT_COLLECTOR_VERSION}
58
restart: on-failure
59
environment:
60
AGENT_HOSTNAME: "docker-host"
61
FORWARDER_URL: "http://agent-forwarder:17123"
62
CADVISOR: "true"
63
CADVISOR_URL: "http://cadvisor:8080/"
64
LOGSTASH_FIELDS: "service=monasca-agent-collector"
65
MONASCA_MONITORING: "true"
66
MONASCA_LOG_MONITORING: "false"
67
OS_AUTH_URL: http://{keystone api ip}:{keystone port}/v3 # keystone(identity) api ip, port
68
OS_USERNAME: admin # openstack admin account
69
OS_PASSWORD: password # openstack admin password
70
OS_PROJECT_NAME: admin # openstack admin project
71
cap_add:
72
- FOWNER
73
volumes:
74
- "/:/rootfs:ro"
75
extra_hosts:
76
- "control:192.168.56.103" # openstack control node host:ip
77
- "compute:192.168.56.102" # openstack compute node host:ip
78
- "compute2:192.168.56.101" # openstack compute node host:ip
79
- "compute3:192.168.56.104" # openstack compute node host:ip
80
81
alarms:
82
image: monasca/alarms:${MON_ALARMS_VERSION}
83
environment:
84
LOGSTASH_FIELDS: "service=monasca-alarms"
85
OS_AUTH_URL: http://{keystone api ip}:{keystone port}/v3 # keystone(identity) api ip, port
86
OS_USERNAME: admin # openstack admin account
87
OS_PASSWORD: password # openstack admin password
88
OS_PROJECT_NAME: admin # openstack admin project
89
depends_on:
90
# - keystone
91
- monasca
92
extra_hosts:
93
- "control:192.168.56.103" # openstack control node host:ip
94
- "compute:192.168.56.102" # openstack compute node host:ip
95
- "compute2:192.168.56.101" # openstack compute node host:ip
96
- "compute3:192.168.56.104" # openstack compute node host:ip
97
98
zookeeper:
99
image: zookeeper:${ZOOKEEPER_VERSION}
100
environment:
101
LOGSTASH_FIELDS: "service=zookeeper"
102
restart: on-failure
103
104
kafka:
105
image: monasca/kafka:${MON_KAFKA_VERSION}
106
environment:
107
KAFKA_DELETE_TOPIC_ENABLE: "true"
108
LOGSTASH_FIELDS: "service=kafka"
109
restart: on-failure
110
depends_on:
111
- zookeeper
112
kafka-watcher:
113
image: monasca/kafka-watcher:${MON_KAFKA_WATCHER_VERSION}
114
environment:
115
BOOT_STRAP_SERVERS: "kafka"
116
PROMETHEUS_ENDPOINT: "0.0.0.0:8080"
117
LOGSTASH_FIELDS: "service=kafka-watcher"
118
depends_on:
119
- kafka
120
ports:
121
- "18080:8080"
122
kafka-init:
123
image: monasca/kafka-init:${MON_KAFKA_INIT_VERSION}
124
environment:
125
ZOOKEEPER_CONNECTION_STRING: "zookeeper:2181"
126
KAFKA_TOPIC_CONFIG: segment.ms=900000 # 15m
127
KAFKA_CREATE_TOPICS: "\
128
metrics:64:1,\
129
alarm-state-transitions:12:1,\
130
alarm-notifications:12:1,\
131
retry-notifications:3:1,\
132
events:12:1,\
133
kafka-health-check:1:1,\
134
60-seconds-notifications:3:1"
135
LOGSTASH_FIELDS: "service=kafka-init"
136
depends_on:
137
- zookeeper
138
139
mysql:
140
image: mysql:${MYSQL_VERSION}
141
environment:
142
MYSQL_ROOT_PASSWORD: secretmysql
143
LOGSTASH_FIELDS: "service=mysql"
144
ports:
145
- "3306:3306"
146
mysql-init:
147
image: monasca/mysql-init:${MYSQL_INIT_VERSION}
148
environment:
149
MYSQL_INIT_DISABLE_REMOTE_ROOT: "false"
150
MYSQL_INIT_RANDOM_PASSWORD: "false"
151
LOGSTASH_FIELDS: "service=mysql-init"
152
153
# keystone 부분 주석 처리
154
# keystone:
155
# image: monasca/keystone:${MON_KEYSTONE_VERSION}
156
# environment:
157
# KEYSTONE_HOST: keystone
158
# KEYSTONE_PASSWORD: secretadmin
159
# KEYSTONE_DATABASE_BACKEND: mysql
160
# KEYSTONE_MYSQL_HOST: mysql
161
# KEYSTONE_MYSQL_USER: keystone
162
# KEYSTONE_MYSQL_PASSWORD: keystone
163
# KEYSTONE_MYSQL_DATABASE: keystone
164
# LOGSTASH_FIELDS: "service=keystone"
165
# depends_on:
166
# - mysql
167
# ports:
168
# - "5001:5000"
169
# - "35357:35357"
170
171
monasca-sidecar:
172
image: timothyb89/monasca-sidecar:${MON_SIDECAR_VERSION}
173
environment:
174
LOGSTASH_FIELDS: "service=monasca-sidecar"
175
176
monasca:
177
image: monasca/api:${MON_API_VERSION}
178
environment:
179
SIDECAR_URL: http://monasca-sidecar:4888/v1/ingest
180
LOGSTASH_FIELDS: "service=monasca-api"
181
KEYSTONE_IDENTITY_URI: http://{keystone api ip}:{keystone port}/v3 # keystone(identity) api ip, port
182
KEYSTONE_AUTH_URI: http://{keystone api ip}:{keystone port}/v3 # keystone(identity) api ip, port
183
KEYSTONE_ADMIN_USER: admin # openstack admin account
184
KEYSTONE_ADMIN_PASSWORD: password # openstack admin password
185
depends_on:
186
- influxdb
187
# - keystone
188
- mysql
189
- zookeeper
190
- kafka
191
- monasca-sidecar
192
- memcached
193
ports:
194
- "8070:8070"
195
extra_hosts:
196
- "control:192.168.56.103" # openstack control node host:ip
197
- "compute:192.168.56.102" # openstack compute node host:ip
198
- "compute2:192.168.56.101" # openstack compute node host:ip
199
- "compute3:192.168.56.104" # openstack compute node host:ip
200
monasca-persister:
201
image: monasca/persister:${MON_PERSISTER_VERSION}
202
environment:
203
LOGSTASH_FIELDS: "service=monasca-persister"
204
restart: on-failure
205
depends_on:
206
- monasca
207
- influxdb
208
- zookeeper
209
- kafka
210
211
thresh:
212
image: monasca/thresh:${MON_THRESH_VERSION}
213
environment:
214
NO_STORM_CLUSTER: "true"
215
WORKER_MAX_HEAP_MB: "256"
216
LOGSTASH_FIELDS: "service=monasca-thresh"
217
depends_on:
218
- zookeeper
219
- kafka
220
221
monasca-notification:
222
image: monasca/notification:${MON_NOTIFICATION_VERSION}
223
environment:
224
NF_PLUGINS: "webhook"
225
LOGSTASH_FIELDS: "service=monasca-notification"
226
STATSD_HOST: monasca-statsd
227
STATSD_PORT: 8125
228
depends_on:
229
- monasca
230
- zookeeper
231
- kafka
232
- mysql
233
234
grafana:
235
image: monasca/grafana:${MON_GRAFANA_VERSION}
236
environment:
237
GF_AUTH_BASIC_ENABLED: "false"
238
GF_USERS_ALLOW_SIGN_UP: "true"
239
GF_USERS_ALLOW_ORG_CREATE: "true"
240
GF_AUTH_KEYSTONE_ENABLED: "true"
241
GF_AUTH_KEYSTONE_AUTH_URL: http://{KEYSTONE_IP}:25000
242
GF_AUTH_KEYSTONE_VERIFY_SSL_CERT: "false"
243
GF_AUTH_KEYSTONE_DEFAULT_DOMAIN: "Default"
244
LOGSTASH_FIELDS: "service=grafana"
245
ports:
246
- "3000:3000"
247
depends_on:
248
# - keystone
249
- monasca
250
extra_hosts:
251
- "control:192.168.56.103" # openstack control node host:ip
252
- "compute:192.168.56.102" # openstack compute node host:ip
253
- "compute2:192.168.56.101" # openstack compute node host:ip
254
- "compute3:192.168.56.104" # openstack compute node host:ip
255
256
grafana-init:
257
image: monasca/grafana-init:${MON_GRAFANA_INIT_VERSION}
258
environment:
259
LOGSTASH_FIELDS: "service=grafana-init"
260
depends_on:
261
- grafana
262
263
monasca-statsd:
264
image: monasca/statsd:${MON_STATSD_VERSION}
265
environment:
266
FORWARDER_URL: http://agent-forwarder:17123
267
LOG_LEVEL: WARN
268
ports:
269
- "8125/udp"
Copied!
  • Monasca-Docker Server 설치 및 시작
1
$ sudo docker-compose up -d
Copied!

5. Elasticserarch 서버 설치

  • dependencies 설치
1
$ sudo apt-get update
2
$ sudo apt-get install openjdk-8-jdk
Copied!
  • Elasticsearch 설치
1
$ wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/2.3.1/elasticsearch-2.3.1.deb
2
$ dpkg -i elasticsearch-2.3.1.deb
Copied!
  • 사용자 그룹 추가 - Elasticsearch
1
$ sudo usermod -a -G elasticsearch “사용자 계정”
Copied!
  • Elasticsearch configuration 파일 수정
1
$ cd /etc/elasticsearch && sudo vi elasticsearch.yml
2
3
# Use a descriptive name for your cluster:
4
#
5
cluster.name: escluster1
6
7
8
# Use a descriptive name for the node:
9
#
10
node.name: node-1
11
12
# Lock the memory on startup:
13
#
14
bootstrap.mlockall: true
15
16
17
# Set the bind address to a specific IP (IPv4 or IPv6):
18
#
19
network.host: 0.0.0.0
20
21
22
# Set a custom port for HTTP:
23
http.port: 9200
24
25
26
index.number_of_shards: 1
27
index.number_of_replicas: 0
Copied!
  • Elasticsearch service 파일 수정
1
$ sudo vi /usr/lib/systemd/system/elasticsearch.service
2
3
# Specifies the maximum number of bytes of memory that may be locked into RAM
4
# Set to "infinity" if you use the 'bootstrap.memory_lock: true' option
5
# in elasticsearch.yml and 'MAX_LOCKED_MEMORY=unlimited' in /etc/default/elasticsearch
6
LimitMEMLOCK=infinity
Copied!
  • Elasticsearch default 파일 수정
1
$ sudo vi /etc/default/elasticsearch
2
3
# The maximum number of bytes of memory that may be locked into RAM
4
# Set to "unlimited" if you use the 'bootstrap.memory_lock: true' option
5
# in elasticsearch.yml.
6
# When using Systemd, the LimitMEMLOCK property must be set
7
# in /usr/lib/systemd/system/elasticsearch.service
8
MAX_LOCKED_MEMORY=unlimited
Copied!
  • Elasticsearch 서비스 시작
1
$ sudo service elasticsearch start
Copied!
  • Elasticserarch 서버 가동 여부 확인
1
$ netstat -plntu | grep 9200
Copied!
  • mlockall 정보가 “enabled” 되었는지 확인
1
$ curl -XGET 'localhost:9200/_nodes?filter_path=**.mlockall&pretty'
Copied!

6. logstash 설치

  • logstash repository 추가
1
$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
2
OK
3
$ echo 'deb http://packages.elastic.co/logstash/2.2/debian stable main' | sudo tee /etc/apt/sources.list.d/logstash-2.2.x.list
4
deb http://packages.elastic.co/logstash/2.2/debian stable main
Copied!
  • logstash 설치
1
$ apt-get update
2
$ apt-get install -y logstash
Copied!
  • /etc/hosts 파일 수정
1
$ sudo vi /etc/hosts
2
3
“private network ip” “hostname”
4
ex) 192.168.0.103 host logstash elasticsearch
Copied!
  • SSL certificate 파일 생성
1
$ cd /etc/logstash
2
$ sudo openssl req -subj /CN=”hostaname” -x509 -days 3650 -batch -nodes -newkey rsa:4096 -keyout logstash.key -out logstash.crt
Copied!
  • filebeat-input.conf 파일 생성
1
$ cd /etc/logstash
2
$ sudo vi conf.d/filebeat-input.conf
3
4
input {
5
beats {
6
port => 5443
7
type => syslog
8
ssl => true
9
ssl_certificate => "/etc/logstash/logstash.crt"
10
ssl_key => "/etc/logstash/logstash.key"
11
}
12
}
Copied!
  • syslog-filter.conf 파일 생성
1
$ cd /etc/logstash
2
$ sudo vi conf.d/syslog-filter.conf
3
4
filter {
5
if [type] == "syslog" {
6
grok {
7
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
8
add_field => [ "received_at", "%{@timestamp}" ]
9
add_field => [ "received_from", "%{host}" ]
10
}
11
date {
12
match => [ "syslog_timestamp", "yyyy-MM-dd HH:mm:ss.SSS" ] # openstack의 log output에 따라 포멧 변경필요.
13
}
14
}
15
}
Copied!
  • output-elasticsearch.conf 파일 생성
1
$ cd /etc/logstash
2
$ sudo vi conf.d/output-elasticsearch.conf
3
4
output {
5
elasticsearch { hosts => ["”your elastic ip”:9200"] # 설치된 환경의 IP 정보
6
hosts => "”your elastic ip”:9200" # 설치된 환경의 IP 정보
7
manage_template => false
8
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
9
document_type => "%{[@metadata][type]}"
10
}
11
}
Copied!
  • logstash 서비스 시작
1
$ sudo service logstash start
Copied!
  • logstash 서비스 확인
1
$ sudo service logstash start
Copied!

7. Reference : Cross-Project(Tenant) 사용자 추가 및 권한 부여

Openstack 기반으로 생성된 모든 Project(Tenant)의 정보를 하나의 계정으로 수집 및 조회하기 위해서는 Cross-Tenant 사용자를 생성하여, 각각의 Project(Tenant)마다 조회할 수 있도록 멤버로 등록한다. Openstack Cli를 이용하여 Cross-Tenant 사용자를 생성한 후, Openstack Horizon 화면으로 통해 각각의 프로젝트 사용자 정보에 생성한 Cross-Tenant 사용자 및 권한을 부여한다. 1. Cross-Tenant 사용자 생성
1
$ openstack user create --domain default --password-prompt monasca-agent
2
$ openstack role create monitoring-delegate
Copied!
  1. 1.
    Project 사용자 추가
    각각의 프로젝트 멤버관리에 추가한 Cross-Tenant 사용자 정보를 등록한다.
    추가한 Cross-Tenant 사용자를 선택 후, 생성한 Role을 지정한다.
Last modified 8mo ago