What is the admin logs
Admin logs give you access to easy to use logs via the admin portal. Without this functionality configured, one would need to grep the logs to extract data from them. The admin logs make it easy to see the aggregated network usage and well as account details such as login, sent and received history.
Purpose of this guide
This is the is to detail how to install and configure Logstash and Elasticsearch to enable dashboard logs within the webadmin UI.
⚠ Please Note:
Java is required to be installed. Please follow this guide to install the correct version of Java: (https://www.liquidweb.com/kb/install-java-8-on-centos-7/)
Begin the installation
- Add the elasticsearch repository.
# vi /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1 - Install elasticsearch using yum.
# yum install elasticsearch -y
Start and enable the Elasticsearch service
# systemctl start elasticsearch
# systemctl enable elasticsearch - Add the logstash repository.
# vi /etc/yum.repos.d/logstash.repo
[logstash-2.2]
name=logstash repository for 2.2 packages
baseurl=http://packages.elasticsearch.org/logstash/2.2/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1 - Install Logstash using yum.
# yum install logstash -y
Start and enable the Logstash service.
# systemctl start logstash
Further server side configuration is now required to enable logging to work within the webadmin. Please see below for steps to install and activate the logging within the webadmin.
# systemctl enable logstash - Create the following directory:
# mkdir /usr/share/atmail/elasticsearch
Change the permissions on the newly created directory.
# chown elasticsearch:elasticsearch /usr/share/atmail/elasticsearch
- Add the following to elasticsearch.yml
# vi /etc/elasticsearch/elasticsearch.yml
...
path.data: /usr/share/atmail/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: localhost
http.port: 9200 - Make sure the firewall is opened for port 9200.
# firewall-cmd --perm --add-port 9200/tcp --permanent
# firewall-cmd --reload
# systemctl restart firewalld - Add template
- Change file permission.
# chmod 755 /usr/share/atmail/mailserver/webui/install/elasticsearch_template.txt
- Execute the file.
# /usr/share/atmail/mailserver/webui/install/elasticsearch_template.txt
- To verify elasticsearch, restart elasticsearch and ensure that it's running:
# systemctl restart elasticsearch
# systemctl status elasticsearch - Then run the command:
# curl -XGET http://localhost:9200
You should get a JSON response (note: your output may look slightly different depending on the version):
{ "name" : "12frSd3",
Troubleshoot: check log in /var/log/elasticsearch.
"cluster_name" : "elasticsearch",
"cluster_uuid" : "LkNZ_Bw1Sz-UftFWE7cysg",
"version" : {
"number" : "[VERSION]",
"build_hash" : "877a590",
"build_date" : "2018-04-12T16:25:14.838Z",
"build_snapshot" : false,
"lucene_version" : "[VERSION]"
},
"tagline" : "You Know, for Search"
}
- Change file permission.
- Verify logstash
- Make sure Logstash works with command line
# sudo -u atmail /opt/logstash/bin/logstash -e 'input { stdin{} } output { stdout{} }'
- Wait until you see:
... Settings: Default pipeline workers: 1 Logstash startup completed
- Type 'hello world' as input, then you should see some output as below
... Hello world
It is NOT necessary for Logstash to run as root:root. It should work as long as it has enough permission to access the syslog files. Please adjust the permission depending our your system configuration.
2018-05-03T05:38:48.530Z git.local Hello world
^C
When Logstash running at non root user ( default will be logstash:logstash ).
Logstash will log errors to /var/log/logstash/logstash.log when permissions are not enough.
If it even has no permission to write to /var/log/logstash/*, the service stops. - Copy the following into the logstash file
# vi /etc/default/logstash
... LS_USER="root" LS_GROUP="root" ...
- Copy, atmail-maillog.conf, atmail-apiserver.conf, mailserver.conf, atmail-output.conf to /etc/logstash/conf.d
# cp /usr/share/atmail/mailserver/webui/install/atmail-*.conf /etc/logstash/conf.d
- Check output Config
# vi /etc/logstash/conf.d/atmail-output.conf
- Make sure the hosts value are correct
output { elasticsearch { hosts => ["localhost:9200"] index => "logstash-%{type}-%{+YYYYMM}" document_type => "doc_%{type}" } stdout { codec => rubydebug } }
- Check the apiserver config
# vi /etc/logstash/conf.d/atmail-apiserver.conf
Make sure the path value are correct.
input { file { path => "/var/log/atmail/api-audit.log" ... } } filter { ... }
- Check the mail server config
# vi /etc/logstash/conf.d/atmail-mailserver.conf
Make sure the path value are correct.
input { file { path => "/var/log/atmail/ms-audit.log" ... } } filter { ... }
- Check maillog config
# vi /etc/logstash/conf.d/atmail-maillog.conf
Make sure the path value are correctinput { file { path => "/var/log/maillog" ... } } filter { ... }
- Restart logstash service
# systemctl restart logstash
- Update exim to log the delivery size
Ansible is used to build the exim config file. For this reason, we need to update the mailserver database and then use the admin to publish to ensure that the correct config is set in the exim config file.
Login to mariadb, use the mailserver database and run the following update command.# mysql -uroot -p MariaDB [(none)]> use mailserver MariaDB [mailserver]> update inventory set configValue = '+tls_peerdn +subject +smtp_confirmation +sender_on_delivery +delivery_size' where configVariable='log_selector';
Restart mariadb and exim
# systemctl restart mariadb exim
- In the atmail admin, navigate to 'Services' and press the publish button.
- Enable logs in the dashboard
Enable logs in the admin portal by navigation to Admin > Settings > Log settings > and find the Enable dashboard logs option. Set this to on and ensure that the 'Log storage setting' is localhost:9200. You should now be able to access logs - View system logs.
- View account logs.
The logs have been successfully configured.
It is recommend you visit the below for further configuration of your installation:
Comments