In this series of posts, I run through the process of aggregating logs with Wildfly, Filebeat, ElasticSearch and Kibana.

In Log Agrgegation - ElasticSearch & Kibana I went through creating the ElasticSearch domain but now we need to create some logs to play with.

For this post, I setup a simple Wildfly instance with a basic Java application to produce some logs.

I have provisioned a simple EC2 instance with Wildfly and deployed a really simple application to it.

In a setting requiring this process to be more repeatable I would typically build out a Cloudformation template with a solid AWS::CloudFormation::Init section (in the LaunchConfiguration Metadata) to achieve this, but in this case with a throwaway instance, the following UserData block will suffice:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

#!/bin/bash -x
yum remove java-1.7.0-openjdk -y
yum install java-1.8.0-openjdk -y
aws --region eu-west-3 s3 cp s3://log-agg-qtjsda123/wildfly-10.1.0.Final.zip .
unzip wildfly-10.1.0.Final.zip -d /opt/wildfly
aws --region eu-west-3 s3 cp s3://log-agg-qtjsda123/standalone.xml /opt/wildfly/wildfly-10.1.0.Final/standalone/configuration/standalone.xml
aws --region eu-west-3 s3 cp s3://log-agg-qtjsda123/wildfly.conf /etc/default/wildfly.conf
cp /opt/wildfly/wildfly-10.1.0.Final/docs/contrib/scripts/init.d/wildfly-init-redhat.sh /etc/init.d/wildfly
chmod 755 /etc/init.d/wildfly
aws --region eu-west-3 s3 cp s3://log-agg-qtjsda123/logaggregation.war /opt/wildfly/wildfly-10.1.0.Final/standalone/deployments/log-agg.war
service wildfly start

That’s probably quite a lot to take in, but briefly:

• lines 2 & 3 remove java 7, install java 8
• lines 4 & 5 downloads the Wildfly zip and extracts it to the /opt/wildfly directory
• lines 6 & 7 downloads a couple custom configuration files to tweak Wildfly
• lines 8 & 9 copies a template init script to /etc/init.d/wildfly to configure the Wildfly service
• line 10 grabs the simple Java application, it will get auto-deployed with the context path log-agg when Wildfly starts
• finally, line 11 starts the wildfly servce

The application simply logs to Wildfly’s /var/log/server.log file whenever the /log-agg/healthcheck URL is hit, which produces lines like:

2018-01-31 13:32:38,747 DEBUG [default task-2] com.thecuriousdev.logaggregation.healthcheck.HealthCheckServlet Hit /healthcheck

If this was your only server then it might not be out of a question to perhaps simply SSH to the box and tail the logs, but in a microservices style deployment this becomes inpractical real quick.

Of course, with the tailing example, you will likely be logging into a new box every time you do a deploy which gets tired pretty quickly :)

Next post, I’ll go into utilising Filebeat to get our logs from the log file to our ElasticSearch Domain and Kibana.