Setting up Monitoring of IBM Integration Bus V10+ Message flow stats (IIBMonFree)

Setting Up Instructions

Version 1.0, 10 November 2015

Applicability

These instructions are for the setup of the free splunk app called Monitoring of IBM Integration Bus V10+ Message flow stats (IIBMonFree) available from Square Bubble. We assume you have installed the app before you reached here.

Instructions

We recommend you read all the instructions before you start the install. Items in quotes such as 'this' are labels that appear literally as directed.

To ensure you have installed the app, navigate to the landing page. This can be done using the one of the following methods

  • Using the URL:

    http://server:port/locale/app/IIBMonFree

  • From the Splunk web app home page (app/launcher/home), press the IIBMonFree app in the App column on the left
  • From the splunk 'Manage Apps' page, click 'launch app' on the IIBMonFree row

It is a good idea to bookmark the landing page as you will return to it many times and it also contains a lot of very useful information and links.

In this guide we will cover a basic setup where the Splunk app will directly connect to the IBM Integration Bus embedded MQTT Server and an advanced setup where data will pass through an intermediary.

Basic Setup

This diagram shows a basic setup:

Basic Setup

To set this up, you need to enable the flow stats to be sent via the embedded MQTT server and enable the MQTT server to listen on an interface that the splunk server can connect to.

By default, we found that the MQTT server is configured to listen on the loopback address (127.0.0.1). Unless you are running splunk on the same server, you need to change the bind address in the file /var/mqsi/components/nodename/config/nodename. To listen on all interfaces set this to 0.0.0.0, otherwise choose the appropriate interface. Then restart the server with the commands:

mqsichangeproperties nodename -b pubsub -o MQTTServer -n enabled -v false

followed by

mqsichangeproperties nodename -b pubsub -o MQTTServer -n enabled -v true

On linux, you can also check this is running using the ps command as follows:

[...]$ ps -ef | grep bipMQTT

To work out what port the MQTT server is listening on, use the following command:

mqsireportproperties nodename -b pubsub -o MQTTServer -n port

If you haven't already, you will need to switch on message flow data by using the following command:

mqsichangeflowstats nodename -s -g -j -c active -o json

The above commands should be run from a member of the mqbrkrs group.

To get the data into splunk,

  1. return to the landing page (via bookmark/shortcut is recommended)
  2. select the inputs link. You can also navigate to this using the splunk menu 'Settings' | 'Data Inputs' | 'IIBMonFree'.
  3. Press the 'New' button
  4. Enter the details of the IIB server as follows:
    • Name : Whatever name you wish to call this input such as IIBServer1
    • Demo : enter false
    • IIBhost : enter the hostname or IP address of the IIB Host
    • IIBport : enter the port used by the MQTT server such as 11883
    • MQTTClientId : this is optional and only needs to be set if you wish to trace this traffic or differentiate it from other MQTT client connections
    • More Setting: Should never be used
  5. Press the 'Next' button. This should immediately start the connection and listening.

Advanced Setup

There are a number of options to setup access to facilitate a variety of deployment models, largely to be used where the splunk server is unable to connect to the IIB Server. This may be as a result of networking or security policies. There are 2 intermediaries that can be used (and combined), which include:

  • IIBMonFreeX, a complementary program which extracts the data either locally or from a remote server and forwards the data to the Splunk server, and/or
  • a Splunk heavy forwarder. Which when this app is loaded can be used to extract the data and/or forward the data to another Splunk server

The main difference between the two options above is that the Splunk forwarder can be chained (i.e. can be deployed in multiple intermediaries), whereas IIBMonFreeX cannot. We do however recommend the use of IIBMonFreeX as this will not incur any additional license costs.

IIBMonFreeX

This diagram shows a possible deployment using IIBMonFreeX

Guides are provided to help install and setup IIBMonFreeX.

Splunk heavy forwarder

Splunk heavy forwarders are instances of Splunk enterprise that simply forward data. They can be used to overcome networking constraints that prevent a direct connection. In our app, they can be deployed as follows:

To enable this deployment, you need to:

  1. Configure the Splunk server as a receiver, using this guide from splunk
  2. Install the intermediary Splunk, with the IIBMonFree app and set up the forwarder using this guide from splunk, for each required intermediary.

You then need to set up the data input on the intermediary as defined in the basic setup. This is only required to be done on the Splunk forwarder that needs to connect to the MQTT server.

Combining Intermediaries

This diagram shows a deployment using both IIBMonfreeX and a Splunk heavy forwarder. This would be applicable where no incoming connections to the MQTT server are permitted and the IIB server does not have connectivity to the Splunk server.

The instructions for setting up this deployment are contained in the previous sections.

Viewing the data

To see the data, return to the landing page and select 'Dashboards' | 'Overview Of Message Flow Volumes'. The data should appear here. You can change the time period being displayed at the top of the dashboard, remember to press the 'Submit' for any change to take effect

In addition to the overview dashboard, there are dashboards that allow you to drill down into the number of messages by:

  • Message Flow,
  • Application,
  • IIB Server and
  • IIB Node.

These are available from the landing page and allow further drill down of the data, as well as the time period