BindPlane Documentation

Apache Hadoop

Suggest Edits

Logs Collected

The information being sent to Google Stackdriver Logging is collected from

  • DataNode log files
  • ResourceManager log files
  • NameNode log files
    Below are a few examples of Apache Hadoop events being sent to Google Stackdriver Logging using the BindPlane Log Agent.
11621162

Hadoop General NameNode Logs Example

11761176

Hadoop General Resource Manager Logs Example

11791179

Hadoop Yarn Resource Manager Logs Example

Supported Versions

Apache Hadoop: 2.7.x+

Log Collection Setup

Follow these steps to setup log collection from Apache Hadoop.

Pre-requisites

  • Locate the file path for General and Yarn DataNode log files.
  • Locate the file path for General and Yarn ResourceManager log files.
  • Locate the file path for General and Yarn NameNode log files

Configure an Apache Hadoop Source

  1. Install the BindPlane Log Agent on the host system.
  2. Login to BindPlane and select the Logs tab.
284284

Logs Tab

  1. Select the Sources tab.
249249

Sources Tab

  1. In the top-right portion of the screen, click on the Add Source Configuration button
576576

Add Source Configuration Button

  1. Choose Apache Hadoop
  2. Fill out the Apache Hadoop log configuration options.
495495

Apache Hadoop Log Configuration Form

Updated about 3 years ago