BindPlane Documentation

Apache Hadoop

Suggest Edits

Logs Collected

The information being sent to Google Stackdriver Logging is collected from

  • DataNode log files
  • ResourceManager log files
  • NameNode log files
    Below are a few examples of Apache Hadoop events being sent to Google Stackdriver Logging using the BindPlane Log Agent.
Hadoop General NameNode Logs ExampleHadoop General NameNode Logs Example

Hadoop General NameNode Logs Example

Hadoop General Resource Manager Logs ExampleHadoop General Resource Manager Logs Example

Hadoop General Resource Manager Logs Example

Hadoop Yarn Resource Manager Logs ExampleHadoop Yarn Resource Manager Logs Example

Hadoop Yarn Resource Manager Logs Example

Supported Versions

Apache Hadoop: 2.7.x+

Log Collection Setup

Follow these steps to setup log collection from Apache Hadoop.

Pre-requisites

  • Locate the file path for General and Yarn DataNode log files.
  • Locate the file path for General and Yarn ResourceManager log files.
  • Locate the file path for General and Yarn NameNode log files

Configure an Apache Hadoop Source

  1. Install the BindPlane Log Agent on the host system.
  2. Login to BindPlane and select the Logs tab.
Logs TabLogs Tab

Logs Tab

  1. Select the Sources tab.
Sources TabSources Tab

Sources Tab

  1. In the top-right portion of the screen, click on the Add Source Configuration button
Add Source Configuration ButtonAdd Source Configuration Button

Add Source Configuration Button

  1. Choose Apache Hadoop
  2. Fill out the Apache Hadoop log configuration options.
Apache Hadoop Log Configuration FormApache Hadoop Log Configuration Form

Apache Hadoop Log Configuration Form

Updated over 2 years ago