Logs Collected
The information being sent to Google Stackdriver Logging is collected from
- DataNode log files
- ResourceManager log files
- NameNode log files
Below are a few examples of Apache Hadoop events being sent to Google Stackdriver Logging using the BindPlane Log Agent.
Hadoop General NameNode Logs Example
Hadoop General Resource Manager Logs Example
Hadoop Yarn Resource Manager Logs Example
Supported Versions
Apache Hadoop: 2.7.x+
Log Collection Setup
Follow these steps to setup log collection from Apache Hadoop.
Pre-requisites
- Locate the file path for General and Yarn DataNode log files.
- Locate the file path for General and Yarn ResourceManager log files.
- Locate the file path for General and Yarn NameNode log files
Configure an Apache Hadoop Source
- Install the BindPlane Log Agent on the host system.
- Login to BindPlane and select the Logs tab.
Logs Tab
- Select the Sources tab.
Sources Tab
- In the top-right portion of the screen, click on the Add Source Configuration button
Add Source Configuration Button
- Choose Apache Hadoop
- Fill out the Apache Hadoop log configuration options.
Apache Hadoop Log Configuration Form
Updated about a year ago