The information being sent to Google Stackdriver Logging is collected from
- DataNode log files
- ResourceManager log files
- NameNode log files
Below are a few examples of Apache Hadoop events being sent to Google Stackdriver Logging using the BindPlane Log Agent.
Apache Hadoop: 2.7.x+
Follow these steps to setup log collection from Apache Hadoop.
- Locate the file path for General and Yarn DataNode log files.
- Locate the file path for General and Yarn ResourceManager log files.
- Locate the file path for General and Yarn NameNode log files
- Install the BindPlane Log Agent on the host system.
- Login to BindPlane and select the Logs tab.
- Select the Sources tab.
- In the top-right portion of the screen, click on the Add Source Configuration button
- Choose Apache Hadoop
- Fill out the Apache Hadoop log configuration options.
Updated about a year ago