2014年7月15日 星期二

MapReduce call native library: A MEMO

Reference:
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/NativeLibraries.html

My revision:
This example shows you how to distribute a shared library, mylib.so, and load it from a MapReduce task.

1. First copy the library to the HDFS, say path: hdfs://ip:port/libraries/mylib.so

2. The job launching program should contain the following: 
Job job = new Job(conf)
job.addFileToClassPath(new Path(" hdfs://ip:port/libraries/mylib.so"));

3. The mapper and the reducer can contain: System.loadLibrary("mylib.so");

沒有留言:

張貼留言