Compiling and Running Hadoop MapReduce using Java 8

This is a post summarizing my experience trying to compile the source code for Hadoop using Java8 and running the Hadoop system using Java 8.

I am using an older version of Hadoop, hadoop-1.0.3 and developer preview JDK 8 from Oracle.

The first thing I had to fix is the build.xml file in HADOOP_HOME directory (the base directory). I get the error saying

“class javac1.8”, it turned out to be a bug in ant library. My ant was 1.6.0. I upgraded my ant to 1.9.2 and it is gone. It is a known issue.


The next thing I had to fix are the source files. Apparently Java 8 have stricter requirement on generic types. I had to fix two files (Note: if you are using a more recent version of Hadoop, these bugs have been fixed for java8 support). 



The fixes are pretty simple and you can google the error messages to find the relevant fix in Hadoop bug reports.


Once I did all these, I get errors when I actually use 1.8 lambda syntax in my source code modifications. I had to set

<property name=”javac.version” value=”1.6″/> to

<property name=”javac.version” value=”1.8″/> to enable using lambda syntax in the source code I did.

After this, I tried to run Hadoop and I got the rror

“Exception in thread “main” java.lang.UnsupportedClassVersionError: org/apache/hadoop/util/PlatformName : Unsupported major.minor version 52.0″

I googled around and realized that the problem is that I compiled the source code using Java 1.8 but I was running it using 1.6. I changed the JAVA_HOME variable set in conf/ to point to the JAVA8 runtime and it works now!

Hope this helps



This entry was posted in Hadoop Research, HJ-Hadoop Improvements. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s