Monthly Archives: February 2014

Presentation Feedback for Engi 600

Recently I gave a 10 min talk at ENGI 600, a course taught by Prof. Janice Hewitt at Rice University. My slides can be found here HJ-Hadoop-presentation-10-min. This is a post summarizing the feedback from the students in the class. Note: … Continue reading

Posted in Presentation | Leave a comment

How does Kill a task work in Hadoop

This is a post summarizing how Hadoop kills a task when something went wrong with the task. (Ran for too long, etc). It is important for me because I have multiple virtual JVMs (each hoisting a map task) running in … Continue reading

Posted in Hadoop Research, HJ-Hadoop Improvements | Leave a comment

Further changes to make Parallel Jvm in Hadoop work

This is a post looking into possible changes to fix an exception that I am seeing in running parallel JVM in hadoop. The issue is that some of the map attempts crashed and the reduce tasks are not able to … Continue reading

Posted in Habanero Java, Hadoop Research, Uncategorized | Leave a comment

Meeting notes with Prof. Mellor-Crummey on Feb 17th

This is a summary for a meeting with Prof. Mellor-Crummey on the latest progress, TODO tasks and feedback for the HJ-Hadoop research. The meeting covered three major parts Recap on the two approaches for the multi-core parallelization for Hadoop, ParMapper … Continue reading

Posted in Uncategorized | Leave a comment

Using external libraries in Hadoop

This is a post summarizing my attempt to use a third party library in modifying the source code for Hadoop.

Posted in Hadoop Research, HJ-Hadoop Improvements | Leave a comment

Compiling and Running Hadoop MapReduce using Java 8

This is a post summarizing my experience trying to compile the source code for Hadoop using Java8 and running the Hadoop system using Java 8. I am using an older version of Hadoop, hadoop-1.0.3 and developer preview JDK 8 from … Continue reading

Posted in Hadoop Research, HJ-Hadoop Improvements | Leave a comment

Meeting notes with Prof. Cox on Feb 7th

This is a summary for meeting notes with Prof. Alan Cox on Feb 7th. We discussed some more results on KMeans application, improved heap memory utilization measurement, future plans. This document summarized the important parts of the meeting. Running time … Continue reading

Posted in Habanero Java, Hadoop Research, HJ-Hadoop Improvements | Leave a comment

Java Heap Dump and Visual VM

This is a post summarizing my experiments with using heap dumps to analyze the objects in heap memory of the JVM.

Posted in Java | Leave a comment

Monitoring JVM memory usage with MX Beans

This is a post summarizing my initial research into reliable ways to measure the heap memory usage for the JVM. Previously, I have used the linux top command and look at resident memory for a rough estimate of the heap … Continue reading

Posted in Java | Leave a comment

Meeting Notes with Prof. John Mellor-Crummey

This is a summary of my meeting with Prof. John Mellor-Crummey on Monday Feb 3rd.  In the 1.5 hr meeting, I talked about the motivation, problem statement, implementation and results of my work on improving Hadoop MapReduces’ performance on multi-core … Continue reading

Posted in Hadoop Research, HJ-Hadoop Improvements | Leave a comment