Category Archives: Hadoop Research

My master thesis on “Optimized Runtime Systems for MapReduce Applications in Multi-core Clusters”

My 83 pages master thesis done at Rice university. It details my research on improving the memory efficiency of Hadoop MapReduce runtime system on multi-core clusters. I focused on hash join, KMeand and KNN applications. My master thesis on “Optimized … Continue reading

Posted in Hadoop Research | Leave a comment

Meeting Notes with Prof. Cox and Prof. Sarkar (End of May, Starting June)

This is a post summarizing the two meetings from the end of May and the start of June. We discussed about the deadlines for the paper, ToDo Items before the deadline and my plan in the next few months.

Posted in Hadoop Research | Leave a comment

Using Top, PS and Awk for benchmarking CPU and memory utilization in a cluster environment

This is a post on my experience building scripts that benchmark the CPU and memory utilization of Hadoop processes in a cluster using Top and Awk Linux commands.

Posted in Habanero Java, Hadoop Research, Tools | Leave a comment

March 30th, Week Meeting Notes

This is a summary of this week’s meetings, outlining a few to do items and potential issues.

Posted in Hadoop Research, HJ-Hadoop Improvements | Leave a comment

How does Kill a task work in Hadoop

This is a post summarizing how Hadoop kills a task when something went wrong with the task. (Ran for too long, etc). It is important for me because I have multiple virtual JVMs (each hoisting a map task) running in … Continue reading

Posted in Hadoop Research, HJ-Hadoop Improvements | Leave a comment

Further changes to make Parallel Jvm in Hadoop work

This is a post looking into possible changes to fix an exception that I am seeing in running parallel JVM in hadoop. The issue is that some of the map attempts crashed and the reduce tasks are not able to … Continue reading

Posted in Habanero Java, Hadoop Research, Uncategorized | Leave a comment

Using external libraries in Hadoop

This is a post summarizing my attempt to use a third party library in modifying the source code for Hadoop.

Posted in Hadoop Research, HJ-Hadoop Improvements | Leave a comment