This repository provides a Maven project as a pre-assignment for Sakamoto's lectures about Hadoop (e.g. BIGCHA and TopSE). The project includes a simulated Hadoop MapReduce framework working on a pure Java environment. Note that this simulated framework has some differences from the official framework.
Please clone this repository and import it as a Maven project into Eclipse (or Intellij IDEA). You should see the detailed instructions in the Excercise[1-6]Main classes.
- Run Eclipse IDE for Java Developers
- File > Import > Maven > Existing Maven Projects
- Set your directory which is cloned from GitHub as Root Directory
- Select all projects
- Push Finish
If you face a problem caused by the inconsistency between the required JDK version and your PC's JDK version, please modify the following statements in pom.xml
.
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.source>1.8</maven.compiler.source>
- Excercise 1
Runjp.ac.nii.exercise1.Excercise1Main
class. - Excercise 2
Modifyjp.ac.nii.exercise2.Excercise2Main
class and add new classes. - Excercise 3
Modifyjp.ac.nii.exercise3.Excercise3Main
class and add new classes. - Excercise 4
Modifyjp.ac.nii.exercise4.AverageCalculationMapper/Reducer
andjp.ac.nii.exercise4.StandardDeviationCalculationMapper/Reducer
classes. - Excercise 5
Modifyjp.ac.nii.exercise5.AverageCalculationMapper/Reducer
andjp.ac.nii.exercise5.StandardDeviationCalculationMapper/Reducer
classes. - Excercise 6
Modifyjp.ac.nii.exercise6.AllPairAggregationMapper/Reducer
,jp.ac.nii.exercise6.RelativityCalculationReducer/Job
, andjp.ac.nii.exercise6.SpecPairAggregationMapper/Reducer
classes.