Using an earlier version of Hadoop (1.2.1 with example)

Start off by installing the latest version of Hadoop because you’ll need the basic infrastructure in place no matter what version you want to use. You can find my tutorial on installing Hadoop on Yosemite using Homebrew here.

  1. Find the version of Hadoop you are specifically looking for, version 1.2.1 can be found here https://archive.apache.org/dist/hadoop/core/hadoop–1.2.1/
  2. Download the tar.gz file and unzip it.
  3. Copy the contents of the folder into /usr/local/Cellar/hadoop
  4. Now just start up your local hadoop configuration, using hstart we created in the introduction tutorial.
  5. Instead of doing the default hadoop jar …  we directly execute the specific hadoop script that came with the new tar file. You can run the file either by
    1. /usr/local/hadoop/1.2.1/bin/hadoop jar ./target/wordcount.jar dataset1.txt $(date +%s)
    2.  or to make it a little easier create a shell script called “setmeup” that will execute the command
      1. #!/bin/bash
      2. #/usr/local/Cellar/hadoop/1.2.1/bin/hadoop jar ./target/bdp-1.0.jar dataSet1.txt $(date +%s)
    3. After you create the script, do a chmod +x ./setmeup to make it an executable.
    4. run ./setmeup to execute the job.
  6. Notice this will produce output files directly in the build / run directory.

Find the working example on Github

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s