WrodCount MapReduce with Scalding

git clone https://github.com/scalding-io/ProgrammingWithScalding

cd ProgrammingWithScalding/chapter2/

mvn clean install

hadoop fs -mkdir -p /data/input

hadoop fs -mkdir -p /data/output

echo “This is a happy day. A day to remember” > /tmp/input.txt

hadoop fs -put /tmp/input.txt /data/input

hadoop jar /root/repo/ProgrammingWithScalding/chapter2/target/chapter2-0-jar-with-dependencies.jar com.twitter.scalding.Tool WordCountJob –local –input /data/input/input.txt –output /data/output/output.txt

cat /data/output/output.txt
a 2
day 1
day. 1
happy 1
is 1
remember 1
this 1
to 1

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s