Page tree
Skip to end of metadata
Go to start of metadata

See the following commands and tutorials to learn how to test IO speed, debug and profile, do log file performance analysis, and more.

How-tos

Test IO speed

Use the following command to test IO speed.

hadoop jar /usr/lib/hadoop/hadoop-test-0.20.2+737.jar TestDFSIO -write -nrFiles 1 -fileSize 2000

Tutorials

Debugging and profiling with IsolationRunner

See: https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#IsolationRunner

Vaidya log file performance analysis

See: https://hadoop.apache.org/docs/r1.2.1/vaidya.html

Profiling

See: http://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Profiling

Debug scripts

See: http://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Debugging

Features

The distributed cache

See: http://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#DistributedCache

The HDFS rebalancing feature

See: http://hadoop.apache.org/docs/r1.2.1/hdfs_user_guide.html#Rebalancer

  • No labels