Tuesday, 3 March 2015

How to activate short-circuit local read on Hdfs & HBase


Editing....
Below are the minimum configs required :

Add below property to hdfs-site.xml

create dir /var/lib/hadoop-hdfs/ on each of the datanode.

<configuration>
  <property>
    <name>dfs.client.read.shortcircuit</name>
    <value>true</value>
  </property>
  <property>
    <name>dfs.domain.socket.path</name>
    <value>/var/lib/hadoop-hdfs/dn_socket</value>
  </property>
</configuration>

You would see dn_socket getting replaced with dn_1090 (if on secure cluster)


To enable on HBase:

just edit hbase-site.xml 

<configuration>
  <property>
    <name>dfs.client.read.shortcircuit</name>
    <value>true</value>
  </property>


No comments:

Post a Comment