[hive][tez] tez 처리중 tez.lib.uris is not defined in the configuration 오류 수정

2017-02-06 02:11:55,396 ERROR [Thread-117] org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread Thread[Thread-117,5,main] threw an Exception.

java.lang.RuntimeException: org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined in the configuration

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:604)

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513)

at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78)

Caused by: org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined in the configuration

at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:173)

tez 처리중 위와 같은 오류가 발생하면 설정파일에 tez.lib.uris 를 설정해주면 된다. 

개별 파일을 저정해도 되고, 폴더를 지정해도 된다. 

    <property>

        <name>tez.lib.uris</name>

        <value>/경로/</value>

    </property>

AWS의 EMR이나 같은 클러스터를 이용할 경우 tez.lib.uris 를 설정해도 tez 실행시점에 

다음의 오류로 인하여 작업이 종료되는 경우가 있다. 

이때는 tez.use.cluster.hadoop-libs 를 설정해 주면 된다. 

Stack trace: ExitCodeException exitCode=1: 

at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)

at org.apache.hadoop.util.Shell.run(Shell.java:479)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)

at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

   <property>

        <name>tez.use.cluster.hadoop-libs</name>

        <value>true</value>

    </property>

테즈 설치 튜토리얼 - https://tez.apache.org/install.html

테스 설정값 - https://tez.apache.org/releases/0.8.2/tez-api-javadocs/configs/TezConfiguration.html