flink를 YARN에서 동작할 때 작업이 ACCEPTED 상태로 대기하면서 다음과 같은 로그가 출력되는 경우가 있습니다.
2022-04-21 16:40:32,601 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2022-04-21 16:40:32,853 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2022-04-21 16:40:33,104 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2022-04-21 16:40:33,356 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
yarn 의 로그를 확인해 보니 다음과 같은 로그가 출력되고 있었습니다. 8030 포트는 리소스매니저 스케줄러 주소가 설정되지 않아서 오류가 발생하고 있습니다.
2022-04-21 15:47:48,785 INFO org.apache.hadoop.ipc.Client [] - Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2022-04-21 15:47:49,785 INFO org.apache.hadoop.ipc.Client [] - Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2022-04-21 15:47:50,786 INFO org.apache.hadoop.ipc.Client [] - Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
2022-04-21 15:47:51,787 INFO org.apache.hadoop.ipc.Client [] - Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)해결 방법
flink-conf.yaml 에 리소스 매니저 설정을 추가하면 됩니다.
flink.hadoop.yarn.resourcemanager.scheduler.address: test-server.com:8030반응형
