LoginSignup
2
1

More than 3 years have passed since last update.

@shiro01

Java(Eclipse)でAWSBatchにジョブ送信

Last updated at Posted at 2019-12-04

Java(Eclipse)でAWSBatchにジョブ送信を試してみたので共有します。

前提

AWSBatchで以下が設定されている。
・ジョブ定義名：sample-job-definition
・キュー名：sample-job-queue
・コンピューティング環境名：sample-compute-environment

Eclipseで以下を実施している。
・AWS Toolkit for Eclipseをインストール。
・AWS Toolkit for EclipseにIAMユーザのアクセスキー、シークレットアクセスキーを設定。
・Gradleプロジェクト作成。

build.gradleの編集、SDKをインストール

dependenciesにaws-java-sdk-batchを追加して、プロジェクトにインストールします。
(プロジェクトを右クリックして Gradleプロジェクトのリフレッシュなどすると処理が開始されます)

dependenciesに追加
dependencies {
    …略
    compile('com.amazonaws:aws-java-sdk-batch:1.11.683')
    // compile group: 'com.amazonaws', name: 'aws-java-sdk-batch', version: '1.11.683'    // この書き方でもOK
    …略
}

Javaサンプルコード

ドキュメントを参考に、以下のお試しコードを作成しました。

サンプルコード
package sample;

import java.util.HashMap;
import java.util.Map;

import com.amazonaws.regions.Regions;
import com.amazonaws.services.batch.AWSBatch;
import com.amazonaws.services.batch.AWSBatchClientBuilder;
import com.amazonaws.services.batch.model.SubmitJobRequest;
import com.amazonaws.services.batch.model.SubmitJobResult;

public class AwsBatchSample {

    public static void main(String[] args) {
        AwsBatchSample awsBatchSample = new AwsBatchSample();
        ab.awsBatchJobSend();
    }

    private void awsBatchJobSend() {
        AWSBatch client = null;
        try {
            client = AWSBatchClientBuilder.standard().withRegion(Regions.AP_NORTHEAST_1).build();
            Map<String,String> parameters = new HashMap<String, String>();
            parameters.put("param1", "JobParam1");
            // ジョブ定義でのコマンド：echo sample-batch Ref::param1
            // ログでの出力結果　　　：sample-batch JobParam1
            SubmitJobRequest request = new SubmitJobRequest()
                                            .withJobName("sample-job-submit")
                                            .withJobQueue("sample-job-queue")
                                            .withJobDefinition("sample-job-definition")
                                            .withParameters(parameters)
                                            ;
            SubmitJobResult response = client.submitJob(request);
            System.out.println(response);
        } catch (Exception e) {
            System.out.println(e);
        } finally {
            if (client != null) {
                client.shutdown();
            }
        }
    }
}
ジョブ送信が成功すると以下が返却される。
{JobName: sample-job-submit,JobId: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa}

はまりポイント

ドキュメントの例では指定されていませんでしたが、
.withRegion(Regions.AP_NORTHEAST_1)を指定しない場合、以下のようなエラーが出力されました。

エラーログ
20:31:11.636 [main] DEBUG com.amazonaws.AmazonWebServiceClient - Internal logging successfully configured to commons logger: true
20:31:11.998 [main] DEBUG com.amazonaws.monitoring.CsmConfigurationProviderChain - Unable to load configuration from com.amazonaws.monitoring.EnvironmentVariableCsmConfigurationProvider@17fc391b: Unable to load Client Side Monitoring configurations from environment variables!
20:31:11.998 [main] DEBUG com.amazonaws.monitoring.CsmConfigurationProviderChain - Unable to load configuration from com.amazonaws.monitoring.SystemPropertyCsmConfigurationProvider@2b30a42c: Unable to load Client Side Monitoring configurations from system properties variables!
20:31:11.998 [java-sdk-http-connection-reaper] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Closing connections idle longer than 60000 MILLISECONDS
20:31:11.999 [main] DEBUG com.amazonaws.monitoring.CsmConfigurationProviderChain - Unable to load configuration from com.amazonaws.monitoring.ProfileCsmConfigurationProvider@43df23d3: Unable to load config file
20:31:12.025 [main] WARN com.amazonaws.internal.InstanceMetadataServiceResourceFetcher - Fail to retrieve token 
com.amazonaws.SdkClientException: Failed to connect to service endpoint: 
    at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)
    at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)
    at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)
    at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:62)
    at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:400)
    at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:369)
    at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:365)
    at com.amazonaws.util.EC2MetadataUtils.getEC2InstanceRegion(EC2MetadataUtils.java:280)
    at com.amazonaws.regions.InstanceMetadataRegionProvider.tryDetectRegion(InstanceMetadataRegionProvider.java:59)
    at com.amazonaws.regions.InstanceMetadataRegionProvider.getRegion(InstanceMetadataRegionProvider.java:50)
    at com.amazonaws.regions.AwsRegionProviderChain.getRegion(AwsRegionProviderChain.java:46)
    at com.amazonaws.client.builder.AwsClientBuilder.determineRegionFromRegionProvider(AwsClientBuilder.java:475)
    at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:458)
    at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
    at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
    at sample.AwsBatchSample.awsBatchJobSend(AwsBatchSample.java:17)
    at sample.AwsBatchSample.main(AwsBatchSample.java:12)
Caused by: java.net.SocketException: Network is unreachable: connect
    at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
    at sun.net.www.http.HttpClient.New(HttpClient.java:339)
    at sun.net.www.http.HttpClient.New(HttpClient.java:357)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1181)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966)
    at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:52)
    at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80)
    ... 16 common frames omitted
20:31:12.025 [main] WARN com.amazonaws.util.EC2MetadataUtils - Unable to retrieve the requested metadata (/latest/dynamic/instance-identity/document). Failed to connect to service endpoint: 
com.amazonaws.SdkClientException: Failed to connect to service endpoint: 
    at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)
    at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)
    at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)
    at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:62)
    at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:400)
    at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:369)
    at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:365)
    at com.amazonaws.util.EC2MetadataUtils.getEC2InstanceRegion(EC2MetadataUtils.java:280)
    at com.amazonaws.regions.InstanceMetadataRegionProvider.tryDetectRegion(InstanceMetadataRegionProvider.java:59)
    at com.amazonaws.regions.InstanceMetadataRegionProvider.getRegion(InstanceMetadataRegionProvider.java:50)
    at com.amazonaws.regions.AwsRegionProviderChain.getRegion(AwsRegionProviderChain.java:46)
    at com.amazonaws.client.builder.AwsClientBuilder.determineRegionFromRegionProvider(AwsClientBuilder.java:475)
    at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:458)
    at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
    at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
    at sample.AwsBatchSample.awsBatchJobSend(AwsBatchSample.java:17)
    at sample.AwsBatchSample.main(AwsBatchSample.java:12)
Caused by: java.net.SocketException: Network is unreachable: connect
    at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
    at sun.net.www.http.HttpClient.New(HttpClient.java:339)
    at sun.net.www.http.HttpClient.New(HttpClient.java:357)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1181)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966)
    at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:52)
    at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80)
    ... 16 common frames omitted
com.amazonaws.SdkClientException: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region.

ざっくりですが、今回は以上です。

参考

https://docs.amazonaws.cn/AWSJavaSDK/latest/javadoc/com/amazonaws/services/batch/AWSBatchClient.html#submitJob-com.amazonaws.services.batch.model.SubmitJobRequest-
https://docs.amazonaws.cn/AWSJavaSDK/latest/javadoc/com/amazonaws/services/batch/model/SubmitJobRequest.html
https://github.com/aws/aws-sdk-java/issues/2062
https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-batch

2
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
2
1