Batch sdk integration #2865
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| *.class | ||
|
|
||
| # Auth files | ||
| *.auth | ||
| *.azureauth | ||
|
|
||
| # Local checkstyle | ||
| *.checkstyle | ||
|
|
||
| # Mobile Tools for Java (J2ME) | ||
| .mtj.tmp/ | ||
|
|
||
| # Package Files # | ||
| *.jar | ||
| *.war | ||
| *.ear | ||
|
|
||
| # Azure Tooling # | ||
| node_modules | ||
| packages | ||
|
|
||
| # Eclipse # | ||
| *.pydevproject | ||
| .project | ||
| .metadata | ||
| bin/** | ||
| tmp/** | ||
| tmp/**/* | ||
| *.tmp | ||
| *.bak | ||
| *.swp | ||
| *~.nib | ||
| local.properties | ||
| .classpath | ||
| .settings/ | ||
| .loadpath | ||
|
|
||
| # Other Tooling # | ||
| .classpath | ||
| .project | ||
| target | ||
| .idea | ||
| *.iml | ||
|
|
||
| # Mac OS # | ||
| .DS_Store | ||
| .DS_Store? | ||
|
|
||
| # Windows # | ||
| Thumbs.db | ||
|
|
||
| # reduced pom files should not be included | ||
| dependency-reduced-pom.xml | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| language: java | ||
|
Member
There was a problem hiding this comment. Why do we need this travis.yml file?
Member
There was a problem hiding this comment. I noted this also. My thinking was that we would clean up docs, remove redundant files, etc after we get the repo imported.
Member
There was a problem hiding this comment. I'm fine with follow-up clean-up.
g2vinay marked this conversation as resolved.
Outdated
|
||
| sudo: false | ||
| jdk: | ||
| - oraclejdk8 | ||
| - openjdk7 | ||
| script: | ||
| - mvn install -DskipTests=true | ||
| - mvn package javadoc:aggregate -DskipTests=true | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| The MIT License (MIT) | ||
|
g2vinay marked this conversation as resolved.
Outdated
|
||
|
|
||
| Copyright (c) 2015 Microsoft | ||
|
|
||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
|
|
||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| [](https://travis-ci.org/Azure/azure-batch-sdk-for-java) | ||
|
|
||
| ## Azure Batch Libraries for Java | ||
|
|
||
| This README is based on the latest released version Azure Batch SDK (5.0.1). If you are looking for other releases, see the [More Information](#more-information) section below. | ||
|
|
||
| The Azure Batch Libraries for Java is a higher-level, object-oriented API for interacting with the Azure Batch service. | ||
|
|
||
|
|
||
| > **5.0.1** is a release that supports all features of Azure Batch service with API version "2018-12-01.8.1". We will be adding support for more new features and tweaking the API associated with Azure Batch service newer release. | ||
|
|
||
| **Azure Batch Authentication** | ||
|
|
||
| You need to create a Batch account through the [Azure portal](https://portal.azure.com) or Azure cli. | ||
|
|
||
| * Use the account name, key, and URL to create a `BatchSharedKeyCredentials` instance for authentication with the Azure Batch service. | ||
| The `BatchClient` class is the simplest entry point for creating and interacting with Azure Batch objects. | ||
|
|
||
| ```java | ||
| BatchSharedKeyCredentials cred = new BatchSharedKeyCredentials(batchUri, batchAccount, batchKey); | ||
| BatchClient client = BatchClient.open(cred); | ||
| ``` | ||
|
|
||
| * The other way is using AAD (Azure Active Directory) authentication to create the client. See this [document](https://docs.microsoft.com/en-us/azure/batch/batch-aad-auth) for detail. | ||
|
|
||
| ```java | ||
| BatchApplicationTokenCredentials cred = new BatchApplicationTokenCredentials(batchEndpoint, clientId, applicationSecret, applicationDomain, null, null); | ||
| BatchClient client = BatchClient.open(cred); | ||
| ``` | ||
|
|
||
| **Create a pool using an Azure Marketplace image** | ||
|
|
||
| You can create a pool of Azure virtual machines which can be used to execute tasks. | ||
|
|
||
| ```java | ||
| System.out.println("Created a pool using an Azure Marketplace image."); | ||
|
|
||
| VirtualMachineConfiguration configuration = new VirtualMachineConfiguration(); | ||
| configuration.withNodeAgentSKUId(skuId).withImageReference(imageRef); | ||
| client.poolOperations().createPool(poolId, poolVMSize, configuration, poolVMCount); | ||
|
|
||
| System.out.println("Created a Pool: " + poolId); | ||
| ``` | ||
|
|
||
| **Create a Job** | ||
|
|
||
| You can create a job by using the recently created pool. | ||
|
|
||
| ```java | ||
| PoolInformation poolInfo = new PoolInformation(); | ||
| poolInfo.withPoolId(poolId); | ||
| client.jobOperations().createJob(jobId, poolInfo); | ||
| ``` | ||
|
|
||
| # Sample Code | ||
|
|
||
| You can find sample code that illustrates Batch usage scenarios in https://github.com/azure/azure-batch-samples | ||
|
|
||
|
|
||
| # Download | ||
|
|
||
|
|
||
| **5.0.1** | ||
|
|
||
| If you are using released builds from 5.0.1, add the following to your POM file: | ||
|
|
||
| ```xml | ||
| <dependency> | ||
| <groupId>com.microsoft.azure</groupId> | ||
| <artifactId>azure-batch</artifactId> | ||
| <version>5.0.1</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.microsoft.azure</groupId> | ||
| <artifactId>azure-client-runtime</artifactId> | ||
| <version>1.6.4</version> | ||
| </dependency> | ||
| ``` | ||
|
|
||
| # Pre-requisites | ||
|
|
||
| - A Java Developer Kit (JDK), v 1.7 or later | ||
| - Maven | ||
| - Azure Service Principal - see [how to create authentication info](./AUTH.md). | ||
|
|
||
|
|
||
| ## Help | ||
|
|
||
| If you encounter any bugs with these libraries, please file issues via [Issues](https://github.com/Azure/azure-batch-sdk-for-java/issues) or checkout [StackOverflow for Azure Java SDK](http://stackoverflow.com/questions/tagged/azure-java-sdk). | ||
|
|
||
| # Contribute Code | ||
|
|
||
| If you would like to become an active contributor to this project please follow the instructions provided in [Microsoft Azure Projects Contribution Guidelines](http://azure.github.io/guidelines.html). | ||
|
|
||
| 1. Fork it | ||
| 2. Create your feature branch (`git checkout -b my-new-feature`) | ||
| 3. Commit your changes (`git commit -am 'Add some feature'`) | ||
| 4. Push to the branch (`git push origin my-new-feature`) | ||
| 5. Create new Pull Request | ||
|
|
||
| # Build Code | ||
| To build the code open a console, navigate to the git repository, and run | ||
| ``` | ||
| maven build | ||
| ``` | ||
|
|
||
| # Test Code | ||
| To run tests, set the following environment variables: | ||
| * AZURE_BATCH_ENDPOINT | ||
| * CLIENT_ID | ||
| * APPLICATION_SECRET | ||
| * AZURE_BATCH_ACCOUNT | ||
| * AZURE_BATCH_ACCESS_KEY | ||
| * STORAGE_ACCOUNT_NAME | ||
| * STORAGE_ACCOUNT_KEY | ||
| Then run any test in src/test/java directory. | ||
|
|
||
| # More Information | ||
| * [Javadoc](http://azure.github.io/azure-sdk-for-java) | ||
| * [http://azure.com/java](http://azure.com/java) | ||
| * If you don't have a Microsoft Azure subscription you can get a FREE trial account [here](http://go.microsoft.com/fwlink/?LinkId=330212) | ||
|
|
||
| **Previous Releases and Corresponding Repo Branches** | ||
|
|
||
| | Version | SHA1 | Remarks | | ||
| |-------------------|-------------------------------------------------------------------------------------------|-------------------------------------------------------| | ||
| | 5.0.1 | [5.0.1](https://github.com/Azure/azure-batch-sdk-for-java/tree/master) | Tagged release for 5.0.1 version of Azure Batch libraries | | ||
| | 5.0.0 | [5.0.0](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v5.0.0) | Tagged release for 5.0.0 version of Azure Batch libraries | | ||
| | 4.0.1 | [4.0.1](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v4.0.1) | Tagged release for 4.0.1 version of Azure Batch libraries | | ||
| | 4.0.0 | [4.0.0](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v4.0.0) | Tagged release for 4.0.0 version of Azure Batch libraries | | ||
| | 3.2.0 | [3.2.0](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v3.2.0) | Tagged release for 3.2.0 version of Azure Batch libraries | | ||
| | 3.1.0 | [3.1.0](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v3.1.0) | Tagged release for 3.1.0 version of Azure Batch libraries | | ||
| | 3.0.0 | [3.0.0](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v3.0.0) | Tagged release for 3.0.0 version of Azure Batch libraries | | ||
| | 2.1.0 | [2.1.0](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v2.1.0) | Tagged release for 2.1.0 version of Azure Batch libraries | | ||
| | 2.0.0 | [2.0.0](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v2.0.0) | Tagged release for 2.0.0 version of Azure Batch libraries | | ||
| | 1.0.0 | [1.0.0](https://github.com/Azure/azure-batch-sdk-for-java/releases/tag/v1.0.0) | Tagged release for 1.0.0 version of Azure Batch libraries | | ||
| | 1.0.0-beta2 | [1.0.0-beta2](https://github.com/Azure/azure-sdk-for-java/tree/1.0.0-beta2) | Tagged release for 1.0.0-beta2 version of Azure Batch libraries | | ||
|
|
||
| --- | ||
|
|
||
| This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| # Azure Batch SDK for Java release notes | ||
|
|
||
| ## Changes in 5.0.1 | ||
| ### Features | ||
| ### Fixes | ||
| - Fix corrupt signing of 5.0.0 Java SDK binary | ||
|
|
||
| ## Changes in 5.0.0 | ||
| ### Features | ||
| - **[Breaking]** Removed support for the `ChangeOSVersion` API on `CloudServiceConfiguration` pools. | ||
| - Removed `PoolOperations.ChangeOSVersion`. | ||
| - Renamed `TargetOSVersion` to `OSVersion` and removed `CurrentOSVersion` on `CloudPool`. | ||
| - Removed `PoolState.Upgrading` enum. | ||
| - **[Breaking]** Removed `DataEgressGiB` and `DataIngressGiB` from `PoolUsageMetrics`. These properties are no longer supported. | ||
| - **[Breaking]** ResourceFile improvements | ||
| - Added the ability specify an entire Azure Storage container in `ResourceFile`. There are now three supported modes for `ResourceFile`: | ||
| - `ResourceFile.withUrl` creates a `ResourceFile` pointing to a single HTTP URL. | ||
| - `ResourceFile.withStorageContainerUrl` creates a `ResourceFile` pointing to an Azure Blob Storage container. | ||
| - `ResourceFile.withAutoStorageContainer` creates a `ResourceFile` pointing to an Azure Blob Storage container in the Batch registered auto-storage account. | ||
| - The `BlobPrefix` property can be used to filter downloads from a storage container to only those matching the prefix. | ||
| - URLs provided to `ResourceFile` via the `ResourceFile.withUrl` method can now be any HTTP URL. Previously, these had to be an Azure Blob Storage URL. | ||
| - **[Breaking]** Removed `OSDisk` property from `VirtualMachineConfiguration`. This property is no longer supported. | ||
| - Pools which set the `DynamicVNetAssignmentScope` on `NetworkConfiguration` to be `DynamicVNetAssignmentScope.Job` can | ||
| now dynamically assign a Virtual Network to each node the job's tasks run on. The specific Virtual Network to join the nodes to is specified in | ||
| the new `JobNetworkConfiguration` property on `CloudJob` and `JobSpecification`. | ||
| - Note: This feature is in public preview. It is disabled for all Batch accounts except for those which have contacted us and requested to be in the pilot. | ||
| - The maximum lifetime of a task is now 180 days (previously it was 7). | ||
| - Added support on Windows pools for creating users with a specific login mode (either `Batch` or `Interactive`) via `WindowsUserConfiguration.LoginMode`. | ||
| - The default task retention time for all tasks is now 7 days, previously it was infinite. | ||
|
|
||
| ### REST API version | ||
| This version of the Batch .NET client library targets version 2018-12-01.8.0 of the Azure Batch REST API. | ||
|
|
||
| ## Changes in 4.0.1 | ||
| ### Fixes | ||
| - Fix missing signing of 4.0.0 Java SDK binary | ||
|
|
||
| ## Changes in 4.0.0 | ||
| ### Features | ||
| - Added the functionality to get which version of the Azure Batch Node Agent(https://github.com/Azure/Batch/blob/master/changelogs/nodeagent/CHANGELOG.md) is running on nodes via the NodeAgentInformation property of ComputeNode. | ||
| - Added additional error handling to task add operations to prevent deadlock scenarios. | ||
| - Removed validation status from count operations. | ||
| - Updated adal, azure-client-runtime, and commons-codec dependencies. | ||
| - Update 'List' methods to return the lower layer PagedList object instead of the standard List. | ||
|
|
||
| ### REST API version | ||
| This version of the Batch Java client library targets version 2018-08-01.7.1 of the Azure Batch REST API. See this [document](https://docs.microsoft.com/en-us/rest/api/batchservice/batch-service-rest-api-versioning#latest-version-2018-08-0171) for detail. | ||
|
|
||
| ## Changes in 3.3.0 | ||
| ### Features | ||
| - `createTasks` rethrow `RuntimeException` catched by internal threads. | ||
| - `createTasks` handle `RequestSizeTooLarge` chunking errors for well behaved tasks. | ||
|
|
||
| ## Changes in 3.2.0 | ||
| ### Features | ||
| - Update comments for some classes and properties. | ||
| - Added the `leavingPool` property to `NodeCounts` class. | ||
|
|
||
| ## Changes in 3.1.0 | ||
| ### Features | ||
| - Added the ability to query pool node counts by state, via the new `listPoolNodeCounts` method on `PoolOperations`. | ||
| - Added the ability to upload Azure Batch node agent logs from a particular node, via the `uploadComputeNodeBatchServiceLogs` method on `ComputeNodeOperations`. | ||
| - This is intended for use in debugging by Microsoft support when there are problems on a node. | ||
|
|
||
| ### REST API version | ||
| This version of the Batch Java client library targets version 2018-03-01.6.1 of the Azure Batch REST API. See this [document](https://docs.microsoft.com/en-us/rest/api/batchservice/batch-service-rest-api-versioning#latest-version-2018-03-0161) for detail. | ||
|
|
||
|
|
||
| ## Changes in 3.0.0 | ||
| ### Features | ||
| - Added the ability to get a discount on Windows VM pricing if you have on-premises licenses for the OS SKUs you are deploying, via `licenseType` on `VirtualMachineConfiguration`. | ||
| - Added support for attaching empty data drives to `VirtualMachineConfiguration` based pools, via the new `dataDisks` property on `VirtualMachineConfiguration`. | ||
| - **[Breaking]** Custom images must now be deployed using a reference to an ARM Image, instead of pointing to .vhd files in blobs directly. | ||
| - The new `virtualMachineImageId` property on `ImageReference` contains the reference to the ARM Image, and `OSDisk.imageUris` no longer exists. | ||
| - Because of this, `imageReference` is now a required property of `VirtualMachineConfiguration`. | ||
| - **[Breaking]** Multi-instance tasks (created using `MultiInstanceSettings`) must now specify a `coordinationCommandLine`, and `numberOfInstances` is now optional and defaults to 1. | ||
| - Added support for tasks run using Docker containers. To run a task using a Docker container you must specify a `containerConfiguration` on the `VirtualMachineConfiguration` for a pool, and then add `taskContainerSettings` on the Task. | ||
|
|
||
| ### REST API version | ||
| This version of the Batch Java client library targets version 2017-09-01.6.0 of the Azure Batch REST API. See this [document](https://docs.microsoft.com/en-us/rest/api/batchservice/batch-service-rest-api-versioning#latest-version-2017-09-0160) for detail. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.