Neptune ML

Terraform module which attempts to be analogous to and creates almost all the same resources as AWS's CloudFormation template for Neptune ML. It strives to be more customizable than the CloudFormation template.

Rather than creating a separate VPC, like in the CloudFormation template, this module will set up everything in an existing VPC. You are therefore required to provide a VPC ID as an input argument when using this module.

Resources in IAM policy documents are, where possible, stricter than their CloudFormation counterparts, and all resources use at-rest encryption by default.

Contributions are welcome.

Requirements

Name	Version
terraform	>= 1.0
aws	>= 5.0
random	>= 2.3.1

Providers

Name	Version
aws	5.26.0
random	3.5.1

Modules

Name	Source	Version
api_gateway	./modules/service-role	n/a
batch_execution	./modules/service-role	n/a
batch_job	./modules/service-role	n/a
ec2	./modules/service-role	n/a
lambda_execution	./modules/service-role	n/a
neptune_export_gateway	./modules/gateway-lambda-integration	n/a
neptune_export_gateway_proxy	./modules/gateway-lambda-integration	n/a
neptune_export_lambda	./modules/lambda	n/a
neptune_export_status_lambda	./modules/lambda	n/a
neptune_ml_iam	./modules/service-role	n/a
neptune_user	./modules/service-role	n/a
s3	./modules/service-role	n/a
sagemaker_execution	./modules/service-role	n/a

Resources

Name	Type
aws_api_gateway_account.neptune_export	resource
aws_api_gateway_deployment.neptune_export	resource
aws_api_gateway_rest_api.neptune_export	resource
aws_api_gateway_rest_api_policy.neptune_export	resource
aws_api_gateway_stage.neptune_export	resource
aws_batch_compute_environment.neptune	resource
aws_batch_job_definition.neptune	resource
aws_batch_job_queue.neptune	resource
aws_cloudwatch_log_group.api_gateway	resource
aws_cloudwatch_log_group.batch	resource
aws_cloudwatch_log_group.neptune	resource
aws_cloudwatch_log_group.sagemaker_notebook	resource
aws_cloudwatch_log_group.sagemaker_processing	resource
aws_iam_access_key.neptune_user	resource
aws_iam_user.neptune_user	resource
aws_iam_user_policy.neptune_user	resource
aws_kms_alias.neptune	resource
aws_kms_key.neptune	resource
aws_launch_template.batch	resource
aws_neptune_cluster.neptune	resource
aws_neptune_cluster_instance.neptune	resource
aws_neptune_cluster_parameter_group.neptune	resource
aws_neptune_parameter_group.neptune	resource
aws_neptune_subnet_group.neptune	resource
aws_s3_bucket.neptune	resource
aws_s3_bucket_policy.neptune	resource
aws_s3_bucket_public_access_block.neptune	resource
aws_s3_bucket_server_side_encryption_configuration.neptune	resource
aws_sagemaker_notebook_instance.neptune	resource
aws_sagemaker_notebook_instance_lifecycle_configuration.neptune	resource
aws_security_group.batch	resource
aws_security_group.neptune	resource
aws_security_group.neptune_export	resource
aws_vpc_endpoint.api_gateway	resource
aws_vpc_endpoint.s3	resource
aws_vpc_endpoint.sagemaker_api	resource
aws_vpc_endpoint.sagemaker_runtime	resource
random_id.this	resource
aws_caller_identity.this	data source
aws_iam_policy_document.api_gateway	data source
aws_iam_policy_document.neptune_kms	data source
aws_iam_policy_document.neptune_s3	data source
aws_iam_policy_document.neptune_user	data source
aws_iam_policy_document.vpce_s3	data source
aws_iam_role.admin	data source
aws_iam_user.admin	data source
aws_region.this	data source
aws_subnet.extra	data source

Inputs

Name	Description	Type	Default	Required
batch_compute_instance_types	List of instance types to use for Batch compute environments.	list(string)	[ "c5" ]	no
cluster_instance_count	Number of instances to run in Neptune cluster.	number	1	no
cluster_parameter_group	Configuration parameters for Neptune cluster as a map of string to any.	map(any)	{ "neptune_autoscaling_config": "{}", "neptune_enable_audit_log": 1, "neptune_lab_mode": "NeptuneML=enabled", "neptune_lookup_cache": 1, "neptune_query_timeout": 120000, "neptune_streams": 0 }	no
create_iam_user	Whether or not to create an IAM user with assumable role to access Neptune ML resources. If true, variable 'neptune_iam_authentication' should also be set to true, and variable 'pgp_key' should also be provided, otherwise the user's secret key will be stored in plain text in the Terraform state file.	bool	false	no
database_instance_type	Neptune DB instance type.	string	"db.t3.medium"	no
extra_subnet_ids	IDs of other subnets where requests to Neptune may originate from.	list(string)	[]	no
instance_parameter_group	Configuration parameters for Neptune instances as a map of string to any.	map(any)	{ "neptune_dfe_query_engine": "viaQueryHint", "neptune_query_timeout": 120000, "neptune_result_cache": 0 }	no
kms_admin_role_names	Names of additional IAM roles that will be administrators of the KMS key. Not required, but strongly recommended. Root is always added.	list(string)	[]	no
kms_admin_user_names	Names of additional IAM users that will be administrators of the KMS key. Not required, but strongly recommended. Root is always added.	list(string)	[]	no
neptune_allow_major_version_upgrade	Whether or not to allow automatic major version upgrades of Neptune cluster.	bool	true	no
neptune_apply_immediately	Whether or not to apply changes to Neptune immediately or wait for maintenance window.	bool	false	no
neptune_backup_window	Window of time to run backups of Neptune cluster.	string	"07:00-09:00"	no
neptune_engine_version	The version of the Neptune engine to run the cluster.	string	"1.2.1.0"	no
neptune_iam_authentication	Whether or not enable IAM authentication for the Neptune cluster.	bool	false	no
neptune_maintenance_window	Window of time to run maintenance of Neptune cluster.	string	"sat:22:00-sun:04:00"	no
neptune_port	The port which Neptune will expose.	number	8182	no
neptune_skip_final_snapshot	Whether or not to skip taking a snapshot of the database before deletion.	bool	false	no
neptune_subnet_ids	IDs of VPC subnets to set up Neptune ML resources in.	list(string)	n/a	yes
pgp_key	For IAM user secret key. A base-64 encoded PGP public key, or a keybase username in the form keybase:some_person_that_exists. If PGP key, provide "unarmored" version (e.g. avoid passing the -a option to gpg --export).	string	null	no
resource_group_name	A name to add as prefix or suffix created resources. For example environment, like test or prod, or company name.	string	""	no
route_table_ids	IDs of route tables to add to S3 VPC Endpoint.	list(string)	[]	no
sagemaker_notebook_instance_type	The instance type of the SageMaker notebook.	string	"ml.t3.medium"	no
sagemaker_notebook_platform_id	The platform ID of the SageMaker notebook.	string	"notebook-al2-v1"	no
serverless_max_capacity	Maximum Neptune Capacity Units (NCUs) for serverless scaling configuration.	number	0	no
serverless_min_capacity	Minimum Neptune Capacity Units (NCUs) for serverless scaling configuration.	number	0	no
tags	Tags to add to resources.	map(string)	{}	no
vpc_endpoint_security_group_ids	IDs of security groups allowed to access VPC endpoints.	list(string)	[]	no
vpc_id	The ID of the VPC to set up Neptune in.	string	n/a	yes

Outputs

Name	Description
kms_key_arn	ARN of KMS key used for encrypting data created by Neptune ML resources
neptune_cluster_arn	ARN of the Neptune cluster
neptune_cluster_endpoint	URL of the Neptune cluster
neptune_cluster_id	ID of the Neptune cluster
neptune_cluster_reader_endpoint	URL of read-only endpoint of the Neptune cluster
neptune_cluster_resource_id	Resource ID of the Neptune cluster
neptune_cluster_subnet_group_id	ID of subnet group for Neptune cluster
neptune_ec2_client_role_arn	ARN of IAM role with AWS managed permission 'AmazonEC2ContainerServiceforEC2Role' attached
neptune_ec2_instance_profile_arn	ARN of instance profile for EC2 client role
neptune_export_api_uri	URL of API Gateway for Neptune exports
neptune_export_security_group_id	ID of security group for Neptune export resources
neptune_export_start_command	Template of CLI command start Neptune exports via AWS Lambda
neptune_export_status_command	Template of CLI command to check status of Neptune exports via AWS Lambda
neptune_iam_auth_role_arn	ARN of IAM role for Neptune IAM auth user
neptune_iam_auth_user_access_key_id	Access key ID of Neptune IAM auth user
neptune_iam_auth_user_arn	ARN of Neptune IAM auth user
neptune_iam_auth_user_secret_access_key	Secret access key of Neptune IAM auth user
neptune_load_from_s3_iam_role_arn	ARN of IAM role permitting Neptune to load files from S3
neptune_ml_iam_role_arn	ARN of IAM role permitting Neptune to create resources for SageMaker
neptune_security_group_id	ID of security group for Neptune ML resources
s3_bucket_name	Name of S3 bucket for loading data into Neptune
sagemaker_notebook_lifecycle_configuration_id	ID of lifecycle configuration used by the SageMaker notebook
sagemaker_notebook_name	Name of the SageMaker notebook