Skip to content

Latest commit

 

History

History
149 lines (116 loc) · 10 KB

README.md

File metadata and controls

149 lines (116 loc) · 10 KB
Raw

Description

This module uploads PBS Pro RPM packages and, optionally, a license file to Google Cloud Storage. This enables machines in a PBS cluster to rapidly download and install PBS at boot or during the building of an image.

Example

The following code snippet uses this module to upload RPM packages to Cloud Storage and make them available as outputs for subsequent modules. It also demonstrates how to give read-only access to service accounts that will be used with PBS clusters. Explicit listing of service accounts is typically necessary if the bucket is provisioned in one project and clusters in other projects.

  - id: pbspro_setup
    source: community/modules/scripts/pbspro-preinstall
    settings:
      client_rpm:    "/path/to/pbs/packages/pbspro-client-2021.1.3.20220217134230-0.el7.x86_64.rpm"
      execution_rpm: "/path/to/pbs/packages/pbspro-execution-2021.1.3.20220217134230-0.el7.x86_64.rpm"
      server_rpm:    "/path/to/pbs/packages/pbspro-server-2021.1.3.20220217134230-0.el7.x86_64.rpm"
      bucket_viewers:
      - [email protected]
    outputs:
    - client_rpm_url
    - execution_rpm_url
    - server_rpm_url

Granting access to PBS Pro packages

This module can be used once to support many clusters by granting read-only access to the bucket to other clusters. Begin by identifying the service accounts used by cluster nodes. If you haven't actively chosen a service account, the default Compute Engine service account is being used. In this case, the service account is

[email protected]

where the X characters should be replaced by the project number of your project:

gcloud projects describe example-project

Supply all service accounts to the bucket_viewers setting as shown in the example above.

Destroying a bucket with versioning enabled

By default, object versioning is enabled on the bucket so that PBS packages can be recovered if they are overwritten or deleted. Buckets with object versioning enabled cannot be deleted without enabling a special force_destroy flag which indicates that the user is aware that they are deleting all version history of the objects.

terraform apply -var force_destroy=true
terraform destroy

Support

PBS Professional is licensed and supported by Altair. This module is maintained and supported by the HPC Toolkit team in collaboration with Altair.

License

Copyright 2022 Google LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Requirements

Name Version
terraform >= 0.13.0
google >= 3.83

Providers

Name Version
google >= 3.83

Modules

Name Source Version
pbspro_bucket terraform-google-modules/cloud-storage/google ~> 3.4

Resources

Name Type
google_storage_bucket_object.client_rpm resource
google_storage_bucket_object.devel_rpm resource
google_storage_bucket_object.execution_rpm resource
google_storage_bucket_object.license_file resource
google_storage_bucket_object.server_rpm resource

Inputs

Name Description Type Default Required
bucket_lifecycle_rules Additional lifecycle_rules for specific buckets. Map of lowercase unprefixed name => list of lifecycle rules to configure. 
list(object({
    # Object with keys:
    # - type - The type of the action of this Lifecycle Rule. Supported values: Delete and SetStorageClass.
    # - storage_class - (Required if action type is SetStorageClass) The target Storage Class of objects affected by this Lifecycle Rule.
    action = map(string)

    # Object with keys:
    # - age - (Optional) Minimum age of an object in days to satisfy this condition.
    # - created_before - (Optional) Creation date of an object in RFC 3339 (e.g. 2017-06-13) to satisfy this condition.
    # - with_state - (Optional) Match to live and/or archived objects. Supported values include: "LIVE", "ARCHIVED", "ANY".
    # - matches_storage_class - (Optional) Comma delimited string for storage class of objects to satisfy this condition. Supported values include: MULTI_REGIONAL, REGIONAL, NEARLINE, COLDLINE, STANDARD, DURABLE_REDUCED_AVAILABILITY.
    # - num_newer_versions - (Optional) Relevant only for versioned objects. The number of newer versions of an object to satisfy this condition.
    # - custom_time_before - (Optional) A date in the RFC 3339 format YYYY-MM-DD. This condition is satisfied when the customTime metadata for the object is set to an earlier date than the date used in this lifecycle condition.
    # - days_since_custom_time - (Optional) The number of days from the Custom-Time metadata attribute after which this condition becomes true.
    # - days_since_noncurrent_time - (Optional) Relevant only for versioned objects. Number of days elapsed since the noncurrent timestamp of an object.
    # - noncurrent_time_before - (Optional) Relevant only for versioned objects. The date in RFC 3339 (e.g. 2017-06-13) when the object became nonconcurrent.
    condition = map(string)
  }))
 
[
  {
    "action": {
      "type": "Delete"
    },
    "condition": {
      "age": 14,
      "num_newer_versions": 2
    }
  }
]
 no
bucket_viewers A list of additional accounts that can read packages from this bucket list(string) [] no
client_rpm Absolute path to PBS Pro Client Host RPM file string n/a yes
deployment_name HPC Toolkit deployment name. Cloud resource names will include this value. string n/a yes
devel_rpm Absolute path to PBS Pro Development RPM file string n/a yes
execution_rpm Absolute path to PBS Pro Execution Host RPM file string n/a yes
force_destroy Set to true if object versioning is enabled and you are certain that you want to destroy the bucket. bool false no
labels Labels to add to the created bucket. Key-value pairs. map(string) n/a yes
license_file Path to PBS Pro license file string null no
location Google Cloud Storage bucket location (defaults to var.region if not set) string null no
project_id Project in which Google Cloud Storage bucket will be created string n/a yes
region Default region for creating resources string n/a yes
retention_policy Google Cloud Storage retention policy (to prevent accidental deletion) any {} no
server_rpm Absolute path to PBS Pro Server Host RPM file string n/a yes
storage_class Google Cloud Storage class string "STANDARD" no
versioning Enable versioning of Google Cloud Storage objects (cannot be enabled with a retention policy) bool false no

Outputs

Name Description
bucket_name Bucket for PBS RPM packages
pbs_client_rpm_url gsutil URL of PBS client RPM package
pbs_devel_rpm_url gsutil URL of PBS development RPM package
pbs_execution_rpm_url gsutil URL of PBS execution host RPM package
pbs_license_file_url gsutil URL of PBS license file
pbs_server_rpm_url gsutil URL of PBS server host RPM package