c. Create a Cluster Config

Now that you installed AWS ParallelCluster and created a default configuration, you can create a configuration file to build a simple HPC system. This file is generated in your home directory.

Generate the cluster with the following settings:

  • Head-node and compute nodes: c5.xlarge instances. You can change the instance type if you like, but you may run into EC2 limits that may prevent you from creating instances or create too many instances.
  • In ParallelCluster 2.9 or above, we will support multiple instance types and multiple queues, but in this lab, we will only create one instance type and one queue.
  • We use a placement group in this lab. A placement group will sping up instances close together inside one physical data center in a single Availability Zone to maximize the bandwidth and reduce the latency between instances.
  • In this lab, the cluster has 0 compute nodes when starting and maximum size set to 8 instances. AWS ParallelCluster will grow and shrink between the min and max limits based on the cluster utilization and job queue backlog.
  • A GP2 Amazon EBS volume will be attached to the head-node then shared through NFS to be mounted by the compute nodes on /shared. It is generally a good location to store applications or scripts. Keep in mind that the /home directory is shared on NFS as well.
  • SLURM will be used as a job scheduler
  • We disable Intel Hyper-threading by setting disable_hyperthreading = true in the configuration file.

For more details about the AWS ParallelCluster configuration options, see the AWS ParallelCluster User Guide.

For now, paste the following commands in your terminal:

cat > my-cluster-config.ini << EOF
[aws]
aws_region_name = ${REGION}

[global]
cluster_template = default
update_check = false
sanity_check = true

[vpc public]
vpc_id = ${VPC_ID}
master_subnet_id = ${SUBNET_ID}

[cluster default]
key_name = pc-intro-key
base_os = alinux2
scheduler = slurm
master_instance_type = c5.xlarge
s3_read_write_resource = *
vpc_settings = public
ebs_settings = myebs
queue_settings = compute

[queue compute]
compute_resource_settings = default
disable_hyperthreading = true
placement_group = DYNAMIC

[compute_resource default]
instance_type = c5.large
min_count = 0
max_count = 8

[ebs myebs]
shared_dir = /shared
volume_type = gp2
volume_size = 20

[aliases]
ssh = ssh {CFN_USER}@{MASTER_IP} {ARGS}
EOF

Now, you are ready to launch a cluster! Proceed to the next section.