System Specifications

Compute Nodes

Type	# Nodes	Per Node				PBS Queue
Type	# Nodes	CPU	GPU	RAM	Local Storage	PBS Queue
Cray EX CPU Nodes	768	2x AMD EPYC™ 7713 64 cores per CPU 128 cores in total	-	512GB DDR4 ECC	-	`normal`
Large Memory Nodes	12	2x AMD EPYC™ 7713 64 cores per CPU 128 cores in total	-	2TB	-	`normal`
Large Memory Nodes	4	2x AMD EPYC™ 7713 64 cores per CPU 128 cores in total	-	4TB	-	`normal`
Cray EX GPU Nodes	64	1x AMD EPYC™ 7713 64 cores per CPU 64 cores in total	4x NVIDIA A100 40GB SXM	512GB DDR4 ECC	-	`normal`
Apollo AI GPU Nodes	12	1x AMD EPYC™ 7713 64 cores per CPU 64 cores in total	4x NVIDIA A100 40GB SXM	512GB	12TB NVMe	`ai`
Apollo AI GPU Nodes	6	2x AMD EPYC™ 7713 64 cores per CPU 128 cores in total	8x NVIDIA A100 40GB SXM	1TB	14TB NVMe	`ai`

Network

HPE Slingshot-10
Dragonfly topology
1x 100 Gbps for CPU nodes
2x 100 Gbps for GPU nodes

Storage

Warning

All data on NSCC systems is governed by the Data Management and Retention Policy. Read it carefully to prevent data loss, maintain proper access, and manage your data securely.

ASPIRE 2A features multiple types of storage for different use cases.

Directory	Filesystem	Capacity	Mount Point	Quota	Use Case
Home	GPFS	25PB	`/home/users/<user-org>/<user-id>` `$HOME`	50GB per user	Long term storage of user data
Project	GPFS	25PB	`/home/project/<project-id>`	Per project allocation	Long term storage of project data shared among members
Scratch	Lustre	10PB	`/scratch/users/<user-org>/<user-id>` `$HOME/scratch`	100TB per user	Temporary storage of user data for better I/O performance (subject to 30-day purge)
Node Local NVMe	XFS	12TB / 14TB per AI GPU node	`/raid/pbs.$PBS_JOBID` `$TMPDIR`	-	Temporary storage of compute job data for better I/O performance (automatically removed at job end)

Data Management Framework

ASPIRE 2A uses multiple storage tiers to balance performance and capacity. Active ("hot") data in Home and Project directories resides on fast NVMe flash and HDD within the GPFS filesystem, while less frequently accessed ("cold") data is migrated to slower, higher-capacity tiers such as tape storage. The HPE Data Management Framework (DMF) automatically manages these migrations to optimize storage usage and system performance.

If access to older files in your Home or Project directories is slower, it is likely because DMF has moved them to a colder tier. Retrieving the data back to the faster tier on demand may take some time.