c. Examine the File System

In this section, you learn about the file system, it’s structure and content.

First, use the following command to list all the files present.

time lfs find /lustre

You should see something similar as in the image below.

lfs find

Now, look at the size of your files by listing the content of the /lustre directory.

ls -lh /lustre

You should see something like the following image:

Lustre ls -lh

Next, use the following command to look at how much data is stored on the Lustre partition for the Metadata Target (MDT) and Object Storage Targets (OSTs).

time lfs df -h

The result should be comparable to the one shown below.

Lustre ls -dh

Notice that there is a discrepancy between

  • The size of the files ~ 468 MB
  • The data actually stored on the Lustre partition: ~ 4.5 MB of content and 5.4 MB of metadata.

This discrepancy is due to the lazy loading functionality of Amazon FSx for Lustre when it is linked to an Amazon S3 bucket. Only the metadata of the objects is retrieved on the MDT, the actual bytes or content of the files is copied at first read.