Configuration Files
The OpenFAM environment uses four separate configuration files, three for the memory and metadata services, and one for the OpenFAM client.
- fam_client_interface_config.yaml: Configures the client interface service.
- fam_memoryserver_config.yaml: Configures the memory servers.
- fam_metadata_config.yaml: Configures the metadata management service.
- fam_pe_config.yaml: Configures the OpenFAM clients.
1. fam_client_interface_config.yaml
| Configuration Name | Description |
|---|---|
| memsrv_interface_type | Memory server interface type used by OpenFAM. Default value is "direct". Other option is "rpc". |
| metadata_interface_type | Metadata server interface type used by OpenFAM. Default value is "direct". Other option is "rpc". |
| rpc_interface | IP address/hostname of Client Interface Service, followed by port number. |
| memsrv_list | List of memory servers. Applicable only if memsrv_interface_type is rpc. Each list entry is given
in this format: <memsrv_id>:<host_address>:<port_no> |
| metadata_list | List of metadata servers. Applicable only if metadata_interface_type is rpc. Each list entry is given
in this format: <metadataserver_id>:<host_address>:<port_no> |
| rpc_framework_type | The control path operations can leverage thallium libraries instead of grpc. Default value is "grpc". Other option is "thallium". |
| provider | If "thallium" option is chosen for rpc_framework_type, it is essential to provide details of provider. Default value is "cxi". Other option is "sockets". |
2. fam_memoryserver_config.yaml
| Configuration Name | Description |
|---|---|
| provider | Libfabric provider to be used by OpenFAM libfabric data path operations. Default value is "sockets". Other supported options are "cxi", "psm2" and "verbs". |
| delayed_free_threads | Number of threads used for delayed free. |
| ATL_threads | Atomic library threads count. Default value is 0, which indicates ATL is disabled. |
| ATL_queue_size | ATL queue size. Number of elements in each queue. Default value is 1000. |
| ATL_data_size | ATL data size per thread(MiB). This is used to allocate buffers and other queue related data structures. Default data size is 1024. |
| fam_backup_path | Absolute path used by the memory server to store archives. Default value is config file location. |
| memory_type | Type of memory used by the memory server to host FAM. Options are "volatile" or "persistent". Default value is "volatile". |
| fam_path | Absolute path to the NVMM data files used by the memory server. Default value is an empty string. |
| rpc_interface | IP address/hostname of memory server, followed by port number. Default value is an "127.0.0.1:8789". |
| resource_release | This option is to enable or disable resource relinquishment in FAM. Default value is "enable". |
| libfabric_port | Libfabric port to be used for data path operations. |
| if_device | NIC name to be used with memory server. Default value is "eth0". |
| rpc_framework_type | The control path operations can leverage thallium libraries instead of grpc. Default value is "grpc". Other option is "thallium". |
| provider | The control path operations can leverage thallium libraries instead of grpc. If "thallium" option is chosen for rpc_framework_type, it is essential to provide provider details. Default value is "cxi". Other option is "sockets". |
3. fam_metadata_config.yaml
| Configuration Name | Description |
|---|---|
| metadata_manager | Metadata manager type used by metadata server. Default is radixtree. Currently radixtree is the only supported KVS. |
| metadata_path | Absolute path where metadata info is stored. By default, it will be placed in /dev/shm/<metadata_manager>. Example: If metadata manager is radixtree, then metadata info would be placed under /dev/shm/radixtree folder. |
| enable_region_spanning | Flag to enable or disable region spanning. By default, it is set to true. |
| region_span_size_per_memoryserver | Region size per memory server (Bytes). Used only if region spanning is enabled. Default value is 1073741824. |
| dataitem_interleave_size | Data interleaving refers to interspersing portions of the data across the memory servers in a specified order or other arrangement. dataitem_interleave_size decides the number of bytes after which data item needs to be striped so that it gets distributed across several memory servers. Default value is "1048576". |
| rpc_framework_type | The control path operations can leverage thallium libraries instead of grpc. Default value is "grpc". Other option is "thallium". |
| provider | If "thallium" option is chosen for rpc_framework_type, it is essential to provide provider details. Default value is "cxi". Other option is "sockets". |
4. fam_pe_config.yaml
| Configuration Name | Description |
|---|---|
| FamThreadModel | Fam thread model used by OpenFAM. Default value is "single". OpenFAM supports single and multi-threaded model. |
| provider | Libfabric provider to be used by OpenFAM libfabric data path operations. Default value is "sockets". Other supported options are "cxi", "psm2" and "verbs". |
| runtime | Runtime option used for OpenFAM. Default value is "PMIX". Other supported options are "PMI2" and "NONE" |
| client_interface_type | Client interface type used by OpenFAM. Default value is "direct". Other option is "rpc" |
| fam_client_interface_service_address | Address of Client Interface Service. Value is specified in this format: <ip_address>:<port_no>. Applicable only if client_interface_type is rpc. Default value is "127.0.0.1:8787" |
| rpc_framework_type | The control path operations can also leverage thallium libraries instead of grpc. Default value is "grpc". Other option is "thallium". |
| openfam_model | Openfam_model specifies if memory servers are resident in same nodes as clients (shared_memory mode) or distributed across nodes other than client. Default value is "memory_server". Other option is "shared_memory". |
| resource_release | This option is to enable or disable resource relinquishment in FAM. Default value is "enable". Other option is "disable". |
Sample configuration files are provided in the source under openfam_parent_directory/config.