Configuration Files

The OpenFAM environment uses four separate configuration files, three for the memory and metadata services, and one for the OpenFAM client.

fam_client_interface_config.yaml: Configures the client interface service.
fam_memoryserver_config.yaml: Configures the memory servers.
fam_metadata_config.yaml: Configures the metadata management service.
fam_pe_config.yaml: Configures the OpenFAM clients.

These configuration files contain the following options.

1. fam_client_interface_config.yaml

Configuration Name	Description
memsrv_interface_type	Memory server interface type used by OpenFAM. Default value is "direct". Other option is "rpc".
metadata_interface_type	Metadata server interface type used by OpenFAM. Default value is "direct". Other option is "rpc".
memsrv_list	List of memory servers. Applicable only if memsrv_interface_type is rpc. Each list entry is given in this format: <memsrv_id>:<host_address>:<port_no>
metadata_list	List of metadata servers. Applicable only if metadata_interface_type is rpc. Each list entry is given in this format: <metadataserver_id>:<host_address>:<port_no>

2. fam_memoryserver_config.yaml

Configuration Name	Description
provider	Libfabric provider to be used by OpenFAM libfabric data path operations. Default value is "sockets". Other supported options are "psm2" and "verbs".
libfabric_port	Libfabric port to be used for data path operations.
delayed_free_threads	Number of threads used for delayed free.
ATL_threads	Atomic library threads count. Default value is 0, which indicates ATL is disabled.
ATL_queue_size	ATL queue size. Number of elements in each queue. Default value is 1000.
ATL_data_size	ATL data size per thread(MiB). This is used to allocate buffers and other queue related data structures. Default data size is 1024.
fam_path	Absolute path to the NVMM data files used by the memory server. Default value is an empty string.

3. fam_metadata_config.yaml

Configuration Name	Description
metadata_manager	Metadata manager type used by metadata server. Default is radixtree. Currently radixtree is the only supported KVS.
metadata_path	Absoulte path where metadata info is stored. By default, it will be placed in /dev/shm/<metadata_manager>. Example: If metadata manager is radixtree, then metadata info would be placed under /dev/shm/radixtree folder.
enable_region_spanning	Flag to enable or disable region spanning. By default, it is set to true.
region_span_size_per_memoryserver	Region size per memory server (Bytes). Used only if region spanning is enabled. Default value is 1073741824.

4. fam_pe_config.yaml

Configuration Name	Description
FamThreadModel	Fam thread model used by OpenFAM. Default value is "single". Currently OpenFAM supports only single-threaded model.
provider	Libfabric provider to be used by OpenFAM libfabric data path operations. Default value is "sockets". Other supported options are "psm2" and "verbs".
runtime	Runtime option used for OpenFAM. Default value is "PMIX". Other supported options are "PMI2" and "NONE"
client_interface_type	Client interface type used by OpenFAM. Default value is "direct". Other option is "rpc"
fam_client_interface_service_address	Address of Client Interface Service. Value is specified in this format: <ip_address>:<port_no>. Applicable only if client_interface_type is rpc. Default value is "127.0.0.1:8787"

Sample configuration files are provided in the source under openfam_parent_directory/config.

Starting Services

All the services and the PE read the respective configuration file from "config" directory placed at the path mentioned by OPENFAM_ROOT environment variable or /opt/OpenFAM.

Alternatively, most of these configuration options can be passed to services as command line arguments. The details of available command line options can be found using -h option of each of the service executable. For Example:

#./memory_server -h
Usage :
        ./memory_server <options>

Options :
        -a/--address        : Address of the memory server (default value is localhost)

        -r/--rpcport        : RPC port (default value is 8789)

        -l/--libfabricport  : Libfabric port (default value is 7500)

        -p/--provider       : Libfabric provider (default value is sockets)

        -f/--fam_path       : Location of FAM (default value is /dev/shm/<username>)

        -v/--version        : Display memory server version

        -i/--init           : Initialize the root shelf

The configuration options passed to the services using command line arguments take precedence over the options present in the respective configuration files.

Similarly, for the PE, configuration options passed using Fam_Options argument of fam_initialize API take precedence over the options present in fam_pe_config.yaml configuration file.