OpenFAM release notes

These notes summarize the different releases of OpenFAM. A more detailed list of list of features, enhancements and bug fixes are added to the source tree in all the released versions, and are provided in the corresponding NEWS.md file. Build and installation instructions are listed in the corresponding README.md file located in the source tree.

OpenFAM version 3.2 (release v3.2.0)

The current version of OpenFAM is 3.2. It contains the following changes and enhancements over version 3.1:

Added support for Slingshot Interconnect with cxi provider. cxi provider additionally support features that improve small I/O performance.
Following features have been incorporated in OpenFAM:
- Added support for huge pages, using which application can improve performance of random scatter/gather from multiple PEs.
- Added support for hybrid MR, using which improves the performance of multi-threaded applications.

OpenFAM version 3.1 (release v3.1.0)

OpenFAM version 3.1 contains the following changes and enhancements over version 3.0:

Added thallium as an alternate RPC framework for control path operations. Thallium can be used as RPC framework by passing --rpc_framework=thallium to openfam_adm script (or in the arg file). Steps for thallium installation can be found here.
To fall back to grpc, --rpc_framework=grpc can be used by application developers. The default option is grpc.
Improvements to memory registrations: OpenFAM 3.0 registered each data item separately with libfabric, leading to excessive memory registrations as the number of data items increases. As a result, scalability suffers because the underlying memory registration caches become inefficient. OpenFAM 3.1 introduces an alternative approach to register entire region in place of registering data items, thereby reducing the number of memory registrations needed.
Performance improvements for slingshot interconnect:
- Few performance improvements have been done while using OpenFAM with slingshot inteconnect. More details can be found here.
OpenFAM APIs changes:
- fam_create_region API has been modified. A new region specific attribute called permissionLevel has been added to a structure called Fam_Region_Attributes. See fam_create_region API for more details.
- A new API by name fam_close has been added. This API reduces the reference count of the region within which the data item represented by the descriptor is allocated. OpenFAM internally uses this reference count for optimal usage of resources. This API helps to release unused resources back to the system. See fam_close API for more details.
Known Issues:

OpenFAM version 3.0 (release v3.0.0)

OpenFAM 3.0 contains the following changes and enhancements over version 2.0:

Added support for volatile and persistent memory types.
Multi-threaded support has been added that enables OpenFAM to be used with multi-threaded applications. See multi-threaded environment for more details.
Performance improvements: Data item interleaving feature has been added in which a given data item will span across multiple memory servers. This has been done to uniformly distribute the load across multiple memory servers, thus reducing the network bottlenecks, and thereby improving the overall performance. Data item interleaving is enabled by default. See Data Item Interleaving for more details.
Added support for multiple contexts. The data path APIs (get,put, scatter, gather and atomics) can be invoked from user defined contexts. See fam_context_open API for more details.
OpenFAM APIs changes:
- fam_create_region API has been modified. Region specific attributes like redundancyLevel, etc should now be specified using a structure called Fam_Region_Attributes.
- Two new APIs have been added to support multiple contexts namely fam_context_open() and fam_context_close(). These APIs provide the flexibility to threads to invoke data path operations in parallel.
- A new API fam_progress has been provided that returns the count of pending data path and atomic operations.
- Few APIs have been added to enable backing up and restoring of data items namely - fam_backup() and fam_restore(). An API has been provided to delete the unused backup in the memory server.

OpenFAM version 2.0 (release v2.0.0)

OpenFAM 2.0 contains the following changes and enhancements over version 1.0:

Scalability improvements: This version supports a larger pool of memory by allowing multiple memory servers to be used. See services for more details.
Performance improvements: Memory management and metadata operations have been optimized.
- Background worker threads (unused in OpenFAM 1.0) were disabled from the memory service.
- Allocation performance was improved by moving the memset operation to zero allocated memory from fam_allocate() to fam_deallocate() path.
- Regions can span multiple memory servers, thus reducing network bottlenecks when multiple data items within the same region are used.
OpenFAM APIs changes:
- fam_size has been replaced with fam_stat.
- fam_copy has been changed to allow data copies between regions, as well as within a region.
- The return type was changed from int to void for the following APIs: fam_initialize, fam_resize_region, fam_change_permissions, fam_put_blocking, fam_get_blocking, fam_gather_blocking, fam_scatter_blocking.
- fam_barrier_all has been added as a supported API.
Exception and error handling was simplified. All OpenFAM APIs returns only one exception type on error, i.e, Fam_Exception.

OpenFAM version 1.0 (release v0.4.0)

Initial public release of OpenFAM reference implementation
Memory allocator models implemented: "Shared memory model" to be used on scale-up platforms and "Memory server model" to be used on scale-out (cluster) platforms.
Memory server model uses libfabric and support sockets, verbs and psm providers.