The MergerFS Plugin For OMV6

January 25th, 2023 - Rev 1.0

Version History:

January 25th, 2023 - Rev 1.0 (First Draft)

MergerFS is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to mhddfs, unionfs, aufs. MergerFS is a good choice for combining a collection of hard drives when other pooling techniques (such as RAID5) should not be used. MergerFS works well with SBC's (Single Board Computers) and other platforms that use USB connected hard drives. MergerFS can be added or removed at any time, without a loss of data. If the merged drive pool fails, data saved to the pool's member drives will be unaffected.

MergerFS should not be used for relational database storage where latency can impact performance. More information on use cases where MergerFS should not be used is available → here.

---

MergerFS has been designed to aggregate drives. Even dissimilar sized drives with different file formats may be used, to create a drive pool that is the rough equivalent of the total space provided by all member drives.

While this OMV plugin makes the MergerFS package easy to integrate into openmediavault, the MergerFS package itself was created by a third party. See the project / author's → web page for more detailed information and support.

• OMV-Extras must be pre-installed.

In OMV6's GUI:
Under System, Plugins, find and highlight openmediavault-mergerfs 6.X, and click the install button.

Before proceeding with the installation, a decision on the storage policy to be used should be made. While there are many storage policies to chose from, for the purposes of this document, the default “Existing path - Most Free Space” and “Most Free Space” will be explained. Explanations for additional storage policy options are available → here.

---

Most Free Space is a single directive that performs exactly what it suggests. If dissimilar drive sizes are used, Most Free Space will direct data toward the largest drive.

---

When the space remaining between all drives is roughly the same, data will be distributed equally among all member drives.

With the added advantage of being able to use different sized drives, the Most Free Space storage policy roughly mimics the storage distribution attribute of RAID5. Typically, the Most Free Space storage policy is a good choice when merged drives will be populated with libraries of large video and music files.

---

Existing Path - Most Free Space contains two directives. The first directive, Existing Path, will direct files to a preexisting folder. For example, if the folder Videos exists on one drive in the merged array, all new files that are saved to Videos will be saved to the same drive - I.E. to the Existing Path. If files are saved to new folder, Documents for example, the second directive takes control. The new folder Documents will be created on the drive with the Most Free Space. Files saved to the Documents, thereafter, will be saved only to drive with the Documents folder. This behavior is known as “path preserving” where paths, and the files that are saved to them, exist on one drive.

Note, in the following, the distribution of paths among member drives. Once a new folder is created, on a drive with the Most Free Space, a path is established. According to the first directive all subsequently saved files will go to the Existing Path.

---

The issue that the Existing Path directive can create is due to the difference in files sizes. When compared to documents or even picture files, music and video files can be massive, ranging from 100's of megabytes to several gigabytes. If users increasingly add large video files to a MergerFS array, the Existing Path directive can easily fill one drive of the array completely. At that point, the array will stop accepting files destined for the full drive.

This issue can be corrected using the Balance Pool Tool which will redistribute data among the array's member drives.

Note For users who may test MergerFS using small virtual drives:

MergerFS balances drives to within 2% fill of all member drives. Populating tiny 8 to 10GB virtual drives with video files that may be as large as 1 or 2GB each (which is roughly 10 to 20% of a member drive) may trigger "ping pong" file copying between drives. Filling tiny drives with huge files makes the "within 2%" balance goal, of the algorithm, unachievable. When testing MergerFS using small virtual drives, use small files such as documents, pictures, or small audio files.

---

For demonstration purposes, this setup will focus on pooling three hard drives.

Under Storage, File Systems, note the drives /dev/sdc1, /dev/sdd1, /dev/sde1. These drives have been previously formatted with EXT4, they're mounted, and will be added the the Merged drive. Also note that /dev/sdc1 contains data.
The Mount Point column is not a default. Optionally, this column can be added by clicking on the grid icon and selecting Mount Point. The Mount Point column displays drives by UUID and their “mounted” absolute path.

---

Under Storage, mergerfs, click the Create button.
In the following, the mount point was named Merger1. The three previously noted drives (/dev/sdc1, /dev/sdd1, /dev/sde1) were selected. The storage policy was changed from the default to Most free space. The remaining settings were left at the defaults.

Click Save

---

Under Storage, mergerfs, Merger1 has been created. Note that member drives are represented by their absolute paths which is based on UUID.

---

Under Storage, File Systems, there's a new Device with a new Mount Point:

- The Label is Merger1
- Note that the Device Name and the Mount Point are the same.
- Under Type, which ordinarily displays the file format type of a Block Device, the Type is FUSE.MERGERFS.
- Note the “copy icon” next to the Mount Point. The copy icon copies the exact Mount Point path to the client workstation's clip board. This may be useful for command line operations.

The “Merged Drive”, Merger1, contains the data that was on /dev/sdc1 along with the free space of the two additional drives. This it the main feature of MergerFS. It can be used to add new drives, as needed, to drives that have existing data, easily creating more storage space.

---

To use the merged drive, Merger1, when setting up a Shard Folder:

Under Storage, Shared Folders, click the Create button.
Since existing data resides on one of the drives, a shared folder will be created using that data. In this instance, the existing data folder “Documents” will be used for this Shared Folder.

• The Name field of this Shared Folder will be Documents.
• In the File System field, click the pop down menu arrow (far right) and select Merger1.
• In the Relative path field, to use the existing Documents folder, I clicked on the path icon (far right), navigated to the Documents folder and selected it. If a folder doesn't exist, the default in the Relative path field will be based on the Name field. A new folder will be created.
• Set Permissions, as needed.

Click on Save

---

The Net Result

(To share to the network, layer an SMB or NFS share onto this Shared Folder.)

---

As noted prior, in the merged drive (Merger1) existing data is contained on /dev/sdc1 and the drive is nearly full. When Merger1 was set up, the Most Free Space storage policy was used which distributes files and folders evenly among member drives. However, this policy will not retroactively balance data among the member drives, evening out storage among all members. Most Free Space will simply fill drives with less data on them. This may work well enough if an existing drive, or drives, are not at the edge of their capacity (where they may be triggering alarms).

If needed or desired, to even out storage among member drives, the Balance Pool tool can be used.

---

Under Storage, mergerfs, Merger1 is selected / highlighted by clicking on it. Then the "Balance Pool" tool is selected.

A window will popup. Click Start. Depending on the amount of data to be moved, this process may take a considerable amount of time. At the conclusion, "End of Line" will be displayed. Close the window.

---

The result of file redistribution is as follows: While the total data under Merger1 has not changed, note that data has been distributed among member drives. (There are minor differences between drives because MergerFS balances to within 2%, by default, and it will not "split" very large files.)

---

The remaining tools are:

• Restart (Restarts the pool if there's an error.)
• Deduplicate (Finds and eliminates duplicate files within the pool. The “practice” versions for each type of deduplicate command provide a trial run to check results before committing.)
• Docs (Links to the MergerFS support pages for MergerFS and MergerFS tools.) While the MergerFS plugin supports Restart, Balance Pool and Deduplicate in the GUI, note that there are several additional MergerFS tools that can be used from the command line.

---

Since MergerFS is frequently used with SnapRAID, replacing a failed MergerFS Drive is part of the final step in recovering a MergerFS / SnapRAID array, as detailed → here.

• MergerFS, by design, distributes folders and files between multiple hard drives. Accordingly, to gain the ability to recover a hard drive along with other recovery features, users are strongly encouraged to consider the addition of SNAPRAID to protect a MergerFS array. Further, → full data backup is recommended for recovery from data disasters.

• Shared Folders should always be added using the merged filesystem mount point.
• If a Shared Folder is added directly to a member drive (this is possible), thereafter, the Balance Pool tool can not be used.
• If relational database files are added directly to a member drive, to get past the latency issue, the Balance Pool tool can not be used.
• Docker containers and image storage should not be added to a mergerfs pool. Dockers use a form of overlayfs that is not compatible with MergerFS.
• For relational database files and/or docker storage, a small SSD or hard drive outside of a merged pool is recommended.

• File Systems:

- MergerFS does not fully support COW (Copy on Write) filesystems like ZFS or BTRFS. When creating a MergerFS array, simple filesystems like EXT4 or XFS are recommended.
- While MergerFS may work with NTFS or FAT (Windows filesystems) when using a package like ntfs-3g, they are non-POSIX file systems. File and folder permission issues may result in MergerFS and in OMV. For the best use experience, Linux native filesystems like EXT4 or XFS are recommended.

→ Source Code

---

We, who support the openmediavault project, hope you’ve found this guide to be useful and that you’ll find your openmediavault server to be efficient, easy to use, and enjoyable.

If you found this plugin guide to be helpful, please consider a modest donation to support the hosting costs of this server.

OMV-Extras.org

Venmo: ryecoaaron