omv7:omv7_plugins:mergerfs [omv-extras.org]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{indexmenu_n>8}}
\\
\\
<html><!---
Remark: Given the similar setup, this doc contains numerous reused screen shots from the OMV6 version of this doc.
---></html>
----
\\

\\
<html><center><b>The MergerFS Plugin For OMV7</b></center></html>

{{ :omvextras_logo4.jpg?400 |}}

====== The MergerFS Plugin For OMV7 ======
\\
\\
<html><center><b>February 21rst, 2025 - Rev 1.0</b></center></html>
\\
\\

===== Summary =====

MergerFS is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to mhddfs, unionfs, aufs.  MergerFS is a good choice for combining a collection of hard drives when other pooling techniques (such as RAID5) should not be used.  MergerFS works well with SBC's (Single Board Computers) and other platforms that use USB connected hard drives.  MergerFS can be added or removed at any time, without a loss of data.  If a member of the merged drive pool fails, data saved to the pool's remaining member drives will be unaffected.     

MergerFS, installed on the root of member drives ([[https://wiki.omv-extras.org/doku.php?id=docs_in_draft:mergerfs&rev=1736293400#option_1|Option 1 below]]), should not be used for relational database storage, where latency can impact performance, or for Dockers where a custom version of overlyfs is used.  In such cases, look at [[https://wiki.omv-extras.org/doku.php?id=docs_in_draft:mergerfs#option_2|Option 2]] where SQL databases and Dockers can be accommodated outside of Shared Folders that are dedicated to MergerFS.

\\
\\

----
 
<html><center><b>MergerFS</b> was designed to aggregate hard drives.</center></html>  
<html><center>A drive pool can be created that is the rough equivalent of the total space provided by all member drives.  Even dissimilar sized drives with different file formats may be used.</center></html>
{{ ::mergerfs-01.jpg?600 |}}

==== Third Party Software Note ====

While this OMV plugin makes the MergerFS package easy to integrate into openmediavault, the MergerFS package itself was created by a third party.  See the MergerFS project / author's -> [[https://github.com/trapexit/mergerfs|web page]] for more detailed information and support.





===== Prerequisites =====

  * [[https://wiki.omv-extras.org/doku.php?id=misc_docs:omv_extras|OMV-Extras]] must be pre-installed.  


===== Installation =====

In OMV6's GUI:\\
Under **System**, **Plugins**, find and highlight **openmediavault-mergerfs 6.X**, and click the **install** button.

===== Configuration =====

==== General ====
Before proceeding with the installation, a decision on the storage policy to be used should be made.  While there are many storage policies to chose from, for the purposes of this document, the default "**Existing path - Most Free Space**" and "**Most Free Space**" will be explained.  Explanations for additional storage policy options are available -> [[https://github.com/trapexit/mergerfs#policy-descriptions|here]].
----
=== Most Free Space ===

<html><center><b>Most Free Space</b> is a single directive that performs exactly what it suggests.</center></html>  
<html><center>If dissimilar drive sizes are used, <b>Most Free Space</b> will direct data toward the largest drive.</center></html>  
{{ ::mergerfs-04.jpg?600 |}}
----
  
<html><center>When the space remaining between all drives is roughly the same, data will be distributed equally among all member drives.</center></html>

{{ ::mergerfs-05.jpg?600 |}}

<html><center>With the added advantage of being able to use different sized drives, the <b>Most Free Space</b> storage policy roughly mimics the storage distribution attribute of RAID5.</center></html>
<html><center>Typically, the <b>Most Free Space</b> storage policy is a good choice when merged drives will be populated with libraries of large video and music files.</center></html>
\\
----

=== Existing Path - Most Free Space ===  

**Existing Path - Most Free Space** contains two directives.  The first directive, **Existing Path**, will direct files to a preexisting folder.  For example, if the folder **Videos** exists on one drive in the merged array, all new files that are saved to **Videos** will be saved to the same drive - I.E. to the **Existing Path**.  If files are saved to new folder, **Documents** for example, the second directive takes control.  The new folder **Documents** will be created on the drive with the **Most Free Space**.  Files saved to the **Documents**, thereafter, will be saved only to drive with the **Documents** folder.  This behavior is known as "path preserving" where paths, and the files that are saved to them, exist on one drive.

Note, in the following, the distribution of paths among member drives.  Once a new folder is created, on a drive with the **Most Free Space**, a path is established.  According to the first directive all subsequently saved files will go to the **Existing Path**.  

----

The issue that the **Existing Path** directive can create is due to the difference in files sizes.  When compared to documents or even picture files, music and video files can be massive, ranging from 100's of megabytes to several gigabytes.  If users increasingly add large video files to a MergerFS array, the **Existing Path** directive can easily fill one drive of the array completely.  At that point, the array will stop accepting files destined for the full drive.\\

{{ ::mergerfs-06.jpg?600 |}}

This issue can be corrected using the **Balance Pool** Tool {{::mergerfs-07-balance_pool.jpg?30|}} which will redistribute data among the array's member drives.\\

<html>
<body>
  <table width="100%" border="0">
    <tr>
      <td colspan="2" style="background-color:#69A5FF;height:30px;">
        <strong><span style="color:#FFFFFF;font-size:110%;">&#160; Note For users who may test MergerFS using small virtual drives:</span></strong>
      </td>
    </tr>
    <tr>
      <td style="background-color:#E6FEFF;height:25px;width:380px;">
        The MergerFS balance tool balances drives to within 2% fill of all member drives. Populating tiny 8 to 10GB virtual drives with video files that may be as large
as 1 or 2GB each (which is roughly 10 to 20% of a member drive) may trigger "ping pong" file copying between drives.  Filling tiny drives with huge files makes the "within 2%" balance goal, of the algorithm, unachievable.  When testing MergerFS using small virtual drives, use small files such as documents, pictures, or small audio files.
    </tr>
  </table>
</body>
</html>
\\
----

===== Setup Option Examples =====
\\
\\
==== Option 1 ====
**Using Entire Drives:**\\
\\
The typical setup of MergerFS is "at the root" of each member drive. Setting up in this manner dedicates each member drive to MergerFS.  All folders and files on each drive are under MergerFS control and are subject to file distribution between drives to balance the MergerFS array.  In the event that the balance tool is used, any number of files and folders may be moved between member drives to achieve the "within 2%" of drive space balance objective.

{{ ::mergerfs-method-01.jpg?nolink&400 |}} 

----
==== Option 2 ====
**Using Root Folders:**

This option creates shared folders at the root of members drives.  Drives with these folders will aggregate files and and folders, within MergerFS dedicated Shared Folders, in the exact same manner as the **Option 1** - "**Full drive**" method.  The difference and advantage is, files and folders //outside// of MergerFS dedicated folders are not under MergerFS' influence.

{{ ::mergerfs-method-02.jpg?nolink&500 |}}
----

**Root Folder Advantages:**

First, it should be noted that using a folder, versus the root of the drive, is not a limitation.  All of the physical space available on a hard drive can be used whether data is stored in a designated folder or at the root of the drive.  However, with MergerFS storage activity contained within a dedicated root folder, on each member drive, drive locations //outside// of MergerFS folders can be used for SQL databases or for Docker Storage.\\
Simply create / save Dockers or SQL DB's to the root of a hard drive or to a Shared Folder that is //outside// of the MergerFS shared folder.  While MergerFS can not move data outside of it's own folders, continuing file and folder growth will be saved "around" SQL DB's and Dockers, thereby balancing storage across drives in the array.  Further, the MergerFS balance tool will have no effect on data that is outside of MergerFS' folders.\\
\\
{{ :omv7:mergerfs-method-02.1.jpg?nolink&450 |}}
\\
----

When considering which of the two set up options to use, read over the information in [[https://wiki.omv-extras.org/doku.php?id=omv7:omv7_plugins:mergerfs#notes|Notes]] and [[https://wiki.omv-extras.org/doku.php?id=omv7:omv7_plugins:mergerfs#file_systems|File Systems]].\\
\\
\\

----

===== Setup Options =====

For users interested in **Option 1**, see Option 1 following.\\
For those who are interested in **Option 2**, follow this link -> [[https://wiki.omv-extras.org/doku.php?id=omv7:omv7_plugins:mergerfs#setting_up_option_2|Setting up Option 2]].\\
\\
\\
----

==== Setting up Option 1 ====

For demonstration purposes, setting up option 1 will focus on pooling three hard drives.\\
----

<html><center>Under <b>Storage</b>, <b>File Systems</b>, note the drives <b>/dev/sdc1</b>, <b>/dev/sdd1</b>, <b>/dev/sde1</b>.</center></html>
<html><center>These drives have been previously formatted with EXT4, they're mounted, and will be added the the Merged drive.  Also note that <b>/dev/sdc1</b> contains data.</center></html>
\\
The **Mount Point** column is not a default.  Optionally, this column can be added by clicking on the grid icon {{::omv6-plugins_remote_mount-04-1.jpg?25|}} and selecting **Mount Point**.  The **Mount Point** column displays drives by UUID and their "mounted" absolute path.

{{ ::mergerfs-03.jpg?800 |}}

----


<html><center>Under <b>Storage</b>, <b>mergerfs</b>, click the <b>Create</b> button.</center></html> 
{{ :omv6-plugins_remote_mount-03-1.jpg?20 |}}
\\
<html><center>In the following, the mount point was named <b>Merger1</b>.</center></html>
<html><center>The three previously noted drives (<b>/dev/sdc1</b>, <b>/dev/sdd1</b>, <b>/dev/sde1</b>) were selected.</center></html>
<html><center>The storage policy was changed from the default to <b>Most free space</b>.</center></html>
<html><center>The remaining settings were left at the defaults.</center></html>

{{ ::mergerfs-08.jpg?800 |}}

<html><center>Click <b>Save</b></center></html>
----

<html><center>Under <b>Storage</b>, <b>mergerfs</b>, <b>Merger1</b> has been created.</center></html> 
<html><center>Note that member drives are represented by their absolute paths which is based on UUID.</center></html>

{{ ::mergerfs-09.jpg?800 |}}

----

<html><center>Under <b>Storage</b>, <b>File Systems</b>, there's a new <b>Device</b> with a new <b>Mount Point</b>:</center></html>

- The **Label** is **Merger1**\\
- Note that the **Device Name** and the **Mount Point** are the same.\\
- Under **Type**, which ordinarily displays the file format type of a Block Device, the **Type** is **FUSE.MERGERFS**.\\
- Note the "**copy icon**" next to the Mount Point.  The copy icon copies the exact Mount Point path to the client workstation's clip board.  This may be useful for command line operations.\\
\\
{{ ::mergerfs-10.jpg?800 |}}

The "Merged Drive", **Merger1**, contains the data that was on /dev/sdc1 along with the free space of the two additional drives.  This it the main feature of MergerFS. It can be used to add new drives, as needed, to drives that have existing data, easily creating more storage space.

----

<html><center>To use the merged drive, <b>Merger1</b>, when setting up a Shard Folder:</center></html>

<html><center>Under <b>Storage</b>, <b>Shared Folders</b>, click the Create button.</center></html>
{{ :omv6-plugins_remote_mount-03-1.jpg?20 |}}
\\
Since existing data resides on one of the drives, a shared folder will be created using that data.  In this instance, the existing data folder "Documents" will be used for this Shared Folder.

  * The **Name** field of this Shared Folder will be **Documents**.
  * In the **File System** field, click the **pop down** menu **arrow** (far right) and select **Merger1**.
  * In the **Relative path** field, to use the existing **Documents** folder, I clicked on the **path icon** (far right), navigated to the **Documents** folder and selected it.  If a folder doesn't exist, the default in the **Relative path** field will be based on the **Name** field.  A new folder will be created.  
  * Set **Permissions**, as needed.

{{ ::mergerfs-11.jpg?800 |}} 

<html><center>Click on <b>Save</b></center></html>
\\
\\

----

<html><center>The Net Result</center></html>
\\
{{ ::mergerfs-12.jpg?800 |}}
\\
To use MergerFS for distributing data among all member drives, all Shared Folders should be created on the MergerFS "filesystem".  (In the case of this example, **Merger1**.)
\\

<html><center>(To share to the network, layer an SMB or NFS share onto this Shared Folder.)</center></html>
----

As noted prior, in the merged drive (**Merger1**) existing data is contained on **/dev/sdc1** and the drive is nearly full.  When **Merger1** was set up, the **Most Free Space** storage policy was used which distributes files and folders evenly among member drives.  However, this policy will not //retroactively// balance data among the member drives, evening out storage among all members.  **Most Free Space** will simply fill drives with less data on them.  This may work well enough if an existing drive, or drives, are not at the edge of their capacity (where they may be triggering alarms). 

If needed or desired, to even out storage among member drives, the **Balance Pool** tool can be used.

----

<html><center>Under <b>Storage</b>, <b>mergerfs</b>, <b>Merger1</b> is selected / highlighted by clicking on it.</center></html>  
<html><center>Then the "<b>Balance Pool</b>" tool is selected.</center></html>

{{ ::mergerfs-13.jpg?800 |}}

<html><center>A window will popup.  Click <b>Start</b>.</center></html>
<html><center>Depending on the amount of data to be moved, this process may take a considerable amount of time.</center></html>
<html><center>At the conclusion, "<b>End of Line</b>" will be displayed.  <b>Close</b> the window.</center></html>

----
\\
<html><center>The result of file redistribution is as follows:</center></html>
<html><center>While the total data under <b>Merger1</b> has not changed, note that data has been distributed among member drives.</center></html>  
<html><center>(There are minor differences between drives because MergerFS balances to within 2%, by default, and it will not "split" very large files.)</center></html>   

{{ ::mergerfs-14.jpg?800 |}}
\\
\\
With the MergerFS array set up, continue on to -> [[https://wiki.omv-extras.org/doku.php?id=omv7:omv7_plugins:mergerfs#additional_tools|additional tools]].
\\
\\
----

==== Setting up Option 2 ====

For demonstration purposes, setting up Option 2 will focus on pooling three 3 separate hard drives, by creating 3 Shared Folders for MergerFS use (one on each drive).\\
----

<html><center>Under <b>Storage</b>, <b>File Systems</b>, note the drives <b>/dev/sdc1</b>, <b>/dev/sdd1</b>, <b>/dev/sde1</b>.</center></html>
<html><center>These drives have been previously formatted with EXT4 and they're mounted.</center></html>

\\
The **Mount Point** column is not a default.  Optionally, this column can be added by clicking on the grid icon {{::omv6-plugins_remote_mount-04-1.jpg?25|}} and selecting **Mount Point**.  The **Mount Point** column displays drives by UUID and their "mounted" absolute path.

{{ ::mergerfs-15.jpg?nolink&800 |}}

----

=== Setting Dedicated Shared Folders for MergerFS ===

Creating a Shared Folder, for each member drive (dev/**sdb1**, dev/**sdc1**, and dev/**sdd1**) is necessary.\\
**Note:** These Shared Folders will be used, **exclusively**, by MergerFS.\\
\\
Under, **Storage**, **Shared Folders**, Click the **Create** "{{:omv6-plugins_remote_mount-03-1.jpg?nolink&20| }}" button.\\
\\
In the **Name** * field:\\
Provide a name that will easily identify the Shared Folder.\\
In the **File System** * field:\\
Select the first drive to be used.  (In this example, it's **/dev/sdb1**)\\
The **Relative path** * field:\\
Accept the default.\\
The **Permissions** field:\\
**Everyone read/write** is recommended.  Note: This won't have an effect on permissions assigned to Shared Folders created within the MergerFS drive.\\
\\

{{ :docs_in_draft:mergerfs-16.jpg?nolink&800 |}}
\\

Repeat this process, creating a designated Shared Folder for each drive that will be in the MergerFS array.  In this example, the following Shared Folders were created, placed on 3 individual drives, .\\
\\
**Merg_SF1** - **/dev/sdb1**\\
**Merg_SF2** - **/dev/sdc1**\\
**Merg_SF3** - **/dev/sdd1**\\

Note the above naming convention.  While the actual name is not important, it's helpful if the Shared Folder name identifies it's intended use. 
----

<html><center><b>The Final Result:</b></center></html> 
<html><center>Each individual drive has it's own MergerFS designated <b>Shared Folder</b>.</center></html>

{{ ::mergerfs-17.jpg?nolink&800 |}} 

----

<html><center><b>Creating the Array:</b></center></html>

<html><center>Under <b>Storage</b>, <b>mergerfs</b>, click the <b>Create</b> button.</center></html> 
{{ :omv6-plugins_remote_mount-03-1.jpg?20 |}}

\\
**Name ***: **Merger1**\\
**File Systems**: ** *__NOT USED__* **\\
**Shared Folders**: Check the boxes for the three Shared Folders, created on each of the three drives, for the MergerFS array.\\
**Create Policy:** The storage policy was changed from the default to **Most Free Space**.\\
\\
The remaining settings were left at the defaults.\\
\\
{{ ::mergerfs-18.jpg?nolink&800 |}}

<html><center>Click <b>Save</b></center></html>
----

<html><center>Under <b>Storage</b>, <b>mergerfs</b>, <b>Merger1</b> has been created.</center></html> 
<html><center>Note that each member drive has a their full mounted path AND each has a shared folder dedicated to MergerFS.  All MergerFS storage will be contained within each drive's folder.</center></html>


{{ ::mergerfs-19.jpg?nolink&800 |}}

----

<html><center>Under <b>Storage</b>, <b>File Systems</b>, there's a new <b>Device</b> with a new <b>Mount Point</b>:</center></html>

- The **Label** is **Merger1**\\
Note that the **Label** column is not a default.  Optionally, this column can be added by clicking on the grid icon {{::omv6-plugins_remote_mount-04-1.jpg?25|}} and selecting **Label**.\\
- Under **Type**, which ordinarily displays the file format type of a Block Device, the **Type** is **FUSE.MERGERFS**.\\
- Note the "**copy icon**" next to the Mount Point.  The copy icon copies the exact Mount Point path to the client workstation's clip board.  This may be useful, later, for command line operations.\\
\\

{{ ::mergerfs-20.jpg?nolink&800 |}}

----


<html><center>To use the merged drive, <b>Merger1</b>, when setting up a Shard Folder for data:</center></html>

<html><center>Under <b>Storage</b>, <b>Shared Folders</b>, click the Create button.</center></html>
{{ :omv6-plugins_remote_mount-03-1.jpg?20 |}}
\\
Since existing data resides on one of the drives, a shared folder will be created using that data.  In this instance, the existing data folder "Documents" will be used for this Shared Folder.

  * The **Name** field of this Shared Folder will be **Documents**.
  * In the **File System** field, click the **pop down** menu **arrow** (far right) and select **Merger1**.
  * In the **Relative path** field, to use the existing **Documents** folder, I clicked on the **path icon** (far right), navigated to the **Documents** folder and selected it.  If a folder doesn't exist, the default in the **Relative path** field will be based on the **Name** field.  A new folder will be created.  
  * Set **Permissions**, as needed.

{{ ::mergerfs-11.jpg?800 |}} 

<html><center>Click on <b>Save</b></center></html>

----

Alternately, if data does not exist:\\

  * **Name** field: User's choice.
  * In the **File System** field, click the **pop down** menu **arrow** (far right) and select **Merger1**.
  * The **Relative path** field:  If a folder doesn't exist, the default in the **Relative path** field will be based on the **Name** field.  A new folder will be created.  
  * Set **Permissions**, as needed.
  * **Save**

----

<html><center><b>The Net Result</b></center></html>
\\
{{ ::mergerfs-12.jpg?800 |}}
\\
<html><center>To use MergerFS for distributing data among all member drives, all <b>Shared Folders</b> should be created on the MergerFS "filesystem".  (In the case of this example, **Merger1**.)</center></html>
\\
<html><center>(To share to the network, layer an SMB or NFS share onto this Shared Folder in accordance with guidance found here -> 
<a href="https://wiki.omv-extras.org/doku.php?id=omv7:new_user_guide#creating_a_smb_cif_samba_network_share">Creating an SMB network share</a>.)</center></html>

----
===== Additional Tools =====

The remaining tools are:
  * **Restart**  (Restarts the pool if there's an error.)
  * **Deduplicate**  (Finds and eliminates duplicate files within the pool.)  The "practice" versions for each type of deduplicate command provide a trial run to check results before committing.
  * **Docs** (Links to the MergerFS support pages for MergerFS and MergerFS tools.) While the MergerFS plugin supports **Restart**, **Balance Pool** and **Deduplicate** in the GUI, note that there are several additional MergerFS tools that can be used from the command line.

----
===== Replacing a Failed MergerFS Drive =====

Since MergerFS is frequently used with SnapRAID, replacing a failed MergerFS Drive is part of the final step in recovering a MergerFS / SnapRAID array, as detailed -> [[https://wiki.omv-extras.org/doku.php?id=omv7:omv7_plugins:snapraid#replacing_a_failed_drive_in_mergerfs|here]]. 
===== Notes =====

  * MergerFS, by design, distributes folders and files between multiple hard drives.  Accordingly, to gain the ability to recover a hard drive along with other recovery features, users are strongly encouraged to consider the addition of **SNAPRAID** to protect a MergerFS array.  Further, -> [[https://wiki.omv-extras.org/doku.php?id=omv7:utilities_maint_backup#backing_up_data|full data backup]] is recommended for recovery from data disasters.

  * Shared Folders, for data storage, should always be added using the merged filesystem mount point.  
  * If relational database files are added directly to a member drive, to get past the latency issue, use [[https://wiki.omv-extras.org/doku.php?id=omv7:omv7_plugins:mergerfs#option_2|Option 2]] - MergerFS with root folders.  The relational database should be installed using the mount point of a physical drive.  This will install the database outside of MergerFS' folder. 
  * Dockers use a form of overlayfs that is not compatible with MergerFS.  If Docker containers and image storage are required, again, [[https://wiki.omv-extras.org/doku.php?id=omv7:omv7_plugins:mergerfs#option_2| Option 2]] - MergerFS with root folders should be used. Docker images and containers should be installed using the mount point of a physical drive.  This will install the Docker related files outside of MergerFS' folder. 
  * If users are already committed to [[https://wiki.omv-extras.org/doku.php?id=omv7:omv7_plugins:mergerfs#option_1|Option 1]]; for relational database files and/or docker storage, a small SSD or hard drive outside of a merged pool is recommended.

=== File Systems: ===

  * MergerFS does not fully support COW (Copy on Write) filesystems like ZFS or BTRFS.  When creating a MergerFS array, simple filesystems like EXT4 or XFS are recommended.\\
  * While MergerFS may work with NTFS or FAT (Windows filesystems) when using a package like ntfs-3g, they are non-POSIX file systems.  File and folder permission issues may result in MergerFS and in OMV.  For the best use experience, Linux native filesystems like EXT4 or XFS are recommended.\\ 
\\





 
 








===== Source Code =====

-> [[https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-mergerfs|Source Code]]

----
===== A Closing Note =====

We, who support the openmediavault project, hope you’ve found this guide to be useful and that you’ll find your openmediavault server to be efficient, easy to use, and enjoyable.\\
\\
If you found this plugin guide to be helpful, please consider a modest donation to support the hosting costs of this server.\\
\\
**OMV-Extras.org**
\\
<html>
<form action="https://www.paypal.com/cgi-bin/webscr" method="post" target="_top">
<input type="hidden" name="cmd" value="_s-xclick">
<input type="hidden" name="hosted_button_id" value="2BQNGSC8HQJME">
<input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donate_SM.gif" name="submit" title="PayPal - The safer, easier way to pay online!" alt="Donate with PayPal button" border="0">
<img alt="" src="https://www.paypal.com/en_US/i/scr/pixel.gif" width="1" height="1" border="0">
</form>
</html>
\\
\\
**Venmo: ryecoaaron** \\
\\