Introduction

So, you can use nearly any OS to host a file server. However, some are purpose built for the task, some do it as a side job and some are just a bare bones server designed to host anything.

I'm going to start with Open Media Vault because it's really simple, basic, free and completely focused on making it easy to host files. In spite of this, it doesn't actually prevent you from hosting whatever else you want on it too.

Pros:

Easy to use
Focused
Few limitations
Built on Debian
Free

Cons:

Not really meant for anything other than file sharing

Summary:

It's a web UI built on Debian with a set of pre-installed tools and plugin expansion focused on letting you host files. Notably, there's also a plugin for running Docker containers. Thanks to the Debian base and the ability to SSH in, you can actually host anything you want on it, so long as you know how to install/run it from command line. If you want to get up and running with a file server at minimal cost (free) and minimal effort, without any real restrictions on what it can do, this is a good option. The community support is decent but there aren't many staff working on it, so responses may be slow.

Now, I'm going to list Ubuntu Server, to give a baseline comparison.

Pros:

Optimized Server OS that can run anything.
Community Support
Free

Cons:

Aside from some pre-installed apps that make remote management easier, it does absolutely nothing to hold your hand.

Summary:

If you want to roll a server from scratch, this is pretty much the de-facto starting point. You can use other distros but this one has the best community support and only the very minimum of apps needed to get started. If you have a decent amount of Linux experience and don't care about hand holding, this is actually a good option. Known to have great community support and what a great many guides are already written for.

Proxmox can host a file server VM or can act as one directly but it's going to come with a lot of restrictions and will generally make you work a lot harder in most scenarios. Built on Debian LTS.

Pros:

Easy to spin up VMs
Easily share resources across PCs
Good for setting up clusters
Free

Cons:

Summary:

It can host a file server VM but it's not ideal for running a file server. It's really made for running in a cluster and I wouldn't really recommend it for any other situation. However, it can function individually.

TrueNAS/FreeNAS is actually a NAS OS but very different from OMV. Built on FreeBSD.

Pros:

Easy to setup file sharing
Scalable
Mostly Free

Cons:

No mergerfs or snapraid support
Limited support for USB drives

If you've got the desire to have scalable storage, this is probably the best free option. People running Proxmox tend to set it up in a VM. The software might be free but you're going to spend extra on the hardware because you're going to need something that supports hardware raid. There's not much in the way of software raid for this OS. Don't even bother trying to use it for something other than file sharing.

unRAID is a paid solution with software raid features.

Pros:

Easy to use
Scalable

Cons:

Expensive
Limited use outside of file sharing

Summary:

There are plenty of people that like it but I find it too expensive and too limiting in what it can do. If you really want to pay to have your hand held for one of the most basic server tasks in the existence of servers, this isn't a bad option.

HexOS is an overpriced alternative to unRAID built on TrueNAS. I do not recommend. If you want to pay to have your hand held, go with unRAID.