Where to start

So, there's basically four types of servers most people will want in their home lab.

This first and most common is storage aka a file server.  There's a range of hardware to discuss here.

The second is appliance type servers.  They're usually single board computers with only one or a few functions being served.

The third is media servers.  They often require the ability to transcode and can also be thought of as a general purpose server.

The fourth is an AI server.  The entry level to this will cost as much as a moderate to high end gaming PC.  In fact, most people will end up converting a gaming PC into their first AI server.  You can technically run AI on lower end hardware but I wouldn't call it functional.

Now, in terms of actual hardware, these are the types of servers you are likely to find:

SBC - Single board computers.  These are the Raspberry Pis and the ZimaBlades of the server world.  They're typically what you find acting as an appliance server.  Can be used for low performance/test clusters.  Price range is about $200 or less.

Mini PCs - This is a relatively new entry to the field.  They actually make great general purpose/media servers.  They can also serve as a significant step up from SBCs for forming a cluster.  Price range tends to be $200 or more.  Honestly, the only reason they're a separate category from SBCs is because of the significant jump in price/performance/features that happens around the $200 mark.

Custom PCs - This is a desktop PC you build yourself from parts.  This is only practical when you either have very specific requirements or when a mini PC is no longer practical for the performance you need.  AI is the primary service I can think of that you'd want one for.

Enterprise Servers - These are a mixed bag and I generally don't recommend them in a home lab environment, unless you specifically need one to learn on, such as when pursuing a Cisco certification.  You'll find almost exclusively used ones in home labs.  Most normal people simply cannot afford a new one, much less consider it a practical expense.  What's worse is proprietary hardware can become a major gotcha here.