Top 10 Software For Your Homelab
You may have lots of reasons for what you run your lab but there is an always growing list of new things to try in your Homelab or server. Check out this quick list of 10 apps that you can try out and see if you can get some new functionality for free through self-hosting.
Docker is a powerful tool that can allow you to host and run apps easily your servers. Now dockers main interface is a command line tool but with and application like Portainer you get a nice GUI interface to manage your hosts docker client.
Portainer supports your running Docker environments, Docker Swarms, local or remote by interfacing either directly with the local docker.sock on your host or the remote docker endpoint. The UI can easily be used for either monitoring the status of containers, reading current logs or building and modifying docker container stacks (Dockerfile).
Its a well thought out interface, well supported with updates and very low maintenance. It try's to guide you where it can to use the more advanced features of docker to your advantage.
This is one of the top-rated and downloaded media servers that you can run to get your own Amazon Prime or Netflix level features with all your own content.
Plex lets you bring all of your own content and it will automatically try to match your files to its database of movies and tv and give you reach features like chapter detection, automatic subtitle detection and stream to devices outside your network or syncing to your devices if you don't want your server to be accessible outside your network.
This application is regularly updated and supported by the Plex company and they are always looking to improve features in the application. Recently the company has been expanding its streaming services further to offer free media streaming through their cloud services.
Their are other options available to you like Emby or Jelly fin so have a check out of all of them and their features and pick the one that gives you the features you want.
This application is best installed on a bare metal machine as it runs as an OS and will allow for more advanced virtualisation features to be used. This OS is installed as a Debian spin with a custom Ubuntu Kernel. This base install has a tiny ram and cpu draw in comparison to other enterprise level virtualisation platforms like ESXi and HyperV so this is a solution suited for lower powered systems as well. One thing to take not of is the if your looking to use Proxmox is that it is using KVM under the hood for virtualisation converting vdmk or ova files to qcow2 can prove to be a chore if your looking to migrate your existing virtual disks.
Proxmox gives you the perfect environment to quickly spin up and down either VM's or Linux containers for your applications to either try things out or iterate on your infrastructure quickly or run your infrastructure efficiently only when needed using Proxmox's easy to use the web interface to manage your new virtualised infrastructure.
Advertising is EVERYWHERE on the internet these days and you can use extensions in your browsers to try and get around these pesky Ads. But what if you could do this for your whole network to have automatic Ad-Blocking for clients that may not have had ad-blocking capabilities initially.
The application can be installed as a container or on a small device like a Raspberry-Pi would work perfectly. You get a nice GUI interface to administer the ad blocking service and setup normal router capabilities like routes, static ip addresses and DHCP servers if need and you want to replace your router with Pi-Hole. Now with the blocking of any network traffic you'll always have a time you need to whitelist a domain that just needs to be connected too even if it leads to some Ads due to the cat and mouse game between us and the Marketing companies. The GUI's of Pi-Hole makes it nice and easy to update and manage these filtering whitelist and black lists either through domains or ip addresses.
Log Management is important, aside from monitoring it will be the thing which provides you with the most information about your systems and what's running on them. There are plenty of options for this problem such as Greylog to ELK and they provide their own unique features and interfaces so consult which one works best for you.
The true power of Paper trail for me however was being able to search logs in real-time via the web browser. Much like ELK and similar software you may host yourself, you can also create Alerts on specific log events which can be pushed by email, slack or other channels should those events occur or when aggregated. The Search bar is also pretty quick when searching back in the logs.
Now if your running applications across multiple containers, multiple hosts or even multiple clouds your gonna get overrun with how you access the logs of these applications to monitor they are running correctly or diagnose any issues you might come across.
Graylog makes it easy to use standard protocols to export your applications logs either through files and R-SYSLOG or through an automatic remote transport like docker and GELF messaging to export the logs. Once centralised you can setup metrics and dashboards from these imported logs to get some more visibility for your applications and infrastructure.
Graylog is recommended to be setup as soon as possible as the advantages gained from being able to quickly see all logs in once place makes debugging even complex issues a breeze.
Netdata is server monitoring on steroids.. Starting off its a local install with a one liner found on the Netdata site. The never ending (seemingly) downward scroll presents graphical real time breakdowns into systems, applications and everything running on the system. Each release provides a greater set of plugins and delving into more far reaching information on applications.
You can run Netdata locally, and use a pattern of Prometheus to pull data from Netdata and push to influx DB and display the output in Grafana. If this sounds like hard work, recently netdata.cloud was launched and you can "claim" your Netdata installs and pull the data into a centralised Netdata Cloud instance. It's early days for the netdata.cloud interface however its slowly improving and is a useful interface for data aggregation.
While many items on this list have "opensourced" alternatives Netdata stands out as a really useful service for both real-time system monitoring and if using Grafana. The fact that Netdata is so usable right out of the box too is a huge advantage of the Netdata application.
Although it may have a high learning curve ansible is a fantastic tool for creating reusable playbooks for comment operations such as shell commands or file copies on your local and remote hosts. This makes complex operations nice and reproducible across multiple hosts allowing for lots of automation across your physical and virtual infrastructure.
Ansible joins the host of tools such as chef or puppet that allows for similar configuring functionality that ansible provides but over the past 5 years ansible has been embraced by the community for its flexibility to build and share configuration scripts for common software or configuration of hosts.
Another tool similar to ansible, Terraform provides you with the ability to write a definition for your virtual infrastructure and apply it to your hosts. If your using technology like docker or virtualisation for your machine you can utilise terraform to create reusable modules of infrastructure that can be configured for their specific use cases.
And example of this would be to create a terraform module that deploys a generic virtual machine with passed in module paraments used to define the resources for that VM. Now in another terraform module can define an array of application machines all with their unique resource configurations for their specific applications. Now looped over our generic virtual machine module we can create identical infrastructure for each VM deployed the exact same way but configured uniquely for each application.
This can allow you to deploy and test identical infrastructure allow for a higher change confidence when updating infrastructure.
Now there are many uses you might have for a web server in your Homelab be it an actual webserver to a reverse proxy so that you can expose your services being run in your Homelab to the internet from a single port under subdomains.
You can easily learn the Nginx configuration language to deploy these configurations for Nginx and get some utlity out of your current applications by securing them behind SSL or to just make it easier for your users to access through a more memorable URL to access your service on.
Nginx can easily be run via its binary installed via apt or with your applications in a lightweight docker container. The docker container deployment is particularly good for deploying application-specific configuration for routing with your applications through a docker-compose file.