After a week of searching for software to set up an address book on my server I found Radicale. Turns out that it provides a calendar service as well. Check out the guide if you want to set it up on your server as well. Link in Server features!