After a week of searching for software to set up an address book on my server I found Radicale. Turns out that it provides a calendar service as well. Check out the guide if you want to set it up on your server as well. Link in Server features!
Radicale – your address book and callendar
- RaspberryPi 4 – USB boot
- Debian Deployment Helper