I know for sure Intel server CPUs (for example, Xeons) don’t have integrated graphics and somebody else mentioned that many Ryzen CPUs don’t either.

That said, I’m not sure if they need integrated graphics to have hardware decoding for certain video codecs (though it makes sense that those things come together). Personally when I was looking into buying a mini-PC to transform into a media server for my living room, I made sure to check in the CPU specs if it had both integrated graphics (as it’s connected to my TV) and hardware decoding for the most recent codecs (I ended up going for a mini-PC with an N100 processor, since it has all that plus its power consumption is pretty low with a TDP of 6W so I can leave it always on as it also doubles as homeserver).