When I make cannoli filling I drain the whey out for 2 or 3 days so it's drier. I use Nick Malageri "s recipe for cannoli filling. I also have seen people use a bit of gelatin in to stabilize the mixture.
Do you use parchment paper on the bottom? Also if you froze it for a bit , that would help remove from the pan.
Also if you try to decorate a freshly baked cake they tend to crumble, so maybe make it the day before and...
What about using a pan size smaller ie. if it's 9 inch cake prepare a cake pan(maybe cheesecake type with removable bottom)that is 8 inches. It would just be one of those cakes that really shouldn't be left sitting out.