What is the most challenging thing about being a manager?

What is the most challenging thing about being a manager?