The only reason to go back to the office is because it's worth your time. Your experience and performance should be improved by being there. If not, don't.
You can't always get what you want, but if you try sometimes—if you keep it real, choose joy, and find the sweet spot—you just might find you get what you need.