The phrase "to leave the bench" typically means a judge or referee is no longer participating or presiding over a trial or game. They are stepping down or ending their role. Full definition
I'm confident that many of them have considered leaving the bench at some point but could never quite bring themselves to do it. (sciencemag.org)
Those judges who leave the bench aren't looking for a $ 50,000 pay raise (which, realistically speaking, is the most I'd ever see them getting). (legalblogwatch.typepad.com)
You're considering leaving the bench behind and finding a job in science policy. (sciencemag.org)