"To leave the center" means to move away from the most important or influential position or to deviate from the main focus or purpose.