The term "literary agent" refers to a person who represents writers and their works, helping them to secure publishing contracts, negotiate advances, and promote their books. In this context, "one's literary agent" means an individual's representative or advocate in the world of literature and book publication.