A "serious handicap" means having a significant disadvantage or limitation that makes it harder for someone to do things that others can do easily. Full definition
But they are also not going to take care of a man with serious handicaps devoting their best years to him. (womenrussia.com)
Hopefully I will live to see the day when having religious faith is a serious handicap for a person seeking an elected office. (religion.blogs.cnn.com)
This proved a serious handicap, as we are to see in later chapters. (religion-online.org)