You will also run into duplex LTV restrictions... so lets say you bought it for 100K cash, put 15K into it... then tried to refi - the banks asks for 6 - 12 months seasoning and max
ltv on a duplex 80 % LTV... you have tied up the funds and end up with 65,000 back... your going to run out of reserves.