If you can't think of a reason why your dog might not want to sit with you perhaps you hug him (dogs don't like hugs) or if he is a long - coated breed maybe he is just too hot sitting next to you?
One of the oldest of the toy breeds, the English Toy Spaniel probably has ancestors from Japan and maybe even China, which bred with European toy spaniels to produce a small, docile dog with a feathery coat and a pushed - in nose.