My bro has worked for me for 7yr. He is now trying to steel my contacts by saying I am leaving to have a baby

My bro and his wife have been contract workers for my Co.for 7 yrs. We work in construction doing a specialty job. I trained them and supported them. He is out telling un-truths on the work sites to gain business. Spending rumors that I am closing my 8+ yr business to get pregnant. Now I have send out a memo for damage control and maybe take him to court. This could hurt both of our futures. Help!
Also, I have a husband and he is the primiar provider for his 2 kid and wife. Does that matter?