Maybe an expert in the field (copyright lawyer) needs to answer but this is what I think the law is.
1. If you work for the builder as his employee, his design.
2. If you modify a plan he owns, I "would say" , his design or someone else. I guess you could claim the portion you drew is yours. Determining which are is yours could be a gray area. In addition, here you may need written permission to modify the design if someone other than the builder owns it. I always check where they got the design to determine any risk here.
3. If you design a plan, even with his guidance and input, it is yours.
4. If you use one of your own plans, modified or not, it is yours.
I also understand that any plan you have the copyright to, the only sure fire court house right he would have, (it is yours) UNLESS you give written ownership to him.
I think anyone who never looked at the copyright law assumes it is theirs because they paid for it. This is not correct.
I would further add, I work with builders that I want a good relationship with. I would never share a design I do for one builder with another builder. I do not use written agreement on use. Many times the builders use the same plan with modifications. I charge for the modifications, but not a full plan. Just my relationship. If someone (never has happened to my knowledge), wants to sell a plan, I would tell them the law. I have explained the law to a few builders who have asked, and they are taken aback that they do not own the plan.