You are probably thinking of the the effect that Tamoxifen may have on lowering IGF-1 levels. This doesn't seem to occur w/ clomiphene, so it may not be a matter of estrogen levels (or estrogen receptor agonism).
Haven't seen any info. that an aromatase inhibitor would inhibit muscle gains via lowering IGF-1, but estrogen w/ testosterone can increase muscle mass more that test alone (at least in cattle - this is why estrogen is in the implants - some of this increase also is due to more "marbling" in the meat - intermuscular fat).
The question is whether the estrogen's effects on muscle gain outweigh its effects on fat gain. I think not.
-Randy