An extensive literature on reputation signaling in prosocial settings has focused on an intrinsic desire for positive reputation. In our paper, we provide experimental evidence that some individuals are averse to both positive and negative reputation and will therefore respond to visibility by signaling that they are an "average altruism type" relative to their audience. We formalize our hypotheses about "wallflower" behavior in a theoretical model. Our experimental results show that instead of uniformly increasing contributions, visibility draws contributions towards the middle of others' contributions. As a result, visibility is associated with higher levels of giving only when in scenarios where others are giving a large amount. We also observe heterogeneity in reputation concerns wallflower behavior is particularly strong for women and can be observed in several different settings.