I don't think it's necessary to refute. In fact, many times what others say is just an expression. It's silly of you to sigh for it. According to what you said above, I think you are similar to me before. I have been troubled by what others say for a long time. I think this is a sign of self-distrust. If you can look down on others in a certain way, you will laugh it off again. But this is a long process.
In addition, what teachers say in class is not necessarily knowledge, but also some thinking about problems. Most people who don't listen to the teacher in class think that what the teacher says is too simple. So I think it's better to listen in class.