Image classification has advanced significantly with deep learning in a well-controlled setting, where the test data are clean and from the same distribution as the training data. However, the deployment of deep learning models in the real world is still full of unknowns. More often than not, well-trained models can come across Out-of-Distribution (OoD) data that are sampled from a different distribution from that of the training data. For example, image classification models may encounter images that do not belong to any of the classes in the training data or simply In-Distribution (ID) images corrupted by noises and blurs. The problem of detecting Out-of-Distribution (OoD) data when using deep neural networks has been analyzed, and a simple yet effective way to improve the inter-class fairness and robustness of several popular OoD detection methods in the literature is proposed in this work.