The current methodology to evaluate default and bankruptcy prediction models is to determine their precision - the percentage of firms predicted correctly. In this study we develop a framework for incorporating Type I (the amount lost from lending to a firm which goes bankrupt) and Type II (the opportunity cost of not lending to a firm which does not go bankrupt) error costs into the evaluation of prediction models. We then test this new framework by comparing the prediction model with a naive model of lending to all firms in the population based on the net profit each would generate. Our results indicate that prediction models can outperform naive models or other models only under certain conditions. This supports our hypothesis that the usefulness of prediction models cannot be fully assessed independently of the costs of forecast errors.