In air defense situations, decision makers have to protect defended assets through assigning available firing units to threatening targets in real-time. To their help they have decision support systems known as threat evaluation and weapon allocation (TEWA) systems. The problem of performance evaluation of such systems is of great importance, due to their critical role. Despite this, research on this problem is close to non-existing. We are discussing the use of survivability and resource usage cost as comparative performance metrics, which can be used for comparing the effectiveness of different system configurations, by using simulations. These metrics have been implemented into a testbed, in which we have performed some comparative experiments. Our results show that changes of individual parts of the threat evaluation and weapon allocation system configuration can have a large effect on the effectiveness of the system as a whole, and illustrate how the metrics and the testbed can be used.