Abstract. The accuracy of the estimation of the wind resource has an enormous effect on the expected rate of return of a wind energy project. Due to the complex nature of the weather and the wind flow over the earth's surface, it can be very challenging to measure and model the wind resource correctly. For a given project, the modeller is faced with a difficult choice of a wide range of simulation tools with varying accuracies (or skill) and costs. In this work, a new method for helping wind modellers choose the most cost-effective model for a given project is developed by applying six different Computational Fluid Dynamics tools to simulate the Bolund Hill experiment and studying appropriate comparison metrics in detail. This is done by firstly defining various parameters for predicting the skill and cost scores before carrying out the simulations as well as for calculating skill and cost scores after carrying out the simulations. Weightings are then defined for these parameters, and values assigned to them for the six tools using a template containing pre-defined limits in a blind test. An iterative improvement process is applied by collecting inputs from the participants of the study. This allows a graph of predicted skill score against cost score to be produced, enabling modellers to choose the most cost-effective model without having to carry out the simulations beforehand. The most effective model is the one with the highest skill score for the lowest cost score, at the flattening-off part of the curve. The results show that this new method is successful, and that it is generally possible to apply it in order to choose the most appropriate model for a given project in advance. This is demonstrated by the good match between the shapes of the skill score against cost score curves before and after the simulations, and by the fact that the tool at the flattening-out point of the curve is the same before and after carrying out the simulations. It is also shown how important it is to take into account other factors that may affect the accuracy and costs of a wind modelling simulation as well as the quality of the aerodynamic equations and the run-time. Several improvements to the method are being worked on, by further examining the discrepancies between the predicted and actual cost and skill scores. Additionally, the method is being extended for calculating all wind directions and the Annual Energy Production, as well as to include mesoscale nesting or forcing. A large number of inputs are being collected as part of a simulation challenge in collaboration with IEA Wind Task 31. The method has a high potential to be extended to a wide range of other simulation applications.