Evaluation Tool

In order to evaluate the performance of submissions to the AbstractSwarm competition, the total waiting time of all agents in an evaluation scenario is calculated. The total waiting time is considered a measure for the performance of a submission, since when the agents find a good solution, they will have overall less idle time during the respective run. Normalizing each value with an estimation of the maximum total waiting time possible ensures that all evaluation scenarios contribute equally to the evaluation. All calculations are carried out by the PerformanceEvaluation tool, a Java implementation that can run alongside the AbstractSwarm framework. After processing the simulation and carrying out all calculations for the normalization, the tool returns the results from each run in tabular and graphical form. With this data, the area under the curve can be calculated for each agent implementation. This value is used to determine the winner and final ranking of all submissions to the competition.