We address the question of multi-task algorithm selection in combinatorial optimisation domains. This is motivated by a desire to simplify the algorithm-selection pipeline by developing a more general classifier that does not require specialised information per domain, and the potential for transfer learning. A minimum requirement to achieve this is to find a common representation for describing instances from multiple domains. We assess the strengths and weaknesses of three candidate representations (text, images and graphs) which can all be used to describe three different application domains. Two setups are considered: single-task selection where one classifier is trained per domain, each using the same representation, and multi-task selection where a single classifier is trained with data from all three domains to output the best solver per instance. We find that the domain-agnostic representations perform comparably with domain-specific feature-based classifiers with the benefit of providing a generic representation that does not require feature identification or computation, and could be extended to additional domains in future.