Mobile Clinical Decision Support Systems (CDSS) aim to assist medical practitioners with decision making and provide an easy access to information on the move using mobile devices such as a mobile phone. Despite many benefits of mobile CDSS, these systems need to undergo a comprehensive evaluation to determine the correctness of the actions that decision maker take and validate the system's knowledge and recommendation. In this paper, we discuss the impact of mobile CDSS on decision making process and outcomes, and describe the multi-criteria evaluation of a mobile field triage decision support system in terms of effectiveness, efficiency, satisfaction and use. In this research, data was collected from a group of qualified paramedics using both quantitative and qualitative methods. The results and findings shed new light on the role of mobile decision support systems in improving healthcare delivery and provided better understanding of the barriers to the adoption of these systems and the impact of different stakeholders' views in evaluating the success of the DSS.