Information and communications technologies (ICTs) are being used increasingly in health care. It is evident that the use of modern ICT offers tremendous opportunities to reduce clinical errors, to support health care professionals, and to increase the efficiency of care. However, there are also hazards associated with such technologies in health care: modern systems are costly, their failures may cause negative effects on patients and staff, and possibly, when inappropriately designed, they may result in spending more time with the computer than with the patient. This all could have a negative impact on the quality of patient care. Therefore, rigorous, generalizable evaluations of ICT applications in health care are absolutely necessary and of great importance for decision-makers and users of future information systems.

Evaluation of health ICT applications is rather difficult for various reasons: The complexity of the evaluation object, comprising technical and human aspects, the complexity of the changing clinical environment, and the low numbers of stakeholders and users willing to perform evaluation studies. Thus, despite a large number of published evaluation studies in medical informatics, many studies are of low quality and do not lead to clear answers to the important questions. These problems have largely been discussed in the medical informatics research community in the last years. Several authors state that evaluators of ICT in health care can learn from the systematic study designs, which are standard for all major clinical trials in health care. Others are of the opinion that there are inherent problems in ICT application evaluation which do not allow the simple transfer of study designs and recommendations from clinical (quantitative) trials to information system evaluation studies and that a multi-method approach would be more appropriate. Overall, it is still unclear how to solve the urgent problems of which knowledge and skills an evaluation researcher needs to perform high-quality studies, and which methods and tools are most useful for which kind of research questions.

The strategic mission of the working group is:

• To promote the necessity of a systematic evaluation of health information systems during their whole life cycle as a precondition for the better support of patient care.
• To promote theory and practice of evaluation of health information systems, taking into account approaches from a variety of scientific fields.
• To develop and promote methods and tools to support the systematic evaluation of the effects of health information systems on structure, process and outcome of patient care, and to give feedback to system builders how to alter their systems to improve effectivity and to avoid negative effects.
  
  

The team coordinated a new book on Applied interdisciplinary theory in health informatics. It was launched during Medinfo 2019 in Lyon:

Scott P, de Keizer N, Georgiou A. Applied interdisciplinary theory in health informatics: a knowledge base for practitioners, Stud Health Tech Inform. Vol 263. 2019. IOS Press.

The recommendations describe scope and content for health IT evaluation courses on master and postgraduate level:

Ammenwerth E, de Keizer N, Brender J, Craven C, Eisenstein E, Georgiou A, Khairat S, Magrabi F, Nykänen P, Otero P, Rigby M, Scott P, Weir C. How to Teach Health IT Evaluation: Recommendations for Health IT Evaluation Courses. Stud Health Technol Inform. 2017;243:3-7.

Example for running health IT evaluation courses from several universities can be found here: https://iig.umit.at/efmi/curricula.htm