Structured interviews with Kahneman

How „Thinking, Fast and Slow“ gives hands-on ideas for improving candidate selection

I cut myself through Kahneman’s bestseller, which is admittedly a bulky book. Whereas I admire Part I-III, it is Part IV (about the way we make choices) which was a tough chapter for me. However, I can highly recommend this book to those being interested in the way humans work and how this plays into principles of economics.

Chapter 21 „Intuition vs. formulas“ covers comprehensive advise on how to improve interview procedures using concrete language beyond the human resources jargon. If you are about to have a training with line managers or a talk to management about why you need to build up structured interview competence, read this chapter first. Or send it out as a copy. It is a great way to describe the importance of structured decision-making without being at risk of throwing around HR buzzwords.

Kahneman describes that experts are always inferior to algorithms – a bold statement that most probably you are about to disagree on. He sees two reasons for his claim. First of all, experts try to be clever by making situations more complex, which reduces validity. Secondly, humans are inconsistent when summarising their judgement of complex information.

How does this apply to interview procedures?

If you are not amongst the lucky ones having well-established structured interview guidelines at hand, Kahneman compiles all you need to know in two paragraphs. The below main points are enriched by my own thoughts and experience.

  • Decide on roughly six success criteria (traits) for the position you are about to fill. They should be as independent from one another as possible and assessable by asking questions during the interview.
  • Decide on how you want to weight each trait in the overall final result of an interview. This is also one of my major learnings after I implemented my first structured interview together with a line manager. If you are unsure on how to weight, rate all traits as equally important.
  • Decide on questions you want to ask to assess each trait, decide on a rating scale (maybe 1-5, make sure to leave room for comments) and discuss, what a good/bad answer would look like for you. The last point is especially crucial, if you have more than two assessors and if you want to brief other assessors later on.
  • By leaving time for open questions at the end of each interview, you can give some power back to the line manager (especially interesting for sceptical line managers). Be careful that the answers to those questions do not interfere with the answers to the standardized questions before.
  • During the interview, each trait should be evaluated one after the other, only rate the next one after you rated the preceding one. This means: continue to the next question only after everybody has evaluated the recent trait based on the current answer to the interview question.
  • When you have several interviewers, mention, that it does not help to cheat. Rather than looking at the rating of others for help, assors should make comments about a candidate answer to justify their ratings afterwards.
  • Plan some minutes after each interview to discuss the results with all interviewers. If the results differ, ask for concrete examples from the candidate’s answers. After the discussion, average all assessors ratings per question.
  • Add up the scores based on the weighting you have agreed upon.
  • Truly believe that you will make a better decision based on this procedure. Be bold and choose the candidate with the best average rating.

The most difficult part of structured interviews and candidate assessment is actually sticking to the procedure. It demands a lot of discipline. Keep this in mind. Sometimes it takes some iterations and trials until everybody can agree on the benefits of making sound decisions. From my experience it’s worth the effort.

A critical perspective on competence-based assessment approaches in relation to HR policies: The potential of social network analysis to reveal authentic learning experience

This text was originally handed in as my final assignment of course TIA130 V15 Applied Research Methods and Design for Information Technologies and Learning within the International Master’s programme Information Technology & Learning at Gothenburg University on March 25st 2015. 


Human Resource (HR) policies are powerful instruments to define continuous professional learning strategies in workplace environments. They constitute definitions of central concepts such as knowledge, learning and competence for a specific organization and thus contribute to distinguish between these terms instead of using them synonymously (Säljö, 2012). Consistency in underlying theories can strengthen the organisation’s capability by combining “human capital (people skills and knowledge), social capital (relationships between people) and organisational capital (the organisation’s processes), and aligning them such that each supports the others” (Harris & Short, 2014, p. 5). In a society where knowledge becomes the most valuable good; including consistent epistemological, pedagogical and assessment related theoretical concepts in HR policies to reveal and evaluate this knowledge becomes increasingly important. Additionally, rising complexity and uncertainty in the world around us demands a focus shift from training to learning, from instructors to learners and from training departments to workplaces as a whole (Harris & Short, 2014). By saying so, future policies must not only embrace management theories but theories of knowledge management and learning to fulfill this increasing demands.

Competence-based assessment enables, besides others, the acknowledgement of pre-defined competences in a systematic way. However, coining the term competence is a challenging undertaking. Most definitions can be attributed to a middle space between competence as collective/universal attribute and competence as individual capacity (Le Deist & Winterton, 2005). The same authors propose a “multi-dimensional holistic competence approach”, which they see as an “opportunity of better aligning educational and work-based provision as well as exploiting the synergy between formal education and experiential learning to develop professional competence” (Le Deist & Winterton, 2005, p. 40). In this approach, competence is illustrated as a tetrahedron, consisting of cognitive competence (knowledge and understanding), functional competence (skills, social competence) and social competence (behavioural and attitudinal competencies) with a meta-competence being the facilitator for acquiring the other competences. The difficulty lies in the demand for describing competence in a multi-dimensional way. Additionally, such approach affects assessment practise, which has to reflect underlying competence assumptions. Another challenge is the ambiguity of the cognitive competence being the only component covering knowledge, while at the same time being inseparable from the other components. This competence approach merely touches upon socio-cultural lenses of knowledge, where this can be created, stored and retrieved through the embedded context the individual finds him-/herself in. Hence, it only implicitly considers workplace networks conceptualizations, such as the workplace exchange network proposed by Cole, Schaninger and Harris (2002). However, making these networks explicit is necessary to reveal not only social networks per se, but another layer of an authentic professional learning experience.

If workplace environments are seen as best represented by socio-cultural learning theories, the future bottleneck will be competence-based assessment practices. This is because firstly, power will shift from employers to employees due to demographic developments and secondly, acknowledgement of competence will continue to be the driving force within labour markets. Constantly re-defining required competences for this assessment practices is a bottomless pit in the light of the demand for more flexible position descriptions and rising expectations of employees. However, expanding the competence approach by measures of social network analysis (SNA) to describe the positions of individual actors and groups as well as their relations (ties) to one another within a certain network can contribute to extenuate the bottleneck effects. In the course of this thesis it will be necessary to give an overview of components which SNA can reveal in relation to learning assessment practises and how they can be interpreted in a meaningful way considering the organizational context. Another perspective SNA offers is the intent to represent actual networks independently of pre-defined competences. As a result, categories such as formal/informal learning could be partially merged by evaluating employees’ networks as a whole. Generating explicit competences can promote misleading dichotomies, such as for example body/mind (is there a strict distinction between cognitive, social and functional competence?) and formal/informal (how does this approach embrace and promote spontaneous learning activities outside of formal schedules for learning activities?). If knowledge and learning is said to be embodied in the contextual environment of the actors in a network, using competence-based assessment approaches might be partly inconsistent.

In my research proposal, I suggest a shift in assessment approaches by exploring SNA as a method of the emerging field of learning analytics. As for the exact measures, I still have to decide on a network principle I would like explore. For this, I will need some more extended literature review on previous research efforts. As for now, I stick to the measure of centrality, being the most important tool for analysing the importance of social network actors (Carrington, Scott, & Wasserman, 2005). All in all, I claim that SNA provides potential to reveal, measure and evaluate employees’ learning networks and their learning experience based on diverse data sources readily available. In a competitive economic environment, advantages will lie within making use of data and having power over developing suitable algorithms for the good of the organization. Social networks represent knowledge to a certain extend and thus should be considered as assessment approach for employee suitability and the value of knowledge in organizational HR strategies.

The research question for my proposal is formulated by

Can centrality measures of social network analysis (SNA) complement competence-based assessment approaches in workplace environments to support the implementation of HR policies based on socio-cultural/embodied learning theories?

Measures of centrality in SNA have been discussed widely but can be narrowed down to the three most important concepts degree, betweenness and closeness (Freeman, 1978). Competency based assessment approaches refer to defining sets of competence as a basis for assessing concepts such as employee performance and potential. HR policies are seen as representations of the strategic perspective on all human resource related topics within a certain organization. Socio- cultural/embodied learning theories represent the branch of learning theories within which learning and knowledge is seen as inseparable from its context and the social networks in which it occurs.

As for prior research it becomes clear that the focus lies on formal school settings and includes mostly students, indicating a gap of researching learning in organizations in general. In addition, most research has been conducted in online environments, leaving aside the potential to use centrality measures for offline social networks as well. In relation to organizational educational settings, it has been proposed to focus more on uncovering the holistic (authentic) professional learning experience than the evaluation of specific learning programmes (Webster-Wright, 2009).

Dawson (2008) used social network analysis to determine the relation between a student’s position in the social network and their reported sense of community. She concluded, that the centrality measures closeness and degrees are positively correlated to the reported sense of community, while betweenness show the opposite correlation. In addition, pre-existing external social networks influence the sense of community experienced to a certain extend. Kovanović, Joksimović, Gašević, and Hatala (2014) investigated the concept of social presence and it’s positive correlation to network centrality measures by taking data from student discussion boards. The relationship between digital learning and network learning has been detected by the study of Ünlüsoy, De Haan, Leander, and Volker (2013). Levels of activism and their correlation to occupational ties for movement participants have been investigated by Tindall, Cormier, and Diani (2012). All of these studies can be valuable starting points for my research proposal to identify more explicit concepts.

The objectives of this study are to evaluate two perspectives on learning in current HR research, namely workforce development and continuous professional learning. Furthermore, it intends to identify the importance of terms such as competence, knowledge, learning and assessment in HR policies and demonstrate the limitations of competence-based assessment approaches from a socio-cultural stance on learning. By illustrating the potential of SNA, it depicts a contributor to an assessment approach which can reveal authentic learning experience in the workplace. To illustrate the potential of SNA, this research investigates the correlation between network position focusing on centrality measures and learning experience by analysing and comparing learner’s position in social networks within a formal online training course and in the internal social media platform in general.

The intended audience for this study are learning policy and strategy makers in workplace environments, particularly focusing on assessment practices and the notion of knowledge in organizational environments. The proposed study is particular useful as it takes a critical stance on competence management, assessment practices and their significance for future continuous professional learning strategies. Furthermore by intending to be even more specific in chosen terminology and methodology, it aims at closing the identified gaps in the literature. One important aspect that is still left open is the identification of specific concepts, which shall be researched. The way this proposal is formulated now is still too broad and bold in assumptions and to less focussed on actual underlying theories. Assessment is one concept which needs clarification, especially to support its importance in this proposal as a whole. The same applies for the importance of knowledge and the particular link to underlying theories such as knowledge management and management theories.

Literature review

Within HR policies wording and terminology are important factors. Rephrasing professional development by continuous professional learning could contribute to exploring the holistic learning experience of professionals. In addition to this shift in terminology to address authentic professional learning, Webster-Wright (2009) identifies a lack of research when it comes to exploring the latter holistic learning experience. Assessment is said to be a critical factor within the epistemology – pedagogy – assessment triangle and reflects assumptions about knowledge and learning. Methods of learning analytics shall be used only when these intersections are clearly understood and considered (Knight, Buckingham, & Littleton, 2014). By revealing social networks of employees, this approach could contribute to respond to future research suggestions investigating coherent models of learning within workplace environments which include formal and informal learning aspects (Kyndt & Baert, 2013).

The gap exists when it comes to relating measures of SNA to the value of an individual’s position within his/her social networks in an organizational context. This is because most studies using measures of SNA are mostly performed in educational settings and relate these measures to competency based assessment results.

Hence, there is a missing link between measures of SNA and their potential for economic network value estimation. Some research might guide directions, such as research into social networks and sense of community (Dawson, 2008) or social capital (Kovanović, Joksimović, Gašević, & Hatala, 2014). However, these studies are again connected to academic performance in formal education settings. Current research additionally attempts to create approaches where data is used to extract competencies and match them with the organizational recruiting requirements. But these approaches are based on defined competencies, an approach I am aiming to complement (or even substitute) by using measures of SNA and borrowing suitable terminology for this purpose. For the development of my proposal, it is necessary to identify those research papers using SNA to describe social networks for the sake of being networks and using appropriate complementary concepts to value them.



Carrington, P. J., Scott, J., Wasserman, S. (2005). Models and methods in social network analysis: Elektronisk resurs. Cambridge: Cambridge University Press.

Cole, M. S., Schaninger, W. S., & Harris, S. G. (2002). The workplace social exchange network: A multilevel, conceptual examination. Group & Organization Management, 27(1), 142-167. doi:10.1177/1059601102027001008.

Dawson, S. (2008). A study of the relationship between student social networks and sense of community. Educational Technology & Society, 11(3), 224–238.

Freeman, L. C. (1978). Centrality in social networks conceptual clarification. Social Networks, 1(3), 215-239. doi:10.1016/0378-8733(78)90021-7.

Gonczi, A. (2004). The New Professional and Vocational Education. In Foley (Ed.) The Dimensions of Adult Education (pp. 19-34). Open University Press.

Harris, R., Short, T. (2014). Workforce development: Perspectives and issues. Singapore: Springer Singapore.

Haythornthwaite, C. (1996). Social network analysis: An approach and technique for the study of information exchange. Library and Information Science Research, 18(4), 323-342. doi:10.1016/S0740-8188(96)90003-1.

Hogan, B., Carrasco, J. A., & Wellman, B. (2007). Visualizing personal networks: Working with participant-aided sociograms. Field Methods, 19(2), 116-144. doi:10.1177/1525822X06298589.

Kovanović, V., Joksimović, S., Gašević, D., Hatala, M. (2014), “What is the source of social capital? The association between social network position and social presence in communities of inquiry,” In Proceedings of 7th International Conference on Educational Data Mining – Workshops, London, UK.

Knight, S., Buckingham Shum, S. & Littleton, K. (2014). Epistemology, assessment, pedagogy: where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2) pp. 23–47.

Kyndt, E., & Baert, H. (2013). Antecedents of employees’ involvement in work-related learning: A systematic review. Review of Educational Research, 83(2), 273-313.

Le Deist, F. D., & Winterton, J. (2005). What is competence? Human Resource Development International, 8(1), 27-46. doi:10.1080/1367886042000338227.

Tan, E. (2014). Human capital theory: A holistic criticism. Review of Educational Research, 84(3), 411-445. doi:10.3102/0034654314532696.

Säljö, R. (2012). Literacy, digital literacy and epistemic practices: The co-evolution of hybrid minds and external memory systems. Nordic Journal of Digital Literacy, 7(1), 5-20.

Tindall, D. B., Cormier, J., & Diani, M. (2012). Network social capital as an outcome of social movement mobilization: Using the position generator as an indicator of social network diversity. Social Networks, 34(4), 387-395. doi:10.1016/j.socnet.2011.12.007.

Tubaro, P., Casilli, A. A., & Mounier, L. (2014). Eliciting personal network data in web surveys through participant-generated sociograms. Field Methods, 26(2), 107-125.

Ünlüsoy, A., De Haan, M., Leander, K., & Volker, B. (2013). Learning potential in youth’s online networks: A multilevel approach. Computers & Education, 68, 522.

Webster-Wright, A. (2009). Reframing professional development through understanding authentic professional learning. Review of Educational Research, 79(2), 702-739. doi:10.3102/0034654308330970.