A critical perspective on competence-based assessment approaches in relation to HR policies: The potential of social network analysis to reveal authentic learning experience

This text was originally handed in as my final assignment of course TIA130 V15 Applied Research Methods and Design for Information Technologies and Learning within the International Master’s programme Information Technology & Learning at Gothenburg University on March 25st 2015. 

Introduction

Human Resource (HR) policies are powerful instruments to define continuous professional learning strategies in workplace environments. They constitute definitions of central concepts such as knowledge, learning and competence for a specific organization and thus contribute to distinguish between these terms instead of using them synonymously (Säljö, 2012). Consistency in underlying theories can strengthen the organisation’s capability by combining “human capital (people skills and knowledge), social capital (relationships between people) and organisational capital (the organisation’s processes), and aligning them such that each supports the others” (Harris & Short, 2014, p. 5). In a society where knowledge becomes the most valuable good; including consistent epistemological, pedagogical and assessment related theoretical concepts in HR policies to reveal and evaluate this knowledge becomes increasingly important. Additionally, rising complexity and uncertainty in the world around us demands a focus shift from training to learning, from instructors to learners and from training departments to workplaces as a whole (Harris & Short, 2014). By saying so, future policies must not only embrace management theories but theories of knowledge management and learning to fulfill this increasing demands.

Competence-based assessment enables, besides others, the acknowledgement of pre-defined competences in a systematic way. However, coining the term competence is a challenging undertaking. Most definitions can be attributed to a middle space between competence as collective/universal attribute and competence as individual capacity (Le Deist & Winterton, 2005). The same authors propose a “multi-dimensional holistic competence approach”, which they see as an “opportunity of better aligning educational and work-based provision as well as exploiting the synergy between formal education and experiential learning to develop professional competence” (Le Deist & Winterton, 2005, p. 40). In this approach, competence is illustrated as a tetrahedron, consisting of cognitive competence (knowledge and understanding), functional competence (skills, social competence) and social competence (behavioural and attitudinal competencies) with a meta-competence being the facilitator for acquiring the other competences. The difficulty lies in the demand for describing competence in a multi-dimensional way. Additionally, such approach affects assessment practise, which has to reflect underlying competence assumptions. Another challenge is the ambiguity of the cognitive competence being the only component covering knowledge, while at the same time being inseparable from the other components. This competence approach merely touches upon socio-cultural lenses of knowledge, where this can be created, stored and retrieved through the embedded context the individual finds him-/herself in. Hence, it only implicitly considers workplace networks conceptualizations, such as the workplace exchange network proposed by Cole, Schaninger and Harris (2002). However, making these networks explicit is necessary to reveal not only social networks per se, but another layer of an authentic professional learning experience.

If workplace environments are seen as best represented by socio-cultural learning theories, the future bottleneck will be competence-based assessment practices. This is because firstly, power will shift from employers to employees due to demographic developments and secondly, acknowledgement of competence will continue to be the driving force within labour markets. Constantly re-defining required competences for this assessment practices is a bottomless pit in the light of the demand for more flexible position descriptions and rising expectations of employees. However, expanding the competence approach by measures of social network analysis (SNA) to describe the positions of individual actors and groups as well as their relations (ties) to one another within a certain network can contribute to extenuate the bottleneck effects. In the course of this thesis it will be necessary to give an overview of components which SNA can reveal in relation to learning assessment practises and how they can be interpreted in a meaningful way considering the organizational context. Another perspective SNA offers is the intent to represent actual networks independently of pre-defined competences. As a result, categories such as formal/informal learning could be partially merged by evaluating employees’ networks as a whole. Generating explicit competences can promote misleading dichotomies, such as for example body/mind (is there a strict distinction between cognitive, social and functional competence?) and formal/informal (how does this approach embrace and promote spontaneous learning activities outside of formal schedules for learning activities?). If knowledge and learning is said to be embodied in the contextual environment of the actors in a network, using competence-based assessment approaches might be partly inconsistent.

In my research proposal, I suggest a shift in assessment approaches by exploring SNA as a method of the emerging field of learning analytics. As for the exact measures, I still have to decide on a network principle I would like explore. For this, I will need some more extended literature review on previous research efforts. As for now, I stick to the measure of centrality, being the most important tool for analysing the importance of social network actors (Carrington, Scott, & Wasserman, 2005). All in all, I claim that SNA provides potential to reveal, measure and evaluate employees’ learning networks and their learning experience based on diverse data sources readily available. In a competitive economic environment, advantages will lie within making use of data and having power over developing suitable algorithms for the good of the organization. Social networks represent knowledge to a certain extend and thus should be considered as assessment approach for employee suitability and the value of knowledge in organizational HR strategies.

The research question for my proposal is formulated by

Can centrality measures of social network analysis (SNA) complement competence-based assessment approaches in workplace environments to support the implementation of HR policies based on socio-cultural/embodied learning theories?

Measures of centrality in SNA have been discussed widely but can be narrowed down to the three most important concepts degree, betweenness and closeness (Freeman, 1978). Competency based assessment approaches refer to defining sets of competence as a basis for assessing concepts such as employee performance and potential. HR policies are seen as representations of the strategic perspective on all human resource related topics within a certain organization. Socio- cultural/embodied learning theories represent the branch of learning theories within which learning and knowledge is seen as inseparable from its context and the social networks in which it occurs.

As for prior research it becomes clear that the focus lies on formal school settings and includes mostly students, indicating a gap of researching learning in organizations in general. In addition, most research has been conducted in online environments, leaving aside the potential to use centrality measures for offline social networks as well. In relation to organizational educational settings, it has been proposed to focus more on uncovering the holistic (authentic) professional learning experience than the evaluation of specific learning programmes (Webster-Wright, 2009).

Dawson (2008) used social network analysis to determine the relation between a student’s position in the social network and their reported sense of community. She concluded, that the centrality measures closeness and degrees are positively correlated to the reported sense of community, while betweenness show the opposite correlation. In addition, pre-existing external social networks influence the sense of community experienced to a certain extend. Kovanović, Joksimović, Gašević, and Hatala (2014) investigated the concept of social presence and it’s positive correlation to network centrality measures by taking data from student discussion boards. The relationship between digital learning and network learning has been detected by the study of Ünlüsoy, De Haan, Leander, and Volker (2013). Levels of activism and their correlation to occupational ties for movement participants have been investigated by Tindall, Cormier, and Diani (2012). All of these studies can be valuable starting points for my research proposal to identify more explicit concepts.

The objectives of this study are to evaluate two perspectives on learning in current HR research, namely workforce development and continuous professional learning. Furthermore, it intends to identify the importance of terms such as competence, knowledge, learning and assessment in HR policies and demonstrate the limitations of competence-based assessment approaches from a socio-cultural stance on learning. By illustrating the potential of SNA, it depicts a contributor to an assessment approach which can reveal authentic learning experience in the workplace. To illustrate the potential of SNA, this research investigates the correlation between network position focusing on centrality measures and learning experience by analysing and comparing learner’s position in social networks within a formal online training course and in the internal social media platform in general.

The intended audience for this study are learning policy and strategy makers in workplace environments, particularly focusing on assessment practices and the notion of knowledge in organizational environments. The proposed study is particular useful as it takes a critical stance on competence management, assessment practices and their significance for future continuous professional learning strategies. Furthermore by intending to be even more specific in chosen terminology and methodology, it aims at closing the identified gaps in the literature. One important aspect that is still left open is the identification of specific concepts, which shall be researched. The way this proposal is formulated now is still too broad and bold in assumptions and to less focussed on actual underlying theories. Assessment is one concept which needs clarification, especially to support its importance in this proposal as a whole. The same applies for the importance of knowledge and the particular link to underlying theories such as knowledge management and management theories.

Literature review

Within HR policies wording and terminology are important factors. Rephrasing professional development by continuous professional learning could contribute to exploring the holistic learning experience of professionals. In addition to this shift in terminology to address authentic professional learning, Webster-Wright (2009) identifies a lack of research when it comes to exploring the latter holistic learning experience. Assessment is said to be a critical factor within the epistemology – pedagogy – assessment triangle and reflects assumptions about knowledge and learning. Methods of learning analytics shall be used only when these intersections are clearly understood and considered (Knight, Buckingham, & Littleton, 2014). By revealing social networks of employees, this approach could contribute to respond to future research suggestions investigating coherent models of learning within workplace environments which include formal and informal learning aspects (Kyndt & Baert, 2013).

The gap exists when it comes to relating measures of SNA to the value of an individual’s position within his/her social networks in an organizational context. This is because most studies using measures of SNA are mostly performed in educational settings and relate these measures to competency based assessment results.

Hence, there is a missing link between measures of SNA and their potential for economic network value estimation. Some research might guide directions, such as research into social networks and sense of community (Dawson, 2008) or social capital (Kovanović, Joksimović, Gašević, & Hatala, 2014). However, these studies are again connected to academic performance in formal education settings. Current research additionally attempts to create approaches where data is used to extract competencies and match them with the organizational recruiting requirements. But these approaches are based on defined competencies, an approach I am aiming to complement (or even substitute) by using measures of SNA and borrowing suitable terminology for this purpose. For the development of my proposal, it is necessary to identify those research papers using SNA to describe social networks for the sake of being networks and using appropriate complementary concepts to value them.

[…]

References

Carrington, P. J., Scott, J., Wasserman, S. (2005). Models and methods in social network analysis: Elektronisk resurs. Cambridge: Cambridge University Press.

Cole, M. S., Schaninger, W. S., & Harris, S. G. (2002). The workplace social exchange network: A multilevel, conceptual examination. Group & Organization Management, 27(1), 142-167. doi:10.1177/1059601102027001008.

Dawson, S. (2008). A study of the relationship between student social networks and sense of community. Educational Technology & Society, 11(3), 224–238.

Freeman, L. C. (1978). Centrality in social networks conceptual clarification. Social Networks, 1(3), 215-239. doi:10.1016/0378-8733(78)90021-7.

Gonczi, A. (2004). The New Professional and Vocational Education. In Foley (Ed.) The Dimensions of Adult Education (pp. 19-34). Open University Press.

Harris, R., Short, T. (2014). Workforce development: Perspectives and issues. Singapore: Springer Singapore.

Haythornthwaite, C. (1996). Social network analysis: An approach and technique for the study of information exchange. Library and Information Science Research, 18(4), 323-342. doi:10.1016/S0740-8188(96)90003-1.

Hogan, B., Carrasco, J. A., & Wellman, B. (2007). Visualizing personal networks: Working with participant-aided sociograms. Field Methods, 19(2), 116-144. doi:10.1177/1525822X06298589.

Kovanović, V., Joksimović, S., Gašević, D., Hatala, M. (2014), “What is the source of social capital? The association between social network position and social presence in communities of inquiry,” In Proceedings of 7th International Conference on Educational Data Mining – Workshops, London, UK.

Knight, S., Buckingham Shum, S. & Littleton, K. (2014). Epistemology, assessment, pedagogy: where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2) pp. 23–47.

Kyndt, E., & Baert, H. (2013). Antecedents of employees’ involvement in work-related learning: A systematic review. Review of Educational Research, 83(2), 273-313.

Le Deist, F. D., & Winterton, J. (2005). What is competence? Human Resource Development International, 8(1), 27-46. doi:10.1080/1367886042000338227.

Tan, E. (2014). Human capital theory: A holistic criticism. Review of Educational Research, 84(3), 411-445. doi:10.3102/0034654314532696.

Säljö, R. (2012). Literacy, digital literacy and epistemic practices: The co-evolution of hybrid minds and external memory systems. Nordic Journal of Digital Literacy, 7(1), 5-20.

Tindall, D. B., Cormier, J., & Diani, M. (2012). Network social capital as an outcome of social movement mobilization: Using the position generator as an indicator of social network diversity. Social Networks, 34(4), 387-395. doi:10.1016/j.socnet.2011.12.007.

Tubaro, P., Casilli, A. A., & Mounier, L. (2014). Eliciting personal network data in web surveys through participant-generated sociograms. Field Methods, 26(2), 107-125.

Ünlüsoy, A., De Haan, M., Leander, K., & Volker, B. (2013). Learning potential in youth’s online networks: A multilevel approach. Computers & Education, 68, 522.

Webster-Wright, A. (2009). Reframing professional development through understanding authentic professional learning. Review of Educational Research, 79(2), 702-739. doi:10.3102/0034654308330970.

Advertisements

EdTech now and then: Combining course contents

The below text has been handed in by me for the MITx course 11.132x „Design and Development of Educational Technology“. After some good results in the peer feedback and a very lifely discussion on the text with a friend I decided to publish it here as well, as I used some thoughts/input from my Master programme (especially for the conclusion).


Abstract

In this assignment I will first describe the blog as a current learning technology and the LÜK (Lernen-Üben-Kontrollieren, german abbreviation for Learning-Practising- Controlling) as an earlier learning technology before I secondly conduct a comparative analysis of both technologies. My descriptions are enriched with personal learning experiences with theses technologies. I conclude with emphasizing that both technologies represent only one of several examples for the shift in learning theories over time and their impacts on educational technology. Additionally a major outcome is the claim for the critical evaluation of Ed Tech and the interdependencies between education and technology.

 

PART 1: CURRENT TECHNOLOGY – Blog

Blog Post (http://pixabay.com/static/uploads/photo/2014/02/13/07/28/wordpress-265132_640.jpg)

A current educational technology is the Blog (or Weblog, as a contraction of the words “web” and “log”), which can be described as a page on the web that operates as an individual accessible journal. Whereas in the earlier days of blogging knowledge about setting-up a homepage was required, Blog platform provider (such as wordpress.com or tublr.com) nowadays enable each individual to create a Blog without previous knowledge [Blood, Rebecca. „Weblogs: A History and Perspective“, Rebecca’s Pocket. 07 September 2000. 18 September 2013.]. When using this technology, the user’s main goal is to produce a text about a topic of interest and make it available for feedback.

Based upon this definition are the assets and drawbacks of this technology. On the one hand a blog can be used as a personalized learning technology adapted to specific needs to keep track of the learning process. Thus the technology’s goal is to offer a platform for keeping a learning journal for feedback purposes and continuous improvement. In addition, it operates as a sharing platform that is open to individual adaptation. However, in this openness lie the drawbacks as well. The technology does not  lead to a specific goal (producing a text is a broad goal) but rather is intended to be open-ended. By doing so, it requires a deep reflection on what the blog should be used for and how it can be used efficiently. Moreover, basic knowledge in producing text, expressing ideas, looking for appropriate resources and constructive feedback is required.

I want to illustrate these insights with the help of describing my first blog (franzidoesblog.wordpress.com) to support my individual learning process. It was set-up when I started my Master’s studies in Information Technology & Learning at the University of Gothenburg in August 2014. Whereas producing a text helps me to express my thoughts and reflect on them, once starting a blog can also generate pressure concerning how often to create a post and what to write about. The blog is useful for me to keep track on my progress and as it is open for adaptation I can implement it the way it suits my purpose. It helps me to improve my writing and reflecting competencies. Nevertheless, without the appropriate target group and range, getting feedback can be a hard task.

PART 2: EARLIER TECHNOLOGY – LÜK

LÜK Kasten (http://www.lernundsprachtherapie.de/files/spiele6.jpg)

The LÜK (or LÜK-Kasten, www.luek.de) is a self-checking device for different developmental stages and ranges of subjects and topics (e.g. learning to read or mathematical calculations) [jayseducation.com]. It consists of up to 24 rectangle chips that can be placed in a flat box. Each chip is printed on both sides: one side with numbers, the other with coloured geometrical shapes. The inner life of the box is also printed with numbers (please refer to the above picture). Lately, it has been released as an App as well which follows the same operating principles (App release notice on luek.de [German only]). The intended primary audience are children from 2 to 13 years. It claims to be a learning system, that follows the latest research on learning. Even though I was working with an early version of LÜK, it is still following the same principles today.

The operating mode is divided into three steps. First, there are guiding books for a variety of contents and levels of expertise that accompany the LÜK. In it, one can find the tasks to work through. Say one is working through task 1 that reads “Add 2 and 2!”. The result “4” has to be calculated mentally and then looked for within the chips. Once the chip with number 4 has been found, it has to be placed on the space in the box printed with 1 (representing exercise 1). If task 2 reads “Add 5 and 2!” “7” has to be found and the chip has to be placed on the corresponding space in the box for exercise 2. One important fact is that the numbers face upwards so that the geometrical figures can not be seen after the chips have been placed. Second, the box is been flipped over when all tasks have been solved. Third, once the box has been flipped, the geometrical figures face upwards and reveal the geometrical figure that has to be checked against the solution printed in the guiding book.

The LÜK resembles a sort of teaching machine. The tasks are divided into several progressing steps and once all are solved an immediate feedback is available. Furthermore, each student works on his/her own pace and is thus guided by an “individual teaching assistant”. The assets of this technology lie in the individual learning pace and the transparent instructions. However, this can be seen as a drawback as well. The learning process is clearly structured and by this can’t be changed. Though the pace might be individual the process structure (the sequence of tasks) stays the same. In addition, the LÜK does offer a variety of topics but the principle stays the same, making it predictable and prone to monotony.

I used the LÜK-Kasten throughout primary school in the late 90s primarily for learning mathematical calculations. As far as I remember, most of the time I enjoyed it. We had to solve both – homework and in-class activities. However, sometimes we tended to switch to the geometrical figures directly and did a jigsaw puzzle instead of calculations by recreating the picture from the guiding book. The impact on learning was present, however seeing it from today’s perspective I doubt the effectiveness. Mostly because it was teaching to calculate without reflecting the task and you got used to the principle very quickly.

PART 3 – COMPARATIVE ANALYSIS

Comparing a Blog and the LÜK-Kasten is like comparing fire and water. Both are elements of the earth but besides this they do not really have anything in common. A blog and LÜK can be used to learn something but whereas the blog is giving a lot of freedom (blog-post as the only limiting factor), LÜK is providing the complete “learning process” (in quotation marks because I would not always call it learning process) including the content. Both approaches are thus very divergent.

To contrast the technologies, I want to try to make them more comparable by exploring instructions for the same developmental stage (I was using them at different levels of age) – let’s say at the age of 10 to learn a new language. Whereas following the instructions in a language class using blogs would lead to producing individual texts including research on it, LÜK would lead to following instructions to answer questions and producing a new geometrical figure. One recognises that LÜK itself is a very isolated and limited system compared to a blog. Each new content has to fit to the instructional idea of a guiding book and answering a set of up to 24 questions. These questions are limited in characters and context, there has to be one correct answer. Furthermore the content might change due to different guiding books but the geometrical shapes (the box itself) does not change. The blog does not give this strict guidance and thus needs the support by a teacher, that helps exploring and managing the technology for learning. But after this the blog offers a wider range of approaching a problem by creating an individual text that can be formated, enriched with pictures, videos and hyperlinks. Each post will be an individual artifact, that can be commented on by others and developed further. With each post the blog as an artifact itself is growing. This introduces a social aspect as well. Whereas LÜK is a rather isolated learning process, a blog lives from feedback and improvement.

When it comes to engagement and motivation it appears as if the blog per se would work better. However, I am critical towards this attitude as especially a younger target group can be scared of writing independently about a topic, publishing a text online and making it available for feedback. But I see the teacher in a powerful role to overcome these obstacles. It might also be a familiarization with traditional behavioristic approaches that result in a certain assumption about learning. By saying so, I mean that children (especially in math) learn from the beginning, that there is always a certain process leading to one correct answer. Finally, this leads to a sceptical attitude towards new approaches such as a blog because they can be so different from what children were used to before. It might be that with the development of more cognitivistic and situative/pragmatistic-sociohistoric approaches in learning practise [terms taken from Greeno, J. G., Collins, A. M. & Resnick, Lauren B., 1996, Cognition and learning.] this might change.

To conclude, a blog can be more thought-provoking, memorable and playful whereas the LÜK can be more structured and more motivating as it leads to direct feedback on the answers. Of course this is my personal view, it is highly likely that others would engage differently with these technologies. For example, one could enjoy the openness of a blog and using it for creative results. Another one could see LÜK as being forced into a learning process that does not suit his/her preferences. All in all, both technologies derive from different learning traditions and mirror the ideas of their time.

PART 4 – CONCLUSION

Can we enrich learning environments with both technologies? If so, how? And even more important, how do we evaluate these technologies?

What becomes more and more important to me than answering these questions is the ability to critically evaluate technologies we (want to) use. Because technology is more than (just) a tool – it is not neutral. Neither does it have a sedulous impact. If it does have impacts we have to be more specific in describing them to not fall into the category of technological determinism [Oliver, M. (2011), Technological determinism in educational technology research: some alternative ways of thinking about the relationship between learning and technology].

In my school days it was important to critically evaluate historical sources and judge their context, purpose and credibility. However, when we talk about technology we tend to be more superficial. But technology – as historical sources – always has been invented and programmed by someone to fulfil a specific need. They do not appear from nowhere.

My above described examples are only one of several examples for the shift in learning theories over time and their impact on educational technology. By arguing, that it has always to be education that is driving technology we are missing the important point I scratched upon above: even if education is giving the primary direction for the development of technologies, it is education that has to be evaluated critically. Because both fields are fields of active and ongoing research – so many technologies which have been hyped are now at the edge to nowhere, and still we are launching into new technologies without taking a deeper look on where they come from, how they work and which needs they are supposed to fulfil.