UTArlingtonX: LINK5.10x Data, Analytics, and Learning or #DALMOOC (Week 1)

So far, the #DALMOOC is one of the most complex online courses I have enrolled. Contentwise it covers “an introduction to the logic and methods of analysis of data to improve teaching and learning”. What is especially challenging is the course structure and the social tools involved.

In this post, I will first describe the course structure and state key messages from week 1’s content (Assignment: “share your reflections on week one in terms of a) content presented, and b) course design”). After this, I will present my bullet points for the four readings as a completion of the key messages (Assignment: “review the additional readings available for week 1 of the course and share your reflections about them”). Finally, I will attach my edited Learning Analytics Tool Matrix (Assigment: Learning Analytics: Tool Matrix) where I conducted some research on learning analytic tools.

Course structure and key messages

What is making the course so complex is the high amount of social tools and pathways to chose from. The basic idea is that there is a Guided Learner Path (“blue pill”) and a Social Learning Path (“red pill”) available. Either, one can chose one of those or get involved in both. To keep it simple, the blue pill is the structure learners are most familiar with: course content is provided as in a typical classroom environment where the teacher is providing the knowledge. The red pill however, is a social approach where learners interact via social media (e.g. Prosolo, Twitter) and share their artifacts.

Based on this structure a range of tools is in use to track the learning progress. Generally speaking, edx provides solely the platform for the course content. Interaction is recorded via Prosolo (a platform connected to edx, to show learning goals and competencies, share thoughts and form groups, fulfill assigments). For example, this blog post will be recorded (or tracked) in Prosolo and thus can be made available for peer assessment. In addition, there are features at which enable the user to track #dalmooc hashtags on Twitter or RSS feeds.

When talking about Learning Analytics, there usually are tools involved that apply the theoretical knowledge. In this course, we will deal with Tableau, Gephi, rapidminer and LightSide. An additional problembank is provided for advanced assignments to work with these tools.

The social learning aspect is supported by a tool called bazaar (Bazaar assignment: Discuss Week 1). Bazaar is a plattform (basically a chat system) that connects learners on demand to discuss course related topics and contents. In my case I was connected on Saturday evening to a very helpful person from India. There is a programmed digital instructor that guided us through the discussion. After an introduction we were to discuss why we take this course, how we define learning analytics, how useful we found the used cluster for learning analytic tools and how it could have been improved. We had a very constructive discussion that benefited from the fact that we had different backgrounds and levels of expertise.

My key messages for this week are

• User always expect usability, especially in online courses. But talking about Learning Analytics means talking about a broad range of data that demands skills to graps these data and make sense of it. For me, the course itself offers an opportunity to find one’s individual way through the vast amount of learning opportunities to engage with the topic of data analytics.
• The field of Learning Analytics offers methods to analyse learners’ behavior in a learning environment and by this providing groundwork for the improvement of learning environments and individual learner’s feedback.
• Analytic Tools are programmed by others and to understand the way they work it is important to be familiar to the methods in use and how they are applied within such tools.

Key messages enriched by reading contents

Usability and complexity of data

[Halevy, A., Norvig, P., & Pereira, F. (2009). The unreasonable effectiveness of data. Intelligent Systems, IEEE, 24(2), 8-12.]

• Don’t wait for (impossible) data collections but combine the already existing data more effectively
• A small set of general rules per se is not better than a large set of applicable data (e.g. for learning a language, it can be easier to have a number of examples memorized than knowing the general rule)
• The use of n-gram models and the “false dichotomy” of natural language processing: deep (hand-coded) approach & statistical approach (“learning n-gram statistics from large corpora”)
• Semantic Web (machines understand semantic documents not human speech/writing) vs. Semantic Interpretation
• The tasks that are left are not indexing but interpreting data/information/language -> using the vast amount of information on the internet to support the interpretation problem -> don’t try to make language “easier” by forming general rules but by making use of the language in use that is available

[Tansley, S., & Tolle, K. M. (Eds.). (2009). The fourth paradigm: data-intensive scientific discovery.]

• Opportunities and challenges of the fourth paradigm of science based on data-intensive computing
• Accessability and “the cloud” as a base for data-intensive science, three basic activities capture, curation and analysis
• Permanent archiving of data as the main goal to improve scientific research
• eScience as where IT meets science, a new paradigm of science, need for improving the tools for data capturing, analysis and visualization, science happens online (Jim Gray on eScience: a transformed scientific method)
• Four areas of application: (1) Earth and Environment, (2) Health and Wellbeing, (3) Scientific Infrastructure, (4) Scholarly Communication and (5) Final Thoughts

The field of Learning Analytics

[Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. JEDM-Journal of Educational Data Mining, 1(1), 3-17.]

• Different terms, e.g. Data Analytics for Learning, Learning Analytics, Educational Data Mining (EDM) & Knowledge Discovery in Databases (KDD)
• Making sense of data in learning environments by discovering effective methods to interpret them, thus providing imediate feedback to improve students performance and course quality
• Methods and Key Applications
  • Improvement of student models (how students act within a learning environment and how this environment can respond)
  • Discovering/ improvement of domain’s knowledge structure (data can be used by automated approaches to discover accurate domain structure models)
  • Studying pedagogical support and determine relative effectiveness
  • Supporting research on educational theories / phenomena by delivering empircal evidence
• Important trends
  • application for online-courses, sensitive and effective e-learning, new areas of study: gaming the system, tools for datamining, student modeling, from relationship mining to prediction, discovery with models
• Provided data become more public: e.g. through online course environments, broader application possible and check easier

[Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. Cambridge Handbook of the Learning Sciences.]

• Learning Analytics (LA) & Educational Data Mining (EDM) to conduct research that benefits the learner and the research community, guided by theories from learning science and education, data mining and analytics &  psychometrics and educational measurement as main sources
• EDM: (1) automated methods (prediction), (2) specific constructs and their relationship, theoretical approaches, (3) application in automated adaption
• LA: (1) human-led methods (understanding), (2) understanding the system of the constructs, theories to understand systems as a whole / that take situationalist approaches, (3) inform and empower learner & instructor
• Growing field because of
  • increasing data quantity (public archives and open online courses), improved data formats (standardized formats for data logging), advances in computing, increased sophistication in tools available (Map Reduce, Apache Hadoop)
• Methods
  • prediction methods: as in Baker/Yacef still most prominent, to infer predicted variable from predictor variables, three types
    • classifiers (predicted variable binary or categorical)
    • regressors (predicted variable continuous)
    • latent knowledge estimation (as a special typ of a classifier))
  • structure discovery: as oposite to prediction, because no priori idea of a predicted variable, 4 common approaches
    • clustering: find data points that naturally group together, most useful when cluster are known in advance
    • factor analysis: closely related to clustering, find clusters and split variable set in latent factor set (not directly observable)
    • social network analysis: reveal structure of interaction by analysing the relationship between individual actors
    • domain structure discovery: finding knowledge structure in educational environment
  • relationship mining: discover unexpected but meaningful relationships between items of a large data set
    • association rule mining (find if-then rules for a data set)
    • correlation mining (find positive/negative correlations between variables)
    • sequential pattern mining (find temporal associations between events)
    • causal data mining (find cause for event or observed construct)
  • distillation of data for human jugdment: analyse data for immediate feedback of research/practitioners (e.g. through heat maps, learning curves and learnograms)
  • discovery with models: use the results of one data analysis within another data analysis (but also cluster analysis or knowledge engineering as input approaches)
• Tools
  • General purpose (e.g. RapidMiner, R, Weka, KEEL, SNAPP) vs. special purposes tools (e.g. DataShop)
  • Open source (e.g. R, Weka) vs. comercial tools (e.g. IBM Cognos, SAS, analytics offerings by Blackboard, Ellucian)
• Impact on Learning Sciences
  • Research on disengagement
  • Student learning in various collaborative settings
• Impact on Practise
  • impact of social dimensions of learning and the impact of learning environment design on subsequent learning success
  • networked learning systems vs. more centralized platforms (e.g. LMS)
• Outlook
  • Growing data sources
  • Expanding range of application: computer games, argumentation, computer-supported collaborative learning, learning in virtual worlds & teacher learning

Learning Analytics Tool Matrix

By clicking on the above headline, my adapted tool matrix can be accessed. It is my point of departure, as I am still working on it. I want to specify the tools I added (printed in Italic), visualize the different phases the tools belong to and work on a better layout. Furthermore I want to add content from the course weeks still to come and some experiences when using the tools.

EdTech now and then: Combining course contents

The below text has been handed in by me for the MITx course 11.132x “Design and Development of Educational Technology”. After some good results in the peer feedback and a very lifely discussion on the text with a friend I decided to publish it here as well, as I used some thoughts/input from my Master programme (especially for the conclusion).

---

Abstract

In this assignment I will first describe the blog as a current learning technology and the LÜK (Lernen-Üben-Kontrollieren, german abbreviation for Learning-Practising- Controlling) as an earlier learning technology before I secondly conduct a comparative analysis of both technologies. My descriptions are enriched with personal learning experiences with theses technologies. I conclude with emphasizing that both technologies represent only one of several examples for the shift in learning theories over time and their impacts on educational technology. Additionally a major outcome is the claim for the critical evaluation of Ed Tech and the interdependencies between education and technology.

 

PART 1: CURRENT TECHNOLOGY – Blog

Blog Post (http://pixabay.com/static/uploads/photo/2014/02/13/07/28/wordpress-265132_640.jpg)

A current educational technology is the Blog (or Weblog, as a contraction of the words “web” and “log”), which can be described as a page on the web that operates as an individual accessible journal. Whereas in the earlier days of blogging knowledge about setting-up a homepage was required, Blog platform provider (such as wordpress.com or tublr.com) nowadays enable each individual to create a Blog without previous knowledge [Blood, Rebecca. “Weblogs: A History and Perspective”, Rebecca’s Pocket. 07 September 2000. 18 September 2013.]. When using this technology, the user’s main goal is to produce a text about a topic of interest and make it available for feedback.

Based upon this definition are the assets and drawbacks of this technology. On the one hand a blog can be used as a personalized learning technology adapted to specific needs to keep track of the learning process. Thus the technology’s goal is to offer a platform for keeping a learning journal for feedback purposes and continuous improvement. In addition, it operates as a sharing platform that is open to individual adaptation. However, in this openness lie the drawbacks as well. The technology does not  lead to a specific goal (producing a text is a broad goal) but rather is intended to be open-ended. By doing so, it requires a deep reflection on what the blog should be used for and how it can be used efficiently. Moreover, basic knowledge in producing text, expressing ideas, looking for appropriate resources and constructive feedback is required.

I want to illustrate these insights with the help of describing my first blog (franzidoesblog.wordpress.com) to support my individual learning process. It was set-up when I started my Master’s studies in Information Technology & Learning at the University of Gothenburg in August 2014. Whereas producing a text helps me to express my thoughts and reflect on them, once starting a blog can also generate pressure concerning how often to create a post and what to write about. The blog is useful for me to keep track on my progress and as it is open for adaptation I can implement it the way it suits my purpose. It helps me to improve my writing and reflecting competencies. Nevertheless, without the appropriate target group and range, getting feedback can be a hard task.

PART 2: EARLIER TECHNOLOGY – LÜK

LÜK Kasten (http://www.lernundsprachtherapie.de/files/spiele6.jpg)

The LÜK (or LÜK-Kasten, www.luek.de) is a self-checking device for different developmental stages and ranges of subjects and topics (e.g. learning to read or mathematical calculations) [jayseducation.com]. It consists of up to 24 rectangle chips that can be placed in a flat box. Each chip is printed on both sides: one side with numbers, the other with coloured geometrical shapes. The inner life of the box is also printed with numbers (please refer to the above picture). Lately, it has been released as an App as well which follows the same operating principles (App release notice on luek.de [German only]). The intended primary audience are children from 2 to 13 years. It claims to be a learning system, that follows the latest research on learning. Even though I was working with an early version of LÜK, it is still following the same principles today.

The operating mode is divided into three steps. First, there are guiding books for a variety of contents and levels of expertise that accompany the LÜK. In it, one can find the tasks to work through. Say one is working through task 1 that reads “Add 2 and 2!”. The result “4” has to be calculated mentally and then looked for within the chips. Once the chip with number 4 has been found, it has to be placed on the space in the box printed with 1 (representing exercise 1). If task 2 reads “Add 5 and 2!” “7” has to be found and the chip has to be placed on the corresponding space in the box for exercise 2. One important fact is that the numbers face upwards so that the geometrical figures can not be seen after the chips have been placed. Second, the box is been flipped over when all tasks have been solved. Third, once the box has been flipped, the geometrical figures face upwards and reveal the geometrical figure that has to be checked against the solution printed in the guiding book.

The LÜK resembles a sort of teaching machine. The tasks are divided into several progressing steps and once all are solved an immediate feedback is available. Furthermore, each student works on his/her own pace and is thus guided by an “individual teaching assistant”. The assets of this technology lie in the individual learning pace and the transparent instructions. However, this can be seen as a drawback as well. The learning process is clearly structured and by this can’t be changed. Though the pace might be individual the process structure (the sequence of tasks) stays the same. In addition, the LÜK does offer a variety of topics but the principle stays the same, making it predictable and prone to monotony.

I used the LÜK-Kasten throughout primary school in the late 90s primarily for learning mathematical calculations. As far as I remember, most of the time I enjoyed it. We had to solve both – homework and in-class activities. However, sometimes we tended to switch to the geometrical figures directly and did a jigsaw puzzle instead of calculations by recreating the picture from the guiding book. The impact on learning was present, however seeing it from today’s perspective I doubt the effectiveness. Mostly because it was teaching to calculate without reflecting the task and you got used to the principle very quickly.

PART 3 – COMPARATIVE ANALYSIS

Comparing a Blog and the LÜK-Kasten is like comparing fire and water. Both are elements of the earth but besides this they do not really have anything in common. A blog and LÜK can be used to learn something but whereas the blog is giving a lot of freedom (blog-post as the only limiting factor), LÜK is providing the complete “learning process” (in quotation marks because I would not always call it learning process) including the content. Both approaches are thus very divergent.

To contrast the technologies, I want to try to make them more comparable by exploring instructions for the same developmental stage (I was using them at different levels of age) – let’s say at the age of 10 to learn a new language. Whereas following the instructions in a language class using blogs would lead to producing individual texts including research on it, LÜK would lead to following instructions to answer questions and producing a new geometrical figure. One recognises that LÜK itself is a very isolated and limited system compared to a blog. Each new content has to fit to the instructional idea of a guiding book and answering a set of up to 24 questions. These questions are limited in characters and context, there has to be one correct answer. Furthermore the content might change due to different guiding books but the geometrical shapes (the box itself) does not change. The blog does not give this strict guidance and thus needs the support by a teacher, that helps exploring and managing the technology for learning. But after this the blog offers a wider range of approaching a problem by creating an individual text that can be formated, enriched with pictures, videos and hyperlinks. Each post will be an individual artifact, that can be commented on by others and developed further. With each post the blog as an artifact itself is growing. This introduces a social aspect as well. Whereas LÜK is a rather isolated learning process, a blog lives from feedback and improvement.

When it comes to engagement and motivation it appears as if the blog per se would work better. However, I am critical towards this attitude as especially a younger target group can be scared of writing independently about a topic, publishing a text online and making it available for feedback. But I see the teacher in a powerful role to overcome these obstacles. It might also be a familiarization with traditional behavioristic approaches that result in a certain assumption about learning. By saying so, I mean that children (especially in math) learn from the beginning, that there is always a certain process leading to one correct answer. Finally, this leads to a sceptical attitude towards new approaches such as a blog because they can be so different from what children were used to before. It might be that with the development of more cognitivistic and situative/pragmatistic-sociohistoric approaches in learning practise [terms taken from Greeno, J. G., Collins, A. M. & Resnick, Lauren B., 1996, Cognition and learning.] this might change.

To conclude, a blog can be more thought-provoking, memorable and playful whereas the LÜK can be more structured and more motivating as it leads to direct feedback on the answers. Of course this is my personal view, it is highly likely that others would engage differently with these technologies. For example, one could enjoy the openness of a blog and using it for creative results. Another one could see LÜK as being forced into a learning process that does not suit his/her preferences. All in all, both technologies derive from different learning traditions and mirror the ideas of their time.

PART 4 – CONCLUSION

Can we enrich learning environments with both technologies? If so, how? And even more important, how do we evaluate these technologies?

What becomes more and more important to me than answering these questions is the ability to critically evaluate technologies we (want to) use. Because technology is more than (just) a tool – it is not neutral. Neither does it have a sedulous impact. If it does have impacts we have to be more specific in describing them to not fall into the category of technological determinism [Oliver, M. (2011), Technological determinism in educational technology research: some alternative ways of thinking about the relationship between learning and technology].

In my school days it was important to critically evaluate historical sources and judge their context, purpose and credibility. However, when we talk about technology we tend to be more superficial. But technology – as historical sources – always has been invented and programmed by someone to fulfil a specific need. They do not appear from nowhere.

My above described examples are only one of several examples for the shift in learning theories over time and their impact on educational technology. By arguing, that it has always to be education that is driving technology we are missing the important point I scratched upon above: even if education is giving the primary direction for the development of technologies, it is education that has to be evaluated critically. Because both fields are fields of active and ongoing research – so many technologies which have been hyped are now at the edge to nowhere, and still we are launching into new technologies without taking a deeper look on where they come from, how they work and which needs they are supposed to fulfil.

“Midterm” review: What does work, what not.

After not having written a blog post for 14 days, I am becoming a little nervous: do I need to “produce” a certain number of posts per week, does the blog not work for me or do I not work as much as I was doing before? It might help to reflect on what I was working on during the last 14 days. I remember my lecturer saying, that writing down what you are working on actually can be a good way of visualizing it.

In terms of class activities the last few days were stuffed with preparations. Just now I realise how hard we were working on finalizing our seminar on B.F. Skinner’s “The Technology of Teaching” and a written assignment on an evaluation plan for an actual research project. We were working in the same group for both tasks, which was not always easy but in the end we managed to get great results with merging all our interesting input. What I really liked about both projects was a good time management and how we split up the tasks. Room for improvement on my side is a better reflection on requirements and individual workload as well as an even deeper analysis of literature.

Besides my own group work, the last days included two classic seminars from other groups on Simon’s “The Sciences of the Artificial” and Papert’s “Mindstorms” and several workshops and lectures. Especially the seminars were interesting as we worked on one selected topic as a whole group. Meaning that these days were prepared by a smaller group and they were to make the whole class work on it. A lot of discussions, a lot of resources, a lot of input – a lot to digest.

During the Wednesday group of academic writing we were working on a preliminary draft of our Master Thesis topic. That was not always easy, as you have to decide against so many options by deciding for another. In the end my draft includes three possible areas to look at:

• How Data-Analytics can improve both: workplace training performance and workplace training quality/effectiveness
• The definition of digital literacy and its implication for recruiting and personnel’s performance
• Training methods lacking “new” input concerning new media use, training & education in a new context

I got some valuable feedback on my thesis statements that I prepared for each area and will work with this to specify my intentions.

Besides the class input I am working with several online platforms to complement the Master courses and/or get a different perspective on what I am learning. These platforms are Coursera, Edx and futurelearn.

I was using Coursera to get a refresher on research methods for the project assignment (course details) and tried to develop my own digital story about Behaviorism (which has to be finalized but following the link you can have a look at my storyboard, course details), whereas currently I am exploring the future for education and take a deeper look at questionnaire design for Social Surveys. (Scheduled for this year are still E-Learning and Digital Cultures, e-learning Ecologies and Performance Assessment in the Virtual Classroom).

On the Edx platform I am starting with MITx: 11.132X Design and Development of Educational Technology (a course focusing on constructivism and cognitivism, it’s development within Edtech and implications for future educational technology), MITx: 11.126X Introduction to Game Design and UTArlingtonX LINK5.10X Data, Analytics, and Learning (“An introduction to the logic and methods of analysis of data to improve teaching and learning.”).

Recently I discovered the platform futurelearn which I want to try out by subscribing for the course Data to Insight: An Introduction to Data Analysis (University of Auckland).

I also took care of my application for the spring semester 2015. The deadline is October 15th and I found some courses that sound really interesting. My intention is to register for two courses in parallel to cover some credits for semester 3 already. This semester consists of 30 credits of choice and instead of taking them all in one semester I try to spread them over the preceding semester, so semester 3 would be available for some studies abroad.

What does work?

• Class activities: Online collaboration and storage of data, blogging about learning experiences, take some inputs and apply them to online courses and viseversa.
• Online Courses: After struggling beginnings it becomes easier for me to select appropriate courses and work through these parts that are important for my purpose.

What does not work?

• Readings: I want to read more resources in addition to the course resources related to my interests/Master thesis.
• The broader picture: A overall (technological) solution to display all resources, learning outcomes, etc. is not (yet) on-hand, rather I am trying to cover this by final assignments and/or transferring knowledge from one project to another.
• More content: I feel that I am lacking the coverage of actual content at this blog, mostly I am writing about vague own opinions than well-grounded essays. This is also what I want to improve in.

All in all my midterm review is quite positive. I think that there always comes a time where you have to push yourself a bit to continue the good work. I am confident, that what I am learning is what I am interested in. Just this morning I was reading the next book for Wednesday’s classics seminar about Turkle’s “The Second Self” – and I am devouring it.