Thursday, 24 April 2014

A bit of #mobile phone history 1922 and onward

A couple of days ago I got hooked on the British Pathé news reels, and here are three wonderful mobile phone related bits of movie. As an adapt mobile learning enthusiast and researcher, I just got so enthusiastic about these bits of moving history. And when looking at them I think: the same ideas are in all of our minds, whether the technology enables them or not.

The first dates back to 1922 and is called Eve's wireless, and it seems this movie accounts the "Home-O-Fone", which is based on the crystal radio (ah memories of my childhood electronic play-set that offered me a crystal radio as an easy bit of DIY option). In that movie you see two women walking down the street and letting 'hubby' know they will be late. Using a fire hydrant for grounding, and an umbrella as an antenna... great! And what do they phone? A record operator! Or at least it seems that way. The lady who first seemed like a phone operator puts on a record and presto, music can be heard in the street.



A second one links the mobile phone (in this case a car phone, 1946) with what would now most likely be seen as a hotspot. Two Italian (Naples) women park the car alongside the street, plug in a socket and use the car phone. A bit like finding a café with wifi to look at your mail :-)
The movie is called Rome: Auto Telephone, but the description says Napoli, Italy.



A third movie dates from 1946, and is entitled: Madame will you walk and talk. In this movie you see a woman making use of a (hefty) mobile phone option, to keep her (lady)friends on top of what is on sales or not, or getting a knowledgeable man to get the car going again, and the last scene of this movie has a homely feel (ducking behind my chair for the remarks that will no doubt be voiced by my feminist friends).

#OER #MOOC online learning papers from Open CourseWare conference

The Open Praxis special issue from the Open CourseWare Conferenence (OCW) 2014 that took place in Ljubliana, Slovenia is just published. This issue features a selection of papers. I gladly include the list of papers at the end of this blogpost, but first starting with an article and research that I found particularly useful.

The most interesting for my research and online learning interests was the one from Cohen, Omollo and Malicke which was an article on "a Framework to Integrate Public, Dynamic Metrics Into an OER Platform". The authors look at the usefulness of opening up meaningful analytics coming from shared Open Educational Resources (OER). I find this of interest, as it is part of something I want to investigate in relationship to peer/expert knowledge creation, MOOC, mobile learning. This paper provides an overview of what the university of Michigan, USA has done, but it can easily be rolled out to other educational and training institutes or organizations, even - in my view - to an international modus operandi.

The authors concluded with:
Metric-sharing is currency in relationships between OER-publishing-platforms and the faculty, staff, and students of universities and businesses who create OER. This is not the only benefit, but it has been a clear one for us: we are giving a population we have identified as a primary audience something they want. Our experience and user research confirm this.
The technical architecture an organization uses to host and reference OER is tied closely to the organization’s ability to share detailed usage data for its OER. Our use of the hierarchical structure of a Drupal-based platform allows us to easily provide metrics for individual courses or resources.
Metrics on a small scale are interesting, especially to their creators. Metrics can inform qualitative investigation, but they do not answer “why” questions (e.g. “why does this course have so many downloads compared to this other?”). To find analytics, metrics are necessary, but not sufficient.
Due to the structure by which OERbit platform stores metadata, Open.Michigan can group OER and its associated metrics in various ways. This positions Open.Michigan to progress from dynamic metrics to dynamic analytics in the future.
Much like seeing nodding and note-taking when you speak in front of an auditorium, seeing evidence of views, downloads, or comments where your OER are published is validation, evidence that there is some likelihood your effort provides real value to others. Having that evidence allows many to justify the additional effort it may take to openly license educational materials to supervisors and administrators. Open.Michigan’s sharing of individual OER metrics sustains development of open resources and allows an open education initiative, such as ours, to build strong relationships with its surrounding community and thus support the development and sharing of OER on a larger scale.

The above article matches in nicely with the OER and MOOC related article authored by Fernández and Webster (University of Madrid) which described "From OCW to MOOC: deployment of OERs in a MOOC, the experience of Universidad Carlos III in Madrid"

The emergence of Massive Open Online Courses (MOOCs) is focusing all its attention on open education. There  is growing interest in creating MOOCs, which can be done by transferring OCW courses to MOOC format. However, a series of doubts arise regarding the pros and cons implied in this transformation. In this paper we discuss the conclusions derived from our experience at Universidad Carlos III de Madrid with a widely disseminated OCW course that was satisfactorily converted into a MOOC. This experience has allowed us to compare two different models of open education initially based on the same content. We also analyze the difficulties incurred in the transformation process and present strategies to successfully carry out this change.

Other articles featured in this Open Praxis issue:

Wikiwijs: An unexpected journey and the lessons learned towards OERPDF
Robert Schuwer, Karel Kreijns, Marjan Vermeulen91-102
Public Expenditure in Education in Latin America. Recommendations to Serve the Purposes of the Paris Open Educational Resources DeclarationPDF
Amalia Toledo Hernández, Carolina Botero, Luisa Guzmán103-113
The potential social, economic and environmental benefits of MOOCs: operational and historical comparisons with a massive ‘closed online’ coursePDF
Andy Lane, Sally Caird, Martin Weller115-123
Formalising informal learning: Assessment and accreditation challenges within disaggregated systemsPDF
Rory McGreal, Dianne Conrad, Angela Murphy, Gabi Witthaus, Wayne Mackintosh125-133
Scenarios for the Use of OpenCourseWare in the Context of Student MobilityPDF
Frederik Truyen, Stephanie Verbeken135-144
From OCW to MOOC: Deployment of OERs in a Massive Open Online Course. The Experience of Universidad Carlos III de Madrid (UC3M)PDF
José Vida Fernández, Susan Webster145-158
Crowd-sourcing (semantically) Structured Multilingual Educational Content (CoSMEC)PDF
Darya Tarasowa, Sören Auer, Ali Khalili, Jörg Unbehauen159-170
An Architecture based on Linked Data technologies for the Integration and reuse of OER in MOOCs ContextPDF
Nelson Piedra, Janneth Alexandra Chicaiza, Jorge López, Edmundo Tovar171-187
A Framework to Integrate Public, Dynamic Metrics Into an OER PlatformPDF
Jaclyn Zetta Cohen, Kathleen Ludewig Omollo, Dave Malicke189-197

Wednesday, 23 April 2014

History from British Pathé movies a treasure trove!

As an avid movie lover and ex-movie critique, I was jumping into the air when I saw the British Pathé journals are opened (while I was on holiday, how dare they!), and are now freely available on their YouTube channel. There is an easy option to subscribe to specific British Pathé channels as well (sportinghistory, vintageFashion). Hopefully innovation will get its own channel soon as well, but fortunately it already has its own collection of movies organized under fantastic innovations.

The movie archive is a treasure trove for any teacher/trainer/history lover. First thing I did was get hold of the copyright on these movies. Because I wondered whether all these movies could be seen as Open Resources, potentially Open Educational Resources (OER). And whether any teacher could use the movies to rehash and make it a rich fit for their own training or teaching purposes. On their Pathé website the copyright is fully reserved, and while looking for YouTube licenses, I do have the impression the standard license is the one used. So it might not be that easy to use footage, or mix it into content that fits your learning goals. I do wonder how much footage can be used for educational purposes (should mail and ask).

Although this is clearly a great set that gives all of us a look into world history, it does come with its own zeitgeist, and its own gatekeepers eye. But then history is always an account of those who have the technology, the knowledge and the vision. The fact that the archive is set free, is like wandering into a movie library ... marvelous, I found 'unused, unissued material' as an anchor point to find fabulous raw movie stuff.
For one: suffragette material: women workers (factory girls) roaming the streets of Camden while being on strike, followed by a 66.000 rich female pageant through London, and of course Emily Davison who jumped onto a horse track and got killed in an attempt to pull down the Kings Horse in 1913, and more (e.g. Taxation without Representation is Tyranny".



And of course a bit of old Belgium, Gallant Little Belgium

Friday, 11 April 2014

#Diabetes, Sue Townsend, Easter a personal mix

This morning sad news was announced as the author Sue Townsend died (well known for her Adrian Mole books, but all round prolific and wonderful writer). Two days ago I finished her book “The woman who went to bed for a year”, a book that compelled me to laugh out loud, reflect upon the spirits of people, and of whom at the end cares the most for each one of us. I really enjoyed the book. As I read the back end of the book two facts sparked an extra, personal interest: blind, kidney transplant. Linking those two together I immediately thought of diabetes complications. After a quick search I found Sue Townsend was just like me a diabetes type 1 person since the 80’s. After reading it, I tucked my feelings on these two facts away to the back of my head.
This morning those feelings stormed to the front again as I read that Sue Townsend was dead. That fact scared the hell out of me. I saw the upcoming years: loosing eye sight, reduced – stopped - kidney function, stroke. I am scared of endings, I am scared of death, scared for me, for my loved ones, for the unknown, for the reducing of my own autonomy.
A letter came a month ago, that I needed to get my eye screening appointment set, I had ignored it. Why? Because I am scared of my own disease at times, and I have the tendency to flee from personal topics of anxiety. I know fleeing never helps on such occasions, but it is clinging on to the utopia of life and how I imagine it: ideal, healthy.
So, an hour ago I got up, picked up the phone and made the dreaded appointment. I need to stay in optimal condition for as long as possible, I owe it to my mum, my son, my partner and myself.
We must all live our own life, with whatever life has in store for us. Being scared only helps in extreme situations where caution is needed, in all other occasions we must face fear, and tackle it by either learning on how to overcome it, or reflecting on what the real basis of this fear is. It also means one must understand and look for that which makes us tick. For me understanding the learning process, looking for solutions that involve technology is a pleasure. Research, talking, thinking. To me learning, either face-to-face, online, any type of blended, is necessary to direct all of us towards trust, life, joining hands. Learning from people we like, support us, whom we support. Professionally I am on a good track.
Personally it is about connecting to people I care about. Partners, friends, colleagues (virtual and IRL), sharing, caring, supporting, dreaming.

Easter is coming, the flowers are out, spring is shining, and at this point in time I live, I love and I am thankful for all the wonderful people and opportunities that surround me. Life is in the living. 

Thursday, 10 April 2014

Complacency, failure, improvement cycle and #pearltrees #pkm14

For what ever reason, I seem to have a personal complacency => failure => improvement cycle. Which means that every few years something that I was good at turns into mush.

Messed up more then one presentation
The latest one concerns presentation skills. So I have been good at it (why do I know: feedback forms, mouth to mouth) and then it turns bad (why do I know? Again feedback forms). I did feel myself slip, but I simply told myself 'I had a day off' and soothed me into not worrying. So what is the typical decay of my presentation skills: I know what I know, I actually know quite a bit about certain topics (mlearning, cMOOC), but then I want to share ALL that I know in one hour slot of presentation AND I rely on my brain to come up with structure ad hoc. This does not happen.
There are multiple reasons, as every and any teacher/trainer will know:

  • pushing too much information forward to the public simply does not transmit the message
  • if you are not a naturally structured person, ad-lib will result in chaos and rambling
  • use simple slides for presenting, and use notes to elaborate on your slides, or add audio so people understand the pictures you use (this means: I always think that slides can be used as content booklets (see my slides for yourself)... but I think I will just need to step away from that. Either present, or offer booklets I guess). As Marshall McLuhan said: the medium is the message , and I scrambled both up resulting in a confused feeling in the heads of the listeners.
  • a starting point and relations or concepts that are obvious to me, is not obvious for others
  • practice, practice, practice to become really good, and aim for the moon while I am at it
  • there was even someone in the audience figuring out if I could be one of their future presenters... well I scared them away big time *sigh*
Actions taken
I want to make sure I do not get trapped in my own world of greatness again (apparently I come from a city known for its citizens to think they are great (Antwerp, Belgium). Those citizens even get called 'señor/a' as a nickname to describe that complacent ego phenomenon. Anyway, I searched for a way to improve:
  • get some pointers on how to present, i.e. expert knowledge
  • corner a future presentation occasion: May 2014 an online forum (which will be on related subject as the presentation I messed up, and the forum also has feedback forms) 
  • practice, practice, practice and get better. Why: I want to give pleasurable insights on topic into people's minds, not chaos!
Because of the personal knowledge management course I am following and its assignments. One of our current assignment is testing out new tools. In the past I have used content mapping as a way to organize and built content and information towards new knowledge. I used CMAP from the University of Florida, USA. But it missed some of the easy social media sharing options (but does have meaningful relational descriptions as an option between to sets of content). As such I strolled through Jane Hart's eLearning tools and I found pearltrees as a new tool to put together content in a jiffy, make it visual and retrievable and share it with others. So sharing my pearltree here ( a just started Coursera course, examples of great public speakers, good books, and presentation tips):

Presenting skills and books / Great public speaker examples in Inge Ignatia de Waard (ignatia_dw)

Collect what you like from your Android devices thanks to the Pearltrees' app

Wednesday, 9 April 2014

Proceedings from recent #MOOC conference

Just a quick post linking to a set of conference proceedings worth reading. eMOOC2014 proceedings of the research track (European MOOC summit in Switzerland, February 2014) which are fully online here. There is a link to a co-authored paper (written by Inge de Waard, Michael Sean Gallagher, Ronda Zelezny-Green, Laura Czerniewicz, Stephen Downes, Agnes Kukulska-Hulme and
Julie Willemson ) Vulnerable Learners in MOOC which I linked to earlier.

But the proceedings link to some magnificent global MOOC cases and experiences and lots of papers on learner drop-out and retention. I just started to read them all through, and the following were already of interest to me (related to my research interest):
Analyzing completion rates in the First French xMOOC by Matthieu Cisel (p. 26)
Scaffolding Self-learning in MOOCs (p. 43) written by Israel Gutiérrez-Rojas, Carlos Alario-Hoyos, Mar Pérez-Sanagustín, Derick Leony, Carlos Delgado-Kloos
MOOC Learning in Spontaneous Study Groups: Does Synchronously Watching Videos Make a Difference? (p. 88) by Nan Li, Himanshu Verma, Afroditi Skevi, Guillaume Zufferey and Pierre Dillenbourg
Signals of Success and Self-directed Learning (p. 18) by Penny Bentley, Helen Crump, Paige Cuffe, Iwona Gniadek, Briar Jamieson, Sheila MacNeill and Yishay Mor




Filtering for Future ProFessional Frontiers #pkm14

As the Personal Knowledge Management (PKM14) course moves into its second week, all the participants are asked to filter their social media / their networks. We are suggested to use more advanced filters: e.g. using feeds from people or/and groups, using automated filters of choice (e.g. hootsuite or tweetdeck to filter the personal twitter and other streams).

First I took a look at hootsuite (suggested by Ronda Zelezny-Green) and tweetdeck (both of these are free to some extend. Another paid option is sproutsocial which has wonderful options, but fits more with an enterprise type of social media stream analysis. I tested both and looked at other user comparisons to get an idea of which tool would suit me. I had both used them briefly in the past, but not to their full potential. And as it has been some years now, it is clear that I am better at understanding what these types of tools can do, and the overall structure of the tools in themselves is an improved user experience as well. For me Hootsuite works (but without integrating it into my browser, that feature was the reason why I stopped using hootsuite in the past: too much bling makes me angry and puts me off a tool - but that is me, not the tool).

After only two days, I already found some immediately relevant information (e.g. more status updates on learning analytics for informal learning, more about weak/strong ties in online communities), but at the same time I am loosing more time as I got lost in an information loop. Again... knowledge management is about finding useful tools, optimizing or personalizing them to fit your own goals, and limiting your time on those tools to get the best experience out of them (for me the best experience is: activating a peer network. One immediate benefit of Hootsuite is being able to schedule tweets, this saves time and will - eventually - keep me from returning to my streams in Hootsuite until a moment in time I consciously choose and limit.

The information streams in Hootsuite are currently based on one list, and keywords (e.g. mLearning, learning analytics). I started to build MyKeyPeople list within twitter: adding those twitterers that are of importance to me, that provide new insights, links, ideas.

But I do realize this is person-based, now - like a fellow participant Kavi ( ) mentioned - knowledge can be distilled from a higher level by using twitter lists and combining these, or to search for good curators/curations and link these to a RSS/feedread or scoopit... I still have to work on those options. So that makes a good to do. 
Speaking of to do lists, another participant of the course (Shane Johnson) mentioned todoist, a cross-platform software to plan your time or project. Will have to see if this works for me. 

Sunday, 6 April 2014

8 years of Working Out Loud thoughts #PKM

As the course on Personal Knowledge Management (PKM) is well on its way, one of the steps to reflect on and improve the PKM is about making a summary of Working Out Loud (or set out actions if this is the first time to be working out loud = sharing what you do through your own social media in order to build an active network that supports you and which you in turn can support).

I have been Working Out Loud for the last 8 years, and my main knowledge areas have been eLearning options as stand-alone options as well as partnerships with other institutes, mobile learning, continued medical education tools for training, and mobile learning solutions for developing regions. This took me on a very steep learning curve. Especially as mobile learning in developing regions was quite a new feature when I started on that track (and top management was not convinced mobile learning would be a good training option - first projects were paid out of my own pocket).

Why did I work in the open? 

  • It allowed me to connect with the few peers that were out there, somewhere on the globe
  • to share what I learned with others who are contemplating to roll out similar projects
  • keeping track of what I did learn, and how I solved certain challenges (a personal learning archive)
  • building a network that I could consult and feed back into. 
  • It is part of professional activism: sharing consciously to plea for open science, open commons, openness overall


Looking back at those last 8 years, how can I make improvements? Which actions will I take?

  • Challenges and failures should be shared more frequently. It does not feel good for the ego, and in some cultures failure just is not discussed, but failure is part of the essence of failure. 
  • Increase curation (synthesizing and disseminating what other peers do). I need to reconnect with my 'top notch 50' peers. I have been reducing my 'reading what others do' in the last few months and this resulted in more of an isolated feeling. The reason why I started to link less to others was due to time management challenges, a professional identity shift (from corporate solution to academic research), and loosing track on my own personal knowledge management overall. 

Friday, 4 April 2014

Part4 #xAPI seminar Andrew Downes on #learningdesign #eln

Almost groggy here after information overload and process... hoping still to make sense a bit. Great seminar!
Part4 #xAPI seminar Andrew Downes on #learningdesign. Wonderful resource: http:// http://tincanapi.co.uk/

And if you want to get an xAPI UK project going, have a look here, at the end there is link to get in touch and start a conversation

If you want to start designing xAPI statements, here is that start:

And a few pointers on how to start tracking xAPI and real world

And an iPhone xAPI statements viewer (free) here:
https://itunes.apple.com/gb/app/experience-api-xapi-statement/id550133878?mt=8

Looking at how Tin Can design affects learning design
He is key UK lead for xAPI at Epic


A xAPI mindset
The xAPI looks at events, not necessarily status of events (latter typical of scorm)
Which results in different learning designs.

I
Did
This
A learner
Succeeded at
A work task


Some elearning


Their personal goal
Computer
completed
me


Think about
What different experiences do or could make up your blend of learning?
What needs to happen in experiences X to trigger a change in experience Y
What is a natural flow for your learners?


Challenges on informal xAPI stuff
Reliability of self-reporting
I did this => prove it!
Will learners report their learning?
Please complete this form => possibly NO (so what can we do to avoid this, or work around. What might be a benefit?) – e.g. training courses only made available to those who do send feedback.
Privacy concerns
We are tracking everything you do => Ummmmm (big brother issue, communicated well, open on it)
Interoperability

It’s not working, is it? => Have you tried turning it off and on again? (So communicating on this is crucial, both with learners, as with developers/experts)
Too much data
Joe moved this mouse 1 pixel => Which direction?
Correlation is not causation. 

Part3 #xAPI and #LRS #learningAnalytics #eln

Part3 Ben Betts from HT2 on Learning Record Store and #xAPI of the seminar, part 1 including general links and frameworks can be seen here, part 2 focusing more on tools can be found here

xAPI (official name) and Tin Can are two names but covering the same thing, bit different ownership.

Ben looks at the anatomy of a statement (showing it IRL).

Learning Record Store: what it is, what it can do
Format is a triplet of data: actor – verb (English) – object (what it is you are doing: name, activity, …)
JSON is the language – light weight xml, each bit identifies

Important stuff:
 each verb used needs to be VERY WELL DEFINED, at this point in time you need to be VERY standardized. E.g. enter : this verb can cover multiple options, so definition is key to make the LRS meaningful.

Example using Curator platform: everytime I interact with the platform an xAPI data will be tracked

Also shows an example of xAPI integration in Google Chrome, which when clicked in that box of Google Chrome is immediately fed back as a statement to xAPI.

More on LRS
LRS is a potential key infrastructure and an LRS is a STANDARD
I was bemoaning social learning, e.g. MOOC learning, how much is done by the learner, yet is lost both for the learner and for the learning system.
Learners should own their learning data. Enabling a learning journey. Owning the JSON, the raw data.  This is what triggered the xAPI in Ben
Data gathering must be such that meaning can be taken out by other systems.

Learning locker
Ben shows his social learning linked to xAPI statements (GREAT!!!!)
Validation of actions is made much easier when observation is put into the equation: if someone else has seen you doing an activity and endorses it, you can believe the action actually took place.

But when Ben was looking at the first data, he saw that the data was dull in many cases
So a couple of experiments was done: one was on giving the learners some data to build upon (like a customer card, filled from the start with 3 stamps => more motivating!).
Combining all learning in the LRS: quantified self, LMS, surfing, mobile apps…
It is not difficult to come up with a LRS, but doing this is not as easy as thinking of it

Challenges of the ecosystem
  • Data carrying challenge: it must be awesomely standardized, otherwise analytic tools will not work (coding/decoding challenge)
  • But some systems you want to keep (e.g. Oracle, statistics tools)
  • Ben uses it to power Open Badges from Mozilla (great for informal mooc learning), xAPI provide an underlying layer for a badge: if you do this, this and this…. You get a badge.
  • And a key attribute of LRS is, that you MUST be able to put it in another LRS (think SCORM). So buying LRS needs to be critical.
  • Because we know so little for the moment, that a lot of the data will be unuseful due to all the changes anyway


Working on for the moment
  • Blending learning platforms – MOOCs (cross platform) (Ask Ben whether he is been contacted for FutureLearn - answer: no known plans)
  • Customising eBooks (is done in the OU)
  • Effective Performance Support – improving learning design
  • Issuing Open Badges
  • Predictive Analysis – are you going to fail? (mentioning OU)
  • Personal Learning Records
  • Personalising Learning Experiences

Ben shows admin dashboard
Because of the wide variety of data, there are a lot of xAPI statements (e.g. used for training engineers: comparing starting engineer learners statements with expert engineer statements – 60.000 statements per day)

Reports can be pulled from the LRS
This can be exported to excel

Why open source
  • LRS is a standard: difficult to differentiate from a tech perspective
  • Chance to shape a key piece of tech for our industry – xAPI needs this so people can experiment
  • OS gives us more marketing opportunities and vastly improved network
  • Personal sense of purpose
  • New revenue stream

Designing for data
85% of organisations will not be able to exploit big data for competitive advantage through 2015 (Gartner, 2013)

Consider 3 big needs
  • Design for analytics
  • Adopt standards
  • Consider the data supply chain – show does data flow through your organization?
See: http://www.accenture.com/microsites/it-technology-trends-2014/Pages/data-supply-chain.aspx

The personal LRS
Based on the core code of the organization LRS, allowing
  • Individual ownership of data (but more focus on quantified self on the front-end)
  • Presentations of experience
  • Customisation of future learning experiences (API)

Possible uses
  • Google circles: using data for putting information to different people
  • This might provide the flipped solicitation room, as the facts of what you know can be seen pre-interview, real action during interview.

Strategic aims
We will become the open source standard for LRS: the de-facto standard for LRS. To achieve this aim and fulfill our vision we have adopted three strategic aims
Develop an enterprise-ready LRS
… did not get all three

Get involved
Next week a new version of the LRS is rolled out for use (? huh, keep an eye out!)
There is also a cloud version (so no set-up needed, Inge ask for this)

On-going projects
  • Tin badges (open cans) – look up Bryan Mathers, 2014 – tin can and open badges
  • Moodle madness
  • Content without borders

Questions for ben
Does he work with FutureLearn
Can xAPI be implemented in smaller systems, e.g. Wordpress?
Where can I find the cloud version of LRS? It will be on license 125 pounds (seems still need to be rolled out)

Some answers:
Grassblade – wordpress – look up (thank you David Glow!)
Design cohorts in ADL as a resource to find what people are working on, projects (http://ymlp.com/zan0J7)

On feasibility of building a research instrument to capture informal learning:
It is possible, but you need to plan enormously up front: REALLY know what you want to track, and how you call it, and how it must be put in.
The instrument must use VERY TIGHT TAG architecture, really defining the meta-tags and the way the people must do it, in order to be able to analyse it.
NO open statements, otherwise it becomes a nightmare to analyse => different ID for each statements

(Inge thinking: you could use an instrument for quantified, then compare it to written learning diaries)