Measuring the right things

I’ve just read a fascinating publication from Microsoft titled “Interoperability: Improving Education” which came about as a result of 10 or so educators and ICT practitioners who were brought together by Microsoft for a meeting running alongside the annual NAACE conference held in Blackpool, England, earlier this year. The brief was to talk about the way that schools use pupil data. And the wisdoms that ensued are contained in an new Microsoft discussion document for school leaders and local authorities, “Interoperability: Improving schools” (download the PDF here).

The contents of this paper provide timely insights for NZ educators because it nicely ties together two pieces of work that are currently the focus of the Ministry of Education. First are the discussions about standardised testing, and the measurement of student progress and achievement, and second are the issues of interoperability as they relate to work being done around data interoperbility between LMSs and SMSs in schools, and the whole area of e-portfolios.

From its title, and the fact that it’s published by a technology company, you could be forgiven for thinking that the document is about a technology solution that will end all our woes. This is not the case. Instead, the document contains a summary of thoughts in response to questions such as:

  • are we collecting the right data?
  • what data should we be collecting?
  • who needs, or wishes to see the data?
  • can we easily move data to where it’s wanted or needed?

The context for the discussion is identified in this excerpt from the introduction:

The last five years has been dominated by discussions about common file formats, and competing systems which support data interchange standards. The International Standards community, through its work on file standards, has helped us reach a situation where students and teachers can easily share assignments, examination submissions and documents.

The same cannot be said for simple data interchange – for example, the simple requirement to automate the process of keeping a list of users up to date within a learning platform without a manual intervention. And those solutions which do exist for this appear to need customisation for each data relationship – between different learning platforms for example.

I believe that we are collecting the data within our educational systems that we need to deliver
improvement, but only those data items that are being seen as “part of the system”. We collect a core of formal learning data in our schools Management Information Systems, and through other systems we collect further datapoints – often disconnected from the core learning data – on health, achievement and engagement.

In the 21st Century, we are seeing a huge growth in learning and engagement outside of the formal education system. As we continue to build extensive connected learning communities, we need to find ways to see the holistic story of a truly connected learner, including their learning in school, in the community and individually. We need to move from a top-down data culture (ie we measure what the managers above us want measured) to an individually driven data culture, where the individual has more input to the data that tells their individual story, and where their past learning journey is used to support their future learning journey.

To achieve this we need to think outside of the strict confines of top-down, organisational data collection. We need to ask questions from a different perspective “How can students self-asses their skills and use that to improve their learning?” and “If we asked a student to tell us how they are doing at school, what data would they share with us?”

I am currently enjoying being a part of separate discussions (online and offline) around each of the issues identified above – but perhaps there’s good cause to reflect here and think about how timely it might be to work like this think tank, and engage in some robust discussions that actually  link both parts of the equation?

Perhaps it would allow us to reach similar conclusions as the English did as a basis for moving forward:

The answer, as it emerged in discussion, is that there‟s arguably too much emphasis on one kind of data. The current pattern of top-down accountability, it‟s suggested creates an emphasis on classroom attainment at the expense of skills and competencies. Or, as Sir Mark Grundy puts it,  “We’re measuring the wrong things.”

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • NewsVine
  • Reddit
  • StumbleUpon
  • YahooMyWeb
  • Google Bookmarks
  • Yahoo! Buzz
  • TwitThis
  • Live
  • LinkedIn
  • Pownce
  • MySpace

No Comments

Leave a reply

Security Code: