La Vie en Code: AI in Paris and The Most Important Question in the World
In March 2019, Learning Equality co-convened a two-day design sprint in Paris in collaboration with UNHCR, Google.org, Vodafone Foundation, and UNESCO. We explored the need for automated curriculum alignment in crisis contexts, and the possible role of artificial intelligence (AI) in recognizing curricular mandates and patterns, and recommending pertinent educational content in return. This work is part of a broader collaboration working with refugees and partner organizations to explore utilizing digital education to support learning in these contexts.
We’re pleased to introduce the Design2Align blog post series. The experience of engaging our professional communities in such a challenging question was as valuable as the outputs themselves, and we want to share the discussions and debates as they may be useful in other’s work. In this series, the team members that spearheaded each team project will unpack their takeaways via a blog post from the Paris sprint. Following these posts, we will publicly release our findings from this two-day sprint, shedding light on the work we can all do to ensure that the benefits of AI are able to reach learners in crisis and emergency contexts.
We’re delighted to kick off this blog post series with a reflection on the sprint’s metadata project, from Learning Equality’s Curriculum Specialist, Shivi Chandra.
Based on my experience as a curriculum specialist in the world of digital resources, here’s the number one most successful content-sharing best practice of all time: do you have a group chat where you share nonsensical YouTube videos with each other? I’m sure the last thing you shared on it probably wasn’t what you think of when you think of the life-changing potential of digital resources, but think of it anyway. Did you laugh? Did you share it? Did you save it for later?
There’s so much that determines whether content “works” for someone, but for a curriculum specialist, whether it did or not is the most important question in the world. That’s why I’m fascinated with these group chats and seek them out whenever we, at Learning Equality, visit our users: if they let me, I’ll peer over their shoulders to see who’s sharing what in Rajasthan (seventh-grade girls, Bollywood music), in Mexico (twentysomethings, stickered selfies), and in Kakuma refugee camp (everyone, hair styles, Office tutorials, conversion rates…).
These lists of links are mesmerizing. There’s an invisible scaffolding of connections beneath each one that imparts relevance to the act of sharing, that gives us that little thrill of yes, it worked for me — senses of humor, memory, identification, group in-jokes, current events. When they recommend things, people seem to be at their most confident in their ability to access, share, tell, annotate, and flow along all the other vectors of digital citizenship that make us passionate about bridging the digital divide.
But when you consider educational content, that scaffolding can become less about connections and more about constraints, the most important of which, of course, is the mandate of the official curriculum. The less visible but still critical constraints are borne by most of the world’s teachers: awful internet connections, lack of time, money, materials, resources, low literacy levels, students in need of remediation before engagement, overpopulated classrooms, and more. By the time all these constraints are applied, the sense of exuberance, playfulness, and possibility that animates our sharing of any other type of content is all but squeezed out of the whole process. Discoverability becomes a disincentive — educators don’t have time to engage in the process of endlessly applying constraints to libraries, and more importantly, when they do, it’s not exactly something fun to group chat home about.
In emergency and crisis contexts, this process isn’t just a disincentive, but a barrier — refugee educators simply don’t have the time or capacity to portion out content between two or more national curricula, determine a sequence, or engage in any of the other preparation steps we often assume someone, somewhere down the line, has the time and capacity to do. This discrepancy is grave, given that for many of the refugee education solutions embraced today, like certification, diploma equivalence, personalized learning, and prerequisite badging, it’s precisely this type of preparation — clunkily termed curricular alignment — which can truly empower the world’s most vulnerable learners to achieve mobility. Alignment is what communicates to both migrants and those who receive them that their previous education is one element of their former lives which cannot be destroyed, but remains a profound asset for their own and the world’s future.
Over the years, we’ve met more than our share of educators, curriculum writers, parents, librarians, technologists, and designers all over the world who are still finding and sharing materials that work, along the organic, conversational pathways they share everything else, to get around the problem of missing curricular alignment. So confronted with the problem of how refugee learners can benefit, we wondered: is there a pattern to how they do it, one we could all benefit from understanding? And how, when that pattern is imperceptible to the human eye?
As today’s tech trends would have it, that’s where another type of eye comes in: AI, to be exact. Imagine now that all of those constraints are applied with sophistication past the human ability to recognize patterns, to vast sets of educational content, so that human teachers don’t have to do the squeezing. When they come in, they enter this aligned library at the point where they can behave the way they would in their group chats: they can explore, they can experiment. They can play.
If that future sounds outlandish to you, it should, but there are quite a lot of people — from all over the world, not just here in California! — talking about it, imagining it, criticizing it, building it, dismissing it, and even laughing at it. We brought some of them together at our first-ever design sprint on the subject of automated curriculum alignment for crisis contexts, to share everything, and engage with everything, on the subject of how we get the right content to the right places, under complicated constraints — for learners we believe need it more than anyone else in the world. We didn’t want to have talks, present papers, or show off our expertise. We wanted to experiment, throw things out, put things on each other’s radars, and have a conversation as momentarily disjointed, frenetic, and inspired as we’d have with our closest friends. We wanted, in short, to cut out the formalities and do a group chat.
Between sectors working on the world’s problems, it’s often the sad case that most believe the others to be basically clueless. Tech companies can feel policymakers to be hidebound and outdated, academics can feel tech companies to be unreflective and shallow, and for educators, all of them blur into a mishmash of patronizing research, cumbersome solutions, and a staggering disregard for the human experience of teaching and learning as it’s lived, every day, in rich encounters between individual people and the world’s knowledge all across the globe. Wherever you sit at the table, at least one of your dining companions probably thinks both your perspective and your methods are the reason the SDGs are still just Gs.
That’s never been more true than for the project of curriculum alignment, a cumbersome meal served across multiple tables!
Fortunately, in our work in emergency and crisis contexts with UNHCR, we have a collaborator who’s great at setting the table. As soon as we began our initiatives to provide digital curriculum in crisis contexts, we quickly found that the initiative was more like a relay race than anything one organization or even one industry could do on its own.
In the end, who gets— or should get — to select the resources in question? What if they aren’t shareable by the technology we have today? What if the educators didn’t understand or sign on to the curriculum in question? And if all of it was rendered irrelevant the moment someone crossed an international border, why were we even spending time on this?
As we went from conference to site visit to meeting, scribbling references, coding field notes, thanking new acquaintances for links, trying to keep up with the baton as it passed between people who often didn’t seem to be sitting at the same table at all, my teammates and I began to feel that the fragmented nature of the sectors working on this problem was actually a strength, and the key to addressing it productively: true, the pieces of the puzzle were scattered — but so many people had a piece. For every complicated question we were dealing with, I was lucky to meet someone who had tried hundreds of answers to it, settled on one, and was ready to tell us war stories about the others, as well as someone who totally disagreed with the first person and someone else who had spent their entire career on the question and heard of neither of them to begin with. Giving these people the time, context, space, and resources to work on something together — not just to patronizingly “study” but truly co-create with the resourceful refugee educators who were actually testing, using, living, and iterating upon our work — became a dream for us.
Paris is a pretty good place to realize a dream. Our collaborators at UNESCO were kind enough to lend us space at their historic headquarters in the 7th arrondissement, allowing friends from Kakuma Refugee Camp to join in real-time, and our longtime supporters at Google.org sent us off with a skilled sprint master, Astrid Weber.
When the sprint began, the group, together, spent some time building out lists of solutions to curriculum alignment problems we’d experienced across our diverse sectors — red cards indicating things that hadn’t worked well, yellow indicating things that were working all right, and green indicating new ideas we hadn’t yet explored fully. And with the most important question in the world at the forefront of my mind, I found myself drawn to one set of solutions: those involving metadata, or the system of information categorization that effectively helps us apply constraints to see whether content works for a given context, or not.
In time, I was joined by my fellow metadata enthusiasts. While the Eiffel Tower’s reflection glittered serenely away in our computer screens, our team — Team MetaMatch — joked that we must have chosen the most unglamorous subject in the room. As teams went up to present, we glanced at each other in bemused dismay when we realized, as one of us said, that we’d embodied “typical metadata nerds” and forgotten to pick a name for our project (of course, we remedied this in the end with the dazzling official title, “Design-Based Incentives for Collecting, Displaying, and Utilizing Educational Metadata” — coming soon to a silver screen near you!).
It may not be the next international blockbuster in the making, but to us, metadata is more like the show you just can’t stop binging, that you go back to no matter what new thing comes out, because it’s the answer to a perennial question that we haven’t even come close to answering satisfactorily enough to fuel the numerous ed-tech solutions which rely on it: even if the most perfect matching system in the world somehow comes into being, how are all the billions of pieces of content that exist going to be categorized for it to pick up the right things?
How is the solution — any solution — going to know not to recommend someone in a low-resource context a well-intentioned lesson plan starting with “Just head to your 3-D printer and…” if no one has flagged that lesson plan to begin with? And who’s the flagger who gets to decide that that lesson plan can be used to teach “mathematics”? Does it teach mathematics, or is it “Science?” “STEM?” “STEAM?” “विज्ञान?” Literacy for the adult learners making vocabulary flashcards off the lesson? Academic writing practice for aspiring science journalists using it as a prompt? Vocational training for future 3-D printer maestros? Problem-solving in the face of technology no one in the classroom has ever seen?
To some extent, these questions had plagued all six of us on this team throughout our careers. We eventually started to converge upon a solution when we noticed that our discussions hinged upon feedback loops between metadata generation and metadata usage — that is, between the process of a content organizer adding metadata, and the process of a user saving the properly categorized content.
We were sure we’d get somewhere by designing user interfaces that could make use of this feedback loop, and interrupt the user’s workflow at the right place to validate the metadata others had added, creating useful categorizations for content that not only AI, but human users could learn from.
In the end, what we created was a three-pronged approach to incentivize the collection of valuable metadata for educators: consisting of:
a) a way for users to construct searches that take contextual metadata into account
b) an interface design to display search results that may or may not satisfy these criteria, and
c) a workflow to gather missing metadata on these results through tying validation to the bookmarking process
What we hope most is that our design provides a set of practical principles for anyone working on any type of system that needs metadata. No matter what their unique problems or frustrations are, spending time at the sprint showed us that between the six of us, we experienced some permutation of most such problems, and were able to capture our frustrations, hopes, and best ideas for how we got past them with this solution. But most of all, out of a frustrating, unglamorous, and often infuriating subject, I believe we achieved a sense of camaraderie, a sense that we’ve all been in this together, for years before we even met, and that we’ll continue to be in this together, for as long as this problem exists. And after all, isn’t that what our group chats are intended to do?
For me, working with this team reminded me that being involved with the world’s problems demands that we live in that group-chat space, never believing that any one of us has the entire solution to a problem, but being available always to be inspired, touched, aggravated, or simply informed of what others are doing, in all our respective spaces across the world. That’s what will help us truly understand when content works: an understanding of the contexts in which it needs to work and a sincere engagement with the people for whom it doesn’t. As long as we can do that, I’ll be satisfied that even for a minute amount of time, with a minute amount of the expertise available to answer it — we were able to pose one answer to the most important question in the world.
Next in the Design2 Align series, we'll highlight the importance of Communities of Practice and how to use tech to enable teachers to share lessons learned back across the world. Stay tuned!
