The Living Schoolbook

This is written as a bit of preparatory material for the Ann Arbor, Michigan OSP 2.5 Design and development meeting. I'm including a bit of history here because I think it reflects a case study on project management and product design that the Sakai and OSP communities are facing as they go forward. I welcome any and all corrections to my historical account of the story.

The rapid pace of development has been (thus far) lead by a few schools and institutions that have local interests in mind and an "itch to scratch". I think that the state of affairs is going to be changing soon, with an emerging "User Experience" working group focusing on heuristics and evaluations of new and existing tools to determine their overall usability. I expect that the OSP community and Sakai courseware tools developers will be able to form a new working group that can articulate a vision and design a system that provides a new experience for portfolio development.

Sakai is an open source effort to create a platform for collaboration and learning. An initial grant from the Mellon Foundation in 2003 funded Stanford, MIT, University of Michigan and Indiana University to combine their knowledge and expertise in software development to collaborate on this project. Over the past several years, the Sakai project has transitioned from grant funding to a foundation model, funded by the contributions its paying member organizations, which include primarily higher education institutions and a few vendors that sell support services around open source software. The egalitarian, meritocratic nature of this large open source community presents challenges to software design and development that are often not experienced in more structured environments.

As the Sakai community has evolved, the focus of the community and the definition of what Sakai is have shifted. As recently as 18 months ago, I think I recall that Sakai was referred to as a framework upon which tools could be built. By using the Sakai service APIs provided by the framework, tool developers could focus on the specific functionality of their tool and not have to worry about basic authentication, file storage and security issues typical of web based application development.

The original four institutions had built several tools (Assignment, Discussions, Resources, etc) on the framework to replace their own aging courseware systems on their respective campuses, mainly based on code from the University of Michigan. These tools had a common “look and feel”, used the same template presentation technology (Velocity) and were considered “core” tools for a Sakai distribution. Furthermore, the developers of these tools were working towards the goal of replacing essentially commodity services for which they had several good examples and that the user base had a lot of familiarity with.

During this period, Indiana University and rSmart (a commercial affiliate of the Sakai Foundation) were funded by the Carnegie Foundation for the Advancement of Teaching partnered to port the already successful “Open Source Portfolio” software developed by the University of Minnesota to Sakai. They decided to use the Java Spring framework and Java Server Pages as their presentation layer for the suite of tools that would collectively be known as OSP 2.0. It is important to note that portfolio systems have not worked their way into the culture of most institutions and the implementations of these systems vary widely from institution to institution. As such, there are no strong mental models to work from since the user base expectation of what a portfolio should be has not been firmly established.

The design team charged with redesigning OSP for the Sakai based system decided to make several changes and extensions to the Minnesota system. Portfolios were to be assembled from XML content snippets authored by students and presented by an XSL processing engine to achieve a web-based portfolio. The system is extremely configurable; so much so that “out of the box” the software does almost nothing and requires a great deal of XML design and development in order to customize it for the specific implementation requirements. I believe that the system needs to be drastically simplified and the workflow streamlined for particular uses. One primary use of portfolio software (and one we are very interested in promoting as a School of Education) is to support students to reflect on their learning and to solicit feedback on their progress toward their own learning objectives.

I think that these two development efforts were happening in isolation from each other, the team developing the “courseware” side of Sakai (Assignments and Discussions) had very little connection with the OSP designers and developers. When Syracuse University began using the Open Source Portfolio software in 2005 we realized that there should be a strong connection between the courseware tools (in which students create the bulk of their content) and the portfolio tools if they were to support an emerging assessment strategy in the college that was focused on assessment of student portfolios. Presentation of the concepts at Sakai conferences and in discussion lists proved that our simple idea made a lot of sense to educators. While no formal “evaluations” were done, we definitely heard the experts in the field respond positively to the general idea. The “infinitely configurable” design of the OSP software was widely seen as an impediment to the adoption and use of the software and the community is currently looking for suggestions for improving the design of the software to support a few basic workflows and teaching strategies or “interaction styles” between teachers and learners.

In the following few blog posts I will explain the organizational tasks that I think need to be accomplished by teachers and students using this software during the various stages of one style of portfolio development, the formative journal.

A Class Journal is a formative, reflective, emergent portfolio.

This data flow diagram illustrates the how teachers and students would interact to create, discuss and assess a private (student to teacher) class journal. Periodic review of student blogs (chronologic, personal, reflective portfolios) is likely to be a common interaction style that many teachers would find useful as an venue for formative feedback.

Blogs frequently allow readers (in this case the course instructor) to leave text comments (which this would also support). However, this interaction style would leverage the ability of students and instructors to identify learning outcomes for the journalling process and allow assessment of each blog entry on an ad hoc basis for formative assessment purposes.

Student blog posts would appear as messages in the instructor's inbox. Instructor's would periodically read the blog posts and send back either plain comments or comments with assessments (as related to tagged goals). An important feature of this interaction style is that it supports the idea that "knowledge is emergent" by allowing students to add new learning outcomes to their Goals and to tag their blog posts with those tags. Assessment of student learning, then, can be done in response to student Goals and Faculty Goals (related to the original activity description).

Again, once the course ends or the student leaves the institution, they have the option to keep a copy of all of these posts, assessments and comments for their own use, much like they do with email.

An important assumption is that a system that can engage in this sort of messaging becomes standardized, allowing students to remix and reuse their content from other activities in multiple classes at multiple institutions.

Discussion data flow in a distributed system looks a lot like email. It should!

This data flow diagram illustrates the how students and teachers could interact in a distributed LMS to engage in a group discussion.

A discussion tool is another typical tool found in courseware. The discussion tool is used to allow a teacher to set of "forums" or "threads" of conversation related to the course material. Like the assignment tool, it also lives and dies with the course worksite.

An alternative approach would be to distribute the messages between the courseware system and a learner centered platform, much like email systems do. The difference between the a distributed leaning management system discussion and traditional email systems would be that this content would be stored in a standardized format that allowed users to easily re-use the content in their other courses or portfolios.

Again, once the course ends or the student leaves the institution, they have the option to keep a copy of all of these discussions for there own use, much like they do with email.

Again, there is a lot of duplicated data in distributed system like this which may end up causing issues for storage. However, this does allow each learner to retain a record of all of the discussions that occurred in their classes...forever if they wish to. Similarly, this frees the institution from maintaining the course indefinitely "just in case" students need access to their data.

An important assumption is that a system that can engage in this sort of messaging becomes standardized, allowing students to remix and reuse their content in multiple classes at multiple institutions.

Assignment data flow IF a system was distributed, allowing students to truly own their data

This data flow diagram illustrates the how students and teachers would interact in a distributed LMS to carry out the traditional graded assignment workflow.

A frequent use of courseware is to assign work to students, collect that work from students, assess the work and release the grades to students. In most courseware systems, this is all done in one tool. The Assignment tool (or the assignment and gradebook tools) "live" in a course. The design of courseware is mainly to facilitate the instructor's ability to manage the course.

An alternative approach would be to distribute the various transactions between two systems, the courseware system and a learner centered platform. Each set of tools would be designed and configured with the primary user's interest in mind.

The student would receive a message in their "inbox" with the assignment instructions, learning outcomes and rubric for the assignment.

The student would then do the work, saving their "drafts" in their own system and eventually sending back a response to the assignment with their work attached. Note that a copy is sent to the instructor, with the original content still on the student system.

The teacher would receive the student work in her "inbox" and (assuming that there is to be no formative feedback) evaluate the work against the rubric and releasethe grades back to all of the students.

There is a lot of duplicated data in a system like this which may end up causing issues for storage. However, this does allow each learner to retain a record of all of the assignments, work and assessment data that occurred in their classes...forever if they wish to. Similarly, this frees the institution from maintaining the course indefinitely "just in case" students need access to their data.

An important assumption is that a system that can engage in this sort of messaging becomes standardized, allowing students to remix and reuse their content in multiple classes at multiple institutions.

I am giving a presentation to a smallish group of 4 people today at FCMS. Here is the "teaser" provided on their site:

Portfolios can be used in a small classroom setting to organize and present a project or can span a student's entire education experience. The portfolio experience can be designed to be a highly structured guidance to scaffold students' learning/thinking or can be left as a blank canvas for students to form their own ideas and organize their own learning experiences. How will Syracuse University think about portfolios? Expect to engage in thoughtful conversation about portfolio evaluation techniques, scaffolding processes and types of portfolios.

I made a screencast of the OSP setup we have here in the School of Education. I wasn't able to interview a student so that I could show off their portfolio, but it really does look pretty good.

Instead, you get to see some test data... Ipsum lorem...

The file is pretty big (56MB) and it doesn't "stream" well. I used IShowU to make it and for $20, its a pretty good little app! I just wish it would create a "fast start" movie. Maybe I missed something in the setup...

Regardless, I tried to take the viewer through the following:

• Drivers behind our pilot
• Our plan for organizing students into cohorts
• The types of sites we use in our instance of Sakai
• Some documentation and a conceptual model I wrote about Portfolio Templates (on confluence).
• Some preliminary web designs for what we wanted the portfolios to look like.
• A description of the different data structures created to support our plan and their availability through the Community Library. (See the attachments to that page of confluence).
• A demonstration of what evaluators get to see when they look at student portfolios published using our "Portfolio Review" Template .
• A description of our alternative method for assessing student portfolios using the Data Point tool.

Data Flow Diagram of SyrCLE:

Courseware and portfolio systems are usually designed as relational databases to allow a minimum of data duplication while giving users different roles and permissions to interact with their content and data. Students and faculty have expectations about how different tools should be used to organize data into familiar structures with implicit rules for interacting with the data in those tools. The design of these tools have not not allowed a lot of interplay between the tools. It is difficult to publish a portfolio that reuses classroom content (ie: assignments and discussion threads) that the student otherwise has access to view and manipulate in the limited manner the tool allows.

Furthermore, if a student enrolls in another institution or another class that doesn't use the same instance of the courseware, the data from the different classes can not be combined to create new content.

What is needed is a student centered content management system that serves as the student's virtual backpack and their main platform for lifelong learning. The interface between their content management system and the classroom management tools deployed in a LMS has not yet been designed.

This data flow diagram is intended to describe the flow of information into, out of and within a distributed, networked LMS that has two main components: course content and student content management tools.

A couple days ago I shared some "reconceptualizing" Sakai and OSP based on some feedback from a couple of our faculty and the problems that Dr. Joseph Shedd (our PI) has shared in our meetings between Weber/UMich/Portland/Virginia Tech.

Our faculty have settled into a 1-4 rating scale for EVERYTHING that we do in the School of Education, regardless of the activity or the standard being measured in our assessment system of Goal Aware tools and OSP.