#Linked data library management system

##Executive overview

Highlights

- Fully featured LMS/ILS functionality
- High-performance, attractive interfaces that work on all platforms
- Simple, reliable system management
- Development ecosystem
- Linked-data-based system
- Constantly up-to-date, secure, compatible
- 100% MARC21 + RDA compatibility

In collaboration, Oslo Public library, Computas and Kodemaker deploy an entirely new take on the library system — an ecosystem which provides all the features of current library systems at the same time as leveraging a performant platform for developing modern library applications. The basic configuration provides a fully featured, flexible backoffice solution for traditional library tasks, a performant data platform with an expert interface for cataloguers and an extensible, reactive user interface for patrons.

Applying modern development and administration techniques, we have simplified the work of maintaining a libary system so that it can be hosted simply in the cloud or locally, according to requirement. Changes and upgrades can be rolled out quickly and according to plan, independently or as part of a main release cycle, which means that users get what they need when they need it. It means that everything can be up-to-date with security and browser-compatibility updates.

The system differs entirely from every other existing system due to the fact that it uses technologies developed specifically for cloud-based deployment and uses linked data for storing bibliographic data. Both of these facts make this system unique in several ways, not least in creating something that is more familiar to users in terms of appearance and speed and is capable in terms of what is to come in the next ten years.

In solving the hard problems of deploying a functional platform for working with linked data, we have not only created an entirely ground-breaking system, but we've also provided a clear model for fuuture development in the domain. Because it is the only system of its kind, backwards compatibility is a core component of the system — for all the hype, MARC is not going to die any time soon — and valid MARC21 and RDA compatibility is built in by default.

Flexible hosting options, modern development techniques and quick change turnaround make this system a compelling choice in an otherwise stagnant market. And it is entirely open source.

##So, we did it

Firstly, yes, we did it. We created the thing that everyone is talking about as the future of library systems. That means that the future isn't the future any more, rather it's a few months ago. The paradox of creating what everyone else assumes to be the future is that it gives a new perspective — the next future.

We're already looking at the next steps; that is to say, we're looking at the next hard problems — creating a system that collaborates with other systems from other providers directly. From our side of the hill, things look quite different.

What does this mean? It means some small light in terms of creating local solutions and some hard work lobbying for other providers to deliver data in a way that will work. This topic is covered at the end of this document, first, however, we'll take a look at what has been achieved.

##About the development

The existing ILS/LMS market proved to have few competitive solutions, where vendors have basically provided re-vamps of existing systems with a few new features for specific sectors; this is testified by a largely unchanged feature set in systems developed in the last decade. Even systems that have been developed from scratch show simple replication of feature sets that exist in already available systems. This lack of market differentiation means that there are no compelling choices today, irrespective of what the marketing departments tell you — it's simply a matter of how much you're willing to pay and how good the support is. This depressing situation lead Oslo public library to choose to do something different: do it themselves.

Doing it themselves was not of course a simple task, there were two developers and a number of project managers and product owners already in place at the library, but the team needed to be expanded with outside resources. This meant hiring consultants for process, development and UX. This team has worked for the last two-and-a-half years to provide an ecosystem that corresponds to the needs of a modern library and provides a basis for creating new services quickly and easily.

The ecosystem is more than a simple technology, it is a development pattern that allows tools to be chosen, integrated, tested and deployed. This means that turnaround is quicker and users get to experience changes rapidly. It also means that there is a flexibility in providing a common data platform with different front-ends meaning that different libraries can share one core system, but present themselves to their users in completely different ways.

The flexibility of the data platform comes from using linked data, which means that local needs can be represented in a structured way that will not impinge on standards compliance. This means an end to the bodge-and-fix approach we have become familiar with in library metadata in the past decades. It also means we can do a lot of new things, like providing work-level aggregations and relations in stead of strings. Linked data is not the easy choice and it has required a number of exploratory design choices that have paid off to provide flexible data with rapid access, however, it is a paradigm shift from traditional relational databases.

##What has been done

The basic platform is an ecosystem of modules provided as containers; this is a modern architectural pattern that allows systems to be extensible and easy to administer. The choices regarding integration patterns have been formalised as REST interfaces, which is a pattern that is common, but nevertheless requires planning to provide interactions between the modules that make sense. Many of the choices regarding the choices in the core data platform are informed by understanding what linked data is and how it can be implemented in a useful way. A lot of this knowledge came from previous projects from within and outside Oslo public library.

The fundamental idea at the outset was that a lot of what a library does is well-catered for by existing systems; this intuition is certainly the case. Typical library tasks around acquisitions and logistics are done deals — integrating library systems with other systems for these purposes does not need to be done again. As a result, Oslo public library chose Koha — an open source ILS with a broad community and probably the most installed ILS (conservatively, some 15 thousand installations) — as a base system for these tasks. This choice provides standards-compliant integration with the library world, but it the extent of this module's remit is only holdings and patrons.

The choice to use Koha in such a way means that holdings and patrons re kept far away from linked data; there is a good reason for this — these are areas where we want to know with absolute certainty what the current state is, and this is a task to which a relational database model is supremely suited. Additionally, only a few small considerations in the linked data makes it possible to create full and valid MARC21 records according to RDA or other cataloguing standards. And, such a basis makes it simple to integrate with existing library processes that depend on data in this format, for example, ILL, Z39.50 and OAI-PMH provision.

Linked data requires a platform that is integrated into the Web and has clear strategies for dealing with directly sourced third-party data; "integrated into the Web" sounds like an empty platitude, and for many who choose to utter it, it is. Providing links to external resources requires several things, trust, caching and indexing. It's largely irrelevant how systems source their data, as long as the data is available reliably. Creating a system that uses linked data is much more than adding links arbitrarily to existing relational databases that understand links as links only by virtue of hard coding in the frontend and provide no support or understanding of links as relations in the backend. It's adding this in that makes a different workflow, and it has been this that has been a challenge.

Leveraging linked data is actually simple, but making use of the data isn't. Roundtripping RDF isn't easy and instead of relying on the kind of syntactic solutions we have hitherto seen, we've chosen a mixed route of server and client applications that understand the graph model. Applying approaches using JSON-LD and JSON PATCH for linked data, we have overcome the barriers that have hindered front-end development previously.

Using these techniques together with front-end Javascript technologies that make modular, iterative development less painful has made reactive, web-based frontends a possibility. It has also allowed us to implement clear, logical workflows for both patrons and back-office users.

#Future development

It has become apparent throughout the work that the extent of knowledge about linked data in the library domain does not extend to practical implementation, but rather focuses on the kind of representational solution that the national library of Sweden implemented already in 2007. This is a major issue that needs to be addressed if linked data is to be implemented at all. Additionally, much of the work on linked data has been informed by knowledge from non-graph data storage solutions and this has proven to provide a weak fundament for actual linked data that works at scale and speed.

What we lack is partners who can deliver the kind of linked data we need reliably; the data is available to some extent, but it is simply not delivered in a way that it can be used. The services bear strong resemblance to typical hobby projects; because the data isn't in use as linked data by those providing it (or the service is provided because it was something that was funded as a project back when linked data was cool), it's not possible to make use of the service in a production environment. For this to happen, more library systems that use linked data directly need to come on line. This isn't realistic given the low aims set by systems that are not linked data in their core.