Scaling Up a Collaborative Consortial Institutional Repository

Category: Uncategorized


The Institute of Museum and Library Services (IMLS) has awarded $248,050 to the Private Academic Library Network of Indiana (PALNI) in partnership with The Partnership for Academic Library Collaboration & Innovation (PALCI) for Hyku for Consortia: Removing Barriers to Adoption as part of the National Leadership Grants for Libraries Program. IMLS received 172 applications requesting more than $47 million in funding and selected 39 applicants to receive awards during this grant cycle. With this award, the partners will increase the flexibility, accessibility, and usability of Hyku, the multi-tenant repository platform system.  

Repositories are a critical piece of library infrastructure, enabling access to many types of digital materials created by an institution’s students, faculty, staff, and researchers. Libraries, cultural heritage institutions, and other organizations also use repositories to provide access to digitized special collections.

In the face of continued budgetary pressures, libraries are seeking cost-saving approaches to their work. Those unable to deploy Institutional Repository (IR) services on their own due to costs or other constraints are increasingly looking to consortia to serve this role. This project specifically seeks to develop Hyku to support the repository needs of library groups by increasing affordability and flexibility. 

PALNI Executive Director Kirsten Leonard notes, “This grant will provide the foundational support for PALNI and PALCI to remove remaining barriers to more widespread deployment of the repository software. Together with input from our new consortia project participants from VIVA and LOUIS, we will create business modeling and a toolkit to support other consortia to provide this service for their members, potentially reaching thousands of libraries.” 

This project will extend work completed under the previous PALNI/PALCI IMLS grant, which resulted in the establishment of Hyku Commons, a production-level, low-cost, multi-tenant repository service shared by the supported institutions of PALNI and PALCI.  This new round of funding will further improve Hyku by directly addressing needs articulated by stakeholders in a scalable, multi-tenant environment.

The project will kick off with a user study and gap assessment to further define existing barriers and software requirements needed to support the adoption of the service. PALNI and PALCI will employ Notch8, an open-source software development firm and long-time contributor to the Hyku project, to deliver enhancements and changes prioritized in the early phases. Rob Kaufman, Notch8’s Founding Partner and current Product Owner for Hyku, sees this as an extraordinary opportunity to increase visibility and adoption of Hyku. “Hyku for Consortia has been one of the key projects in the community, expanding the functionality of Hyku in ways that really matter to the users. Notch8 is excited to continue this partnership into this game-changing new phase.”  

The project will also expand its partnership to include consortial partners LOUIS and VIVA, who will pilot the service and offer feedback critical to ensuring widespread adoption. A Consortial Institutional Repository Toolkit will provide guidelines, documentation, and other materials to support the development of similar collaborative repository services in other consortia.

Jill Morris, Executive Director of PALCI, is excited at the opportunity to drive strategic innovation of community-owned infrastructure. “This project builds on the strengths of consortia and stretches our relationships to leverage our respective strengths. We are thrilled to continue our partnership with PALNI to explore new solutions, business models, and collaborative approaches to building and sustaining our library infrastructure.

Anne Osterman, Director of VIVA, said, “We are delighted to be piloting and supporting this important project as it develops scalable options for groups of libraries.The creation of a truly community-led, open, sustainable, and multi-tenant repository service meets needs long articulated by academic libraries and the consortia that serve them.” 

Teri Oaks Gallaway, Executive Director of LOUIS, expressed her interest in the grant, “One of our strategic goals as a consortium is to explore opportunities with other libraries, consortia, and vendors for the development of an open-source library services platform. This project is a perfect example of how we can pool our collective knowledge and resources to improve upon and expand the reach of a needed tool like Hyku. We are excited to be a part of this opportunity with our partners and colleagues and look forward to supporting the development of this project.”

“As pillars of our communities, libraries and museums bring people together by providing important programs, services, and collections. These institutions are trusted spaces where people can learn, explore and grow,” said IMLS Director Crosby Kemper. “IMLS is proud to support their initiatives through our grants as they educate and enhance their communities.”

Updates for the project will be made available at https://www.hykuforconsortia.org/.


About the Institute of Museum and Library Services:

The Institute of Museum and Library Services is the primary source of federal support for the nation’s libraries and museums. We advance, support, and empower America’s museums, libraries, and related organizations through grantmaking, research, and policy development. Our vision is a nation where museums and libraries work together to transform the lives of individuals and communities. To learn more, visit www.imls.gov and follow us on Facebook and Twitter.

About the Private Academic Library Network of Indiana, Inc. (PALNI): 

PALNI is a non-profit organization supporting collaboration for library and information services to the libraries of its twenty-three supported institutions. Over time, the library deans and directors who sit on the PALNI board have adjusted the organization’s strategic direction as the internet and information services landscape has changed. PALNI has expanded beyond providing a resource management system to sharing expertise in many areas, including strategic planning, reference, information fluency, outreach, data management, and configuration, and has identified greater collaboration in acquisitions as a key goal. www.palni.edu

About The Partnership for Academic Library Collaboration & Innovation (PALCI):

The PALCI organization was originally founded as the ‘Pennsylvania Academic Library Consortium, Inc.,” and was formed in 1996 as a grassroots federation of 35 academic libraries in the Commonwealth of Pennsylvania. Today, PALCI is known as Partnership for Academic Library Collaboration & Innovation, with membership consisting of 74 academic and research libraries, in Pennsylvania, New Jersey, West Virginia, and New York. PALCI’s mission is to enable cost-effective and sustainable access to information resources and services for academic libraries in Pennsylvania and surrounding states. PALCI Members serve over 800,000 students, faculty, and staff at member institutions, through a variety of programs, including the highly-regarded EZBorrow resource sharing service. PALCI also serves as the home for the Affordable Learning PA program, creating a community of practice for open textbooks and related educational resources. http://palci.org

About LOUIS: The Louisiana Library Network

LOUIS is a consortium of public and private college and university libraries in the state of Louisiana. This partnership was formed in 1992 by the library deans and directors at these institutions, in order to create a cost-effective collaboration among the institutions for the procurement of library technology and resources. We are currently forty-seven members strong.

About VIVA

VIVA is the academic library consortium serving 71 nonprofit higher education institutions in Virginia, including 39 state assisted colleges and universities, 31 independent private, nonprofit institutions, and The Library of Virginia. VIVA’s mission is to provide, in an equitable, cooperative, and cost‐effective manner, enhanced access to library and information resources for Virginia’s academic libraries serving the nonprofit higher education community. 

About Notch8:

Founded in San Diego, CA in 2007 by Rob Kaufman, Notch8 is a Ruby on Rails-based web consultancy with additional expertise in React and React Native mobile applications. Today we are a team of 18 developers and technical experts located across three time zones. Since 2016, we have been active with digital repository solutions, primarily through our involvement with the Samvera Community. We are Samvera Partners and both in and out of the Samvera framework, we have contributed to more than 20 projects in the digital repository space. 


(Feature photo by Olya Kobruseva from Pexels)

We thought we’d kick off the new year with a project update. 

At the end of 2020, as we looked to the end of “Phase 2” of our work (improving features for multi-tenant administration), we took some time to review our project goals with Notch8.  In this review, we laid out the deliverables promised in our IMLS grant and determined a deliberately scoped path to complete these goals by Spring 2021.  We both took stock of our progress to date and identified areas to further define our goals for the sake of efficient progress.  This was done with hopeful anticipation of an additional round of grant funding for Phase 3, in which we plan to remove identified barriers to adopting Hyku, both in and outside the consortial community.  

As a refresher, here’s an overview of our grant goals and deliverables:

Collaborative Workflow Support

Development is currently underway for collaborative workflow support.  Updated scoping for this work now includes:

  • An admin can create a new group in a tenant
  • An admin can then assign roles to that group 
  • Users who are added to a group will receive the permissions from that group. 
  • On the user management tab user’s groups as well as any individual permissions granted to the user will be displayed
  • Tenant level roles on the User Matrix will be created 

In a future project phase, we hope to add the Multi-tenant Manager and Multi-tenant Editor roles.  We also want to create a groups and permissions area on the consortia admin page that will create one workflow for adding groups permissions across multiple tenants.

Worktypes

Development is also underway for worktype development.  The OER worktype is completed (specs here), with specifications for the ETD worktype are complete and the “shell” worktype (a copy of the generic workype) already created.  Work now underway and now nearly completed includes:

  • Metadata customization for the ETD worktype
  • Fields will be configured according to the ETD worktype specifications
  • Once the fields are configured Bulkrax mappings will be set up for importing and exporting the ETD worktype.

We’d love to further explore easy creation of worktypes in the future, as well as greater flexibility for controlled vocabularies.

Themed Templates

We’ve done a lot of work gathering specifications and mocking up wireframes representing the themes (IR, cultural heritage, and neutral) we’d like to implement as part of Hyku.  Scoped work in this area for the remainder of this project phase are as follows:

  • A Theme tab will be added under the Appearance page. 
  • On this tab, a user will be able to select a home page theme, a search results page theme, and a work display page theme. This will allow for greater flexibility for repository managers and extend the core offering to a wider range of use cases. 
  • The theme pages will respond to the colors, logos, and feature flippers set in the app.
  • The following Pages will be built as themes (referencing preliminary mockups):
    • 3 Home Page options (Cultural Repository, Institutional Repository, Neutral )
    • Search Pages with Gallery, Masonry, and Slideshow
    • Images Based and Text Based show Pages.

This is an exciting new development path, and we can’t wait to see how it turns out!  In the future we may make some changes to how the template elements function, and possibly additional options to make the theming as flexible and customizable as possible.

For the remaining deliverables (DOI minting, cross tenant searching, and multi-tenant shared works), we’ll continue to gather requirements from our user communities and explore work being completed in complementary projects. As always, we look to integrate our work with the larger Hyku Roadmap, contributing our improvements back to the Hyku base code and avoiding duplicative development efforts whenever possible.  

We’ll continue to post updates on our project here, and please feel free to contact us with any questions.


We’ve been on the virtual road the last two months talking about Hyku for Consortia at two conferences: the USETDA 2020 Conference and the annual Samvera Connect conference.

While Covid-19 has made travel impossible, the bright side is that we have recordings of presentations that can be shared for all. Below is the video from our presentation at the USETDA 2020 conference held September 23-24: Providing Flexibility and Affordability for ETD with HYKU

In October we participated in three events in the annual Samvera Connect Conference, a poster, a panel, and a presentation:

Both the panel and presentation were given on the same day (10/28/2020) and you can find video from both at on Samvera’s YouTube channel.


This post is the second of a two-part look at bulk upload in Hyku.  The first examined the background of bulk operations and why they are difficult to do well. This post focuses specifically on the application of a bulk import solution in Hyku Commons.  The need for bulk upload in this project is similar to those identified by the large Hyku user community.  We too need an “easy in and easy out” data solution for our repository users.  (Photo above by Pexels on Pixabay)

In PALNI’s 2018 white paper, we identified several valued repository attributes, and have since adopted them as the shared vision Hyku for Consortia project.  One of these values speaks directly to the need for bulk upload solutions: “The collaborative institutional repository should be a system which is interoperable and allows free-flow of data. Easy import and export of metadata and objects are possible.” The use cases for bulk ingest are numerous.  Migrations from another platform, repurposing data from external sources such as finding aids, and user preference rank high on the list for why one would rather import works and their metadata in bulk rather than piecemeal.

To further illustrate the need for bulk importing, you will find a list of workflow examples in the Hyku for Consortia project documentation.  These examples provide hypothetical consortial profiles and repository scenarios based on real-life stories contributed by our Product Management Team.  From these scenarios:

  • Scenario 1, “Midwest Library Consortium”: Tenant-only Editor is in the archives department and has a digitized archives collection to add to the repository. He creates a new collection and uses one of the pre-populated admin set choices. He then bulk uploads the content and saves it but does not publish it.
  • Scenario 3, “Wealthy Alumni College”: Tenant-only Editor begins uploading student works in bulk into the repository with draft metadata, licenses, and embargoes. 
  • Scenario 5, “Sunnydale Community College”: Student staff member is made Tenant Editor. She uploads minutes in batches with a spreadsheet of basic metadata. The collection is not yet published.

These scenarios helped us to envision all the ways that Hyku might be used for various IR users and content, and to define our collaborative workflows and user roles.  Also, without us realizing it at the time, they very much highlighted how essential bulk import is to this work. In three out of five of these examples, we envisioned works being uploaded in bulk by Tenant Editors, who might be an archivist/librarian, grad school staff member, or even a trusted student.  These users have metadata in an existing external format, and rekeying hundreds or thousands of metadata values would be a waste of their time.

Shifting away from the hypothetical to the actual, now that we are using Hyku Commons for real-world pilot repositories, the need for bulk import functionality is even more apparent.  For example, one of our partner institutions moved content from Digital Commons to CONTENTdm as a stop-gap when they lost access to the platform due to cost. Now they want to move that content into Hyku. 

Using the Bridge2Hyku project’s CDM Bridge tool, export was a breeze.  We were able to extract all the files and metadata from CONTENTdm in a way that Hyku would understand.  But how to get the described works into Hyku?  The native Hyku batch import did not provide a solution, since it applied identical metadata to each item.  The records we wanted to bulk upload have complete, individual descriptions. We soon learned that this kind of desired bulk import was a much more complicated task, and reached out to Notch8 to find a solution.  

With Notch8’s help, we investigated HyBridge (the import counterpart to CDM Bridge’s export), Cdm_Migrator, and Bulkrax as potential bulk import solutions for Hyku Commons.  We selected Bulkrax for our project because it seemed to work best for our multi-tenant environment and was easiest to configure within our setup.

According to the Samvera Labs webpage, “Bulkrax is a batteries included importer for Samvera applications. It currently includes support for OAI-PMH (DC and Qualified DC), XML, Bagit, and CSV out of the box. It is also designed to be extensible, allowing you to easily add new importers into your application or to include them with other gems. Bulkrax provides a full admin interface including creating, editing, scheduling and reviewing imports.”

Check out this poster from Samvera Connect 2019 for more information about Bulkrax.

Bulkrax poster by Keving Kochanski, used with permission

After working with Notch8 to install and update Bulkrax into Hyku Commons, we viewed developer-supplied walkthrough videos (like this one) and wiki documentation to get a better understanding of how to use the CSV importer.  It is now possible to bulk upload to Hyku Commons with Bulkrax by importing a zipped folder containing a folder of files and a properly formulated CSV file.  The CSV contains rows for each object’s descriptive metadata.  Additionally, the first four fields are administrative fields, which govern how the importer imports the files. 

Administrative Fields

  • item – Lists the name and extension of the item being imported, such as file.jpg. 
  • source_identifier – Establishes a persistent identifier for the object being imported. 
  • model – Identifies the worktype the work will be created as. 
  • collection – Determines what collection(s) the work will be added to. 

One of our challenges is the lack of step by step documentation for these processes.  It’s a complex process and a tad finicky, so a very detailed guide would be helpful.  Another is the need for separate parsers, and the intervention of a developer to create them, for custom worktypes.  For our bulk upload to work for our OER worktype, for example, separate work had to be done to add the parser and to allow the relationships between items mentioned in the last post.  Lastly, there were a few oddities along the way that we reported and were added to the Bulkrax project board so that they can receive feedback from the community.

In considering bulk capabilities for our project, the next step is to look towards bulk export with Bulkrax. This functionality currently exists in limited capacity, but it is in further development for wider usability.  In keeping with the “easy in, easy out” theme, there are many use cases in which we’d desire the ability to export metadata as well as files from the Hyku Commons tenants. Stay tuned for additional developments on this process!


This post is the first of a two-part look at bulk upload and data remediation in Hyku. Part one is going to take a look at the background of bulk operations and why they are difficult to do well. Part two will talk about our specific work to try and address some of these needs in the Hyku for Consortia project. (Photo above by Ryoji Iwata on Unsplash)

Bulk operations in Hyku have a long history. In the initial user survey, conducted way back in 2015, one of the main findings was that Hyku needed to support the “easy in and out” of metadata. Metadata migration/remediation/transformation has always been a major activity in libraries. Think back to what an enormous task retrospective conversion of card catalogs to MARC was. Any library system containing metadata has to be able to manage that data at a large scale. 

The design team for Hyku knew that bulk operations would be a key element to allowing potential users to commit to migrating out of their current tools. Hyku entered a market with a number of existing repositories. This new solution might have been able to solve many of the community’s frustrations with those tools, but only if there was an easy way to migrate to it. The initial requirements and personas therefore both reflected the needs to tools to upload and transform metadata from one system to another. Mockups reflected the need to both migrate data as well as remediate it.  

Summary from an early Hyku design document

This work was then reflected in Github issues during the project development (see: https://github.com/samvera/hyku/issues?q=is%3Aissue+is%3Aopen+bulk), but other more basic needs for repository development (you need a repository to migrate data to, after all) took a higher priority. So a new grant project called Bridge2Hyku picked up where development left off and explored the issue of migration in more depth (https://bridge2hyku.github.io/). Our colleagues at the Bridge to Hyku project did great work analyzing not only how to upload data and objects to Hyku, but also how to get it out of some of the major repository systems currently in use.

All of this work then…but why is metadata migration and bulk creation/upload so difficult? 

The nature of structured data is what makes it so powerful: you can index and search it, you can compare like to like, you can organize and sort. In short, it makes order out of chaos. And as humans, that’s what we naturally do: recognize patterns. But, also like humans, we might all see the world slightly differently. So different metadata schema and repository systems can have their own way of seeing the world. Some are quite simple and allow for the same basic type of description of everything. Others are quite granular, allowing for more nuanced description of subtle details that can be important and powerful. So any system to migrate or convert from one system to another typically relies on a lot of human intelligence to see the patterns and make the connections.

Photo by Luke Chesser on Unsplash

But human capacity is only so much. How do you analyze thousands of records? Analytical tools like Open Refine can be helpful. So can guidelines for general rules on the major categories of migration as shown in crosswalks from other projects. But, as these examples perhaps show, these tools are not simple and not necessarily easy to pick up and learn. So any migration process is either going to require a lot of manual intellectual effort, or the creation of new tools to help with this business of organizing and translating.

The quirks of particular systems can also provide barriers. You may come up with a great crosswalk that works for one system, but doesn’t capture the nuance of another. Within Hyku for example, all works are sorted into worktypes. These types define the metadata schema used, the relationships between objects that can be created, and in some cases, the way that the object itself is presented and handled within the repository. 

Data from other systems that don’t use this type of organization then require an extra step to define the worktype data should be migrated to. The system that data is coming from can also prove a barrier. Some systems are opaque making it hard to know exactly how data is stored. Others make it difficult to export data out. Many systems can provide an XML feed of records through a tool using the OAI-PMH protocol, but these are then just records, not objects themselves. Others might use a newer protocol like ResourceSync for export, but may be incompatible with systems still relying on OAI-PMH.

Finally, issues can come from the very nature of materials themselves. A particular challenge we’ve had with migration relates to the inter-relationships between objects. As I’ve talked about before, and will likely write about on this blog in the future, one of the key needs we found to assist in the uptake of Open Educational Resources (OER) is the availability of related teaching tools or ancillary materials. A freely available textbook is great, but if there are also related quizzes, videos, or lecture slides, an educator has all they need to make the switch. In order to make these materials visible in an OER repository, we need to have the ability to define lots of different types of relationship like “translation of”, “part of”, or “replaced by” (for new editions). 

Creating these relationships may be easy when materials are being uploaded as they are created on an ad-hoc basis. But migrating them to a new environment presents a new challenge: how do you create a relationship between materials that may be next in the queue to be created? There isn’t a simple solution. For us, it’s meant creating some new code to handle the creation of relationships as a second step in the data migration process. The point of this example isn’t necessarily the solution we found to this problem, but the acknowledgment that many other types of materials may present their own unique needs. While uniformity and standardization is good, it’s the balance between standardization and diversity that makes a repository useful.

So bulk operations in repositories is a hard nut to crack. There are similarities in any migration or conversion, but there are also a lot of specific challenges to every situation. In our next post, we ‘ll talk about the development of bulk upload functionality for Hyku Commons and how we addressed challenges in our own work.


PALNI and PALCI are working together on Hyku because we believe in its potential for improving repository workflows and open access for libraries. But we are also working on it because we understand and value the benefits of collaborative work on open source software. On Friday, June 26th, I am doing a presentation for the West Virginia / Western Pennsylvania chapter of ACRL called “Collaborating for Innovation: Developing Consortial Open Source Software at PALCI,” part of which will involve a discussion of Hyku for Consortia. I’d like to take some time this month to delve into the philosophy that I’ll be discussing there about collaboration and consortia.

We’ve written about collaboration before, and so have others. Generally speaking, consortia help to increase the scale of the work that libraries can do. When consortia work together, they can increase that power even further. Collaboration on open source software projects is a particularly good example of the benefits of this kind of cooperation.

First, a little background on our two consortia. PALNI is a consortium of 24 private, academic libraries and PALCI is a consortium of 70 members of varying types of academic libraries from small to large, public to private. We both include aspects of what Lorcan Dempsey calls (in the second link above) the “classic library consortia activities …  some combination of licensing, resource sharing and training, or sometimes manag[ing] a shared library system…” But we are also both trying to increase our value to our networks in new and innovative ways to help them meet new challenges.

One of those challenges we hear about is infrastructure for handling scholarly communication or other types of digital object management. Even with the diversity in our membership, we hear about these issues frequently:

  • Cost: Many repository solutions are simply too expensive, either in actual dollars or staff needed to successfully use them.
  • Adaptability: Many of our members have diverse needs when it comes to repositories. They have both scholarly communication missions as well as managing their own digital library content, and many solutions fit only one or the other successfully.
  • Limited choice: Consolidations and mergers in the vendor market seem to be limiting choice. The fear of getting locked-in to an infrastructure that will become untenable is real.

So here is a clear case where open source software may be helpful. Firstly, because it has a relatively low barrier to entry: the communities supporting many tools are open. While it may take some expertise to really engage, we have that expertise within our networks. There are also potential open source software solutions to these problems that could have a really high impact in meeting member needs. Finally, within PALNI and PALCI we already have some infrastructure and experience in repository management through PALNI’s research and other types of repository services and PALCI’s participation in the HykuDirect Pilot.

Our Hyku for Consortia project then, is an opportunity to help meet these members’ needs in some specific ways. First, we can mitigate the risk they might take on in trying something new. By spreading out that risk among our consortia — each institution supporting the project through a little bit of staff time, or a portion of their membership fee — we decrease that risk for all. Secondly, we also extend the opportunity to our members to help shape a product to specifically suit their needs. We are doing this by including multiple members from both consortia in our Product Management teams, getting feedback from members testing the software, and having open discussions with the membership on our progress. 

The sneakiest benefit of all though is that investment in this particular software (or other open source software projects) can have ripple effects in the rest of the environment. The developments we are working on with Hyku will be highly beneficial to other vendors or providers that may offer the service as well. By integrating newer, better standards, we raise the bar for other competing services. So even if our member libraries don’t participate in this specific project, they can experience the benefits in the long-term.


Within our IMLS grant project, we have been working hard with our product management team and developer Notch8 to define and develop consortially-focused improvements to Hyku. For example, last month we shared some logistics for building collaborative workflows, working towards a master dashboard to control multi-tenant user permissions.

At the same time as these development activities are taking place, this project has also focused on the practical aspect of making the existing version of Hyku usable for our consortial partners to pilot as a working institutional repository.  Our work has thus branched into two separate areas: Development and Production.

The Production arm of our work focuses on readying the existing Hyku Commons product for real-world pilot use starting this summer.  As a result of user testing from both the PALNI and PALCI sides, we’ve been submitting tickets for small bug fixes and minor improvements which are now happening parallel to the development of features as outlined in the IMLS grant.  Notch8 has devoted a lot of resources to our project in both arenas, and we’ve established a great working relationship and clear communication of needs from both sides.

To date we’ve created two clear and separate working instances of Hyku for Development and Production.  First, the Development instance acts in a number of purposes:

  1. A sandbox for PALCI and PALNI institutions to preview and test a Hyku tenant
  2. A staging area for Notch8 to preliminarily roll out updates, bug fixes, and new features

Second, the Production instance is where work is deployed once tested in the Development environment and also where pilot repositories will be built. It will be publicly available as a working repository soon. 

We’ll share the Hyku Commons product (ie, our Production instance) when it has pilot content that is ready to be viewed.  For now, checkout the demo video below for a brief look at our Development environment. PALCI and PALNI libraries can request a test repository in the Development instance using this form.

Five minute Hyku Commons demo


It’s no surprise that recent global events related to the COVID-19 pandemic have affected libraries across the globe. As we focus on keeping distance to slow the spread, one bright spot is that we have been remotely collaborating on our cross-consortia repository from the beginning, so it’s offered a welcome sense of continuity in troubled times to continue the project.

Our last posts outline the goals and planned activities of our projects, and in the interim we’ve made excellent progress on defining the requirements and designing the planned outcomes for the first two of our major development activities: 

  • Building collaborative workflows 
  • Theming and branding development

With this blog post we’d like to focus on introducing what “building collaborative workflows” means to us. Consortial collaboration is more than just sharing costs. We want to create a tool that will allow us to jointly manage a multi-tenant repository infrastructure.  Creating the flexibility for both IR workflows and more “traditional” library-owned content within the same instance of Hyku means enhancing the ability to manage user and tenant settings (enabling different workflows) through the consortial dashboard.

Our process for uncovering a way to address the rather broad task we’d given ourselves leaned into our collaborative process to uncover the places where workflows overlapped and diverged among our consortium members. We asked our Product Management Team to articulate the types of collections they hoped to build with Hyku. They described the types and sources of materials, as well as the people involved, thus identifying where workflows overlapped and diverged among our consortium members. 

From these we next began to brainstorm through narrative scenarios of various workflows. These helped to highlight specific shared workflow tasks as well as gaps in the current Hyku product. We also examined the existing user roles and permissions available within and across tenants and articulated the need for some additional levels of permission through narrative documents, matrices, and visualizations of these shared workflows.

By working through this process we realized that a robust dashboard for user/role assignment, and the expansion of a few more roles, would enable us to manage these flexible workflow options. The current multi-tenant administrative dashboard for Hyku only allows for the creation of new tenants and the creation of users. We would need something far more powerful to assign users to our envisioned permission levels in multiple tenants.

With this basic idea and our specific needs for user levels articulated, we turned to work with our development partners at Notch8. Talking through each of our requirements documents, we have come up with a rough development plan. Some of our expectations will likely be adjusted based on the feasibility and difficulty of implementation, but our goal of “building collaborative workflows” will remain the same. 

  • First is to decouple the “role” functionality in Hyku from the “group” functionality. Currently, permissions are assigned at both levels which can work at cross-purposes. 
  • Next we will develop the dashboard needed to control these permissions. This part will require us to put our creative thinking caps back on to more fully define what this looks like.
  • Finally, we will work on implementing roles at the tenant level through the new dashboard.

We hope you’ve enjoyed this little peak behind the curtain at this behind-the-scenes “collaborative workflow” of our own: a cross-consortial development process between partners in three different states and two different time zones, using shared online tools, working asynchronously but together. We look forward to sharing our results in the future.


At the end of 2019, we posted an introduction to our Hyku project, Scaling Up a Collaborative Consortial Institutional Repository (made possible with support from IMLS).  Now we are sharing some of the high-level goals and phases for the project, as well as a status update.  Stay tuned for progress on these activities!

Goals:

1. Contribute an affordable open source IR tool to consortial communities

2. Develop a model for collaboration and shared infrastructure that is easily adoptable

3. Further grow the Hyku community

Activities:

Phase 1: Specification – In progress

  • Needs assessment for use cases, workflows, and functionality
  • Specification gathering for ETD and OER worktypes and workflows 
  • Collaboration with external advisors feedback
  • Exploration of consortium scale DOI services 
  • Distillation of specifications for development planning

Phase 2: Development – to start in April 2020

  • Building collaborative workflows 
  • Theming and branding development
  • Multi-tenant viewable works and searching
  • Enhance data exports for improved discovery
  • ETD and OER worktype implementation and versioning
  • Integration with external Hyku development efforts 

Phase 3: Pilot and Communication

  • Early development testing
  • Pilot phase
  • Project reporting, documentation, and training 
  • Build out sustainability/governance/business models 
  • Outreach & communications 
  • Contribute code and development efforts back to Hyku/Samvera community

Update:

Working with our Product Management Team’s use case scenarios, we are currently defining improvements we want to make to Hyku to support consortial workflows. So far, we’ve identified the need for more levels of user permissions than standard Hyku offers, and tools to assign users to roles across more than one tenant. These will allow consortia to collaborate together on things like collection development or metadata creation, if they so desire. 

Our next big area for exploration are the needs for easy look and feel customization and feel, and non-repository features of each tenant (things like widgets for displaying featured items, or social media feeds, etc.).  Soon we’ll move on to development of these first two areas, while we continue to flesh out our needs for an ETD worktype and DOI services, among other features. Check back in on this space for more on these in the future.


Earlier this year, in July of 2019, PALCI (the Pennsylvania Academic Library Consortium, Inc.) and PALNI (the Private Academic Library Network of Indiana) were awarded a grant from the Institute of Museum and Library Services to develop the Hyku repository software into a meaningful solution for library consortia. The project team will periodically make updates to this blog on our progress, and we’d like to start with an introduction to the project and a glimpse into what we envision for the future.

The collaboration between PALCI and PALNI began more than a year ago. In 2017, PALCI and PALNI engaged members of the International Coalition of Library Consortia (ICOLC) in meetings at the American Library Association Midwinter and ICOLC Detroit conferences to discuss consortial IR needs where over thirty consortia expressed interest in collaborative IR efforts. After discussions with other consortia, and having completed an extensive environmental scan, the partners identified Hyku to have the highest long-term potential as an affordable, scalable consortial solution. 

In Fall 2017, PALCI piloted a scaled-back, hosted version of Hyku, the only available open source, multi-tenant IR software. The HykuDirect pilot was part of a grant project run by Stanford, DuraSpace, and Digital Public Library of America (DPLA) and funded by IMLS (LG- 70-15-0006-15). Tests showed the software was promising, but additional development work was required to build on the success of this project. In early 2018, PALNI developed a Consortial IR Vision (https://goo.gl/58HCuz), which provides a graphical sketch of the desired multi-tenant environment, maps out specifications for several functional levels, and defines user management needs. This document has been shared and used widely across the consortial and library communities and has been effective in conveying the consortial IR vision to partners and service providers. 

Our new IMLS-funded project will allow us to further develop, test, and evaluate the product in order to move toward a production-ready service. Notch 8, a web development firm with substantial Hyku expertise and a leader of and major contributor to the Samvera community, has been identified as the project’s lead application developer to create a multi-tenant Hyku instance for shared use by our two consortia through this project.

Our first step in the project was to convene a group of representatives from across our two consortia into a Product Management team to help us define the features needed for truly collaborative workflows. These features will enable consortia members to work together in the creation and management of repositories, allowing members to share the burden of administrative tasks like metadata creation, enacting embargoes on content, or clearing copyright if they choose to collaborate. In addition, we will be developing new worktype models for electronic thesis and dissertation (ETD) objects and open educational resources (OER), and working on the ability to customize features for individual tenants.

Our Product Management group have begun their work and we expect to begin heavy development in early 2020. A pilot project will follow in the summer, during which we will develop our plan for future governance and sustainability among consortia members. In the coming months we hope to be able to share here documentation of our plans and progress.