Can anyone provide examples of live production systems that have what are considered a “large” collection? I mean, in default install mode on a standard server, at what level does each system start choking? 1000? 10k? 100k?
I realize this data might be hard to come by, so just throwing it out there
For instance - you would not say that a Wordpress default install on a commodity server could easily handle 10k pages. Maybe 1k or a couple K at most… but most often <1k.
We have some interns from a masters program exploring DC, DCMI, Omeka, and Omeka-S for our collection.
BEN BAKELAAR, MLIS
Digital Architect
Thomas A. Edison Papers
Rutgers, The State University of New Jersey
Thanks! So I now understand they were perhaps one of the first or largest implementations of Omeka?
What about their current description, is it larger than 70k items now?
“The September 11 Digital Archive uses electronic media to collect, preserve, and present the history of September 11, 2001 and its aftermath. The Archive contains more than 150,000 digital items, a tally that includes more than 40,000 emails and other electronic communications, more than 40,000 first-hand stories, and more than 15,000 digital images. In September 2003, the Library of Congress accepted the Archive into its collections, an event that both ensured the Archive’s long-term preservation and marked the library’s first major digital acquisition.”
That text should probably say “150,000 digital objects”. Items is ambiguous in the context of Omeka terminology. But, before the migration from the initial bespoke system (2002) to Omeka Classic (2013), that language accurately described the presence of more than 150,000 digital files or text objects included in the collections. In some cases that means more than one file attached to a single item in the Omeka context. So, now there are roughly 70,300 items publicly available in the collections.
One other extremely large instance of Omeka Classic that is in production is the Florida Memory site: https://www.floridamemory.com/ They have definitely done some performance tweaking and load balancing to server all this content efficiently, but I’m not sure we have the full details of that work.
The most recent dev instance of Omeka S I looked at handled upwards of 30,000 items on a localhost install with no difficulty.
Wow, what a blast from the past Here’s what I can say 3 years later…
“It all depends”
From a technical perspective, however many rows the database can store is the limit. So, 2 billion? I have no idea if MySQL has a limit.
But then you get into how optimized the system is for various use cases. We still run into significant slowness (in my own judgment) with 150k records in Omeka-S v1.4.0. However - exactly how much slowness depends on your system specs and any database tuning/performance optimization you might have done or be able to do.
Now with the new v2+ of Omeka-S, I understand from the release notes they’ve made some significant improvements to internal indexing and search optimization - I just haven’t been able to proceed with testing because we are waiting for some of the site modules to be upgraded to be compatible.
But beyond my overall comment of “it all depends”, which is what I’ve learned in the past 3 years of working with Omeka/Omeka-S - the other factor that would be important to note is that Omeka-S (I believe) is designed to have a shareable pool of resources in which any particular sub-site does not display the entire pool. The developers or staff at RRCHNM might comment on that and amend it, but as far as I can see, if you have a single collection with 1 million records, vs. 100 collections each with 1k-10k records, that’s a big difference in terms of how well Omeka-S will function on the front-end.
Here’s what I can say 3 years later…