Sorting (many) imported files

Hi,
I tried to get help on this topic some months ago to no avail, so I may try it again, I guess:

I’m using omeka classic with the “scripto” plugin for a transcription project (->mv-naumburg.de/nt). “Items” are books with some 200-350 pages. When I import page images (named xxx-0000.jpg to xxx-0350.jpg) using plugin “dropbox”, everything works fine, except that the sort orders at some point goes wrong: there are always some 50 to 80 or even more of the first files sorted to the end i.e. they are sorted like that: xxx-0080.jpg, … xxx-0250.jpg, xxx-0001 … xxx-0079.jpg. The following manual sorting is pretty tedious and takes a lot of time. Is there anything one can do to omit that?
Is it completely stupid to have items with 300 files (apart from the sorting problem everything works well as far as I can see).
Every kind of hint would be appreciated!
Thanks,
Sam

Seems like your files are sorted alphanumerically where one expects them to be sorted numerically.

The php function to sort as expected would be natcasesort. Maybe try to use it in your theme ?

Am I remembering correctly that the problem here is that the order looks correct on the Dropbox interface and is out of order after the files are actually added?

Or is it listed out of order in Dropbox already?

I also had this problem when working on a project for Indiana University’s Lilly Library. The project involved radio scripts from Orson Welles. When we loaded the pages of the scripts through Dropbox, some would be out of order. I could be totally wrong, but I alway felt this was due to processing. If a given file took long to process, other files would complete and drop in ahead of it. I felt this way because for us, the files that were not in order seemed random. We didn’t have the time to come up with a sorting solution and had to fix it manually as well.

Your remembering is correct, file order is correct in filezilla and dropbox and gets mixed up after (i.e. by) import.

Thanks wmcowan, I fear I have to accept there is a bug somewhere in the import-process. I found out that import usually works by importing 20 files at a time (but doesn’t work always), that makes 15 imports for 300 files per item (with some 30 items to go). This is really time-consuming but there seems to be no other way (as I dont’t have any coding skills).

Okay, I believe I see what’s going on here, but I don’t have a straightforward immediate fix for you.

The dropbox plugin is just not setting order at all for the files it inserts. This was once pretty reliable that MySQL would, in the absence of a specific order, order things in the order they were inserted, but that’s really no longer been the case as MySQL has changed its behavior over the years.

There’s two basic ways to fix this:

  1. Change the Dropbox plugin so it specifically sets an order on every file it inserts. The problem here is that the code that would allow Dropbox to set an order on ingest is a change to the Omeka core that hasn’t yet been released, so we’d have to make a new core release and make the new Dropbox depend on that version.
  2. Change the Omeka core to sort “unordered” files differently. We can come at this from the opposite direction and just change the query the core uses to order files to account for this problem. Of course, this has the same issue: it requires an Omeka core release, but it doesn’t require any changes to the Dropbox plugin.

OK, I think I got it. As in my case - when “items” are books etc. and “files” represent pages, sorting (no matter if alphanumerical or numerical) is essential. Perhaps you can implement a sorting mechanism into one of the next releases of Omeka, that would be a real improvement.

Anyway, thank you very much for your explanation and thanks to the whole Omeka team for providing that great software for free.
Siegfried

This topic was automatically closed 250 days after the last reply. New replies are no longer allowed.