Migrating to S3 filestorage?


I’m researching the feasibility of migrating a site from using filesystem storage to Amazon S3, and wanted to ask in this forum to see what thoughts & knowledge other folks could add.

I know that Archive Repertory plugin won’t work with S3, but curious if anyone else has done this and has any tips or suggestions or notes about the pros/cons?

Also – in particular, is there a working process to transition existing item files to S3 as part of the migration?


I can’t speak to Archive Repertory as an outside plugin.

On S3 generally, you can definitely transition from using local storage to S3 (and back again). Moving to S3 just requires you to put the files from your local “files” folder in the bucket you’ll be using. There are various methods of doing this… lately I’ve used the aws s3 sync command from the AWS official CLI.

Oh awesome, I had no idea it would be that easy!

Obviously you also then have to enable the configuration for S3 in Omeka, but yeah, moving is easy. Omeka’s “Storage” system basically just asks for a path: when you’re using the default local storage, those paths are relative to the files folder, while for S3 they’re relative to the configured bucket. So it’s just a matter of moving the files so they’re where Omeka expects them to be. And in the reverse direction, switching back to local storage just entails doing the sync in the other direction to pull all the files stored in S3 down into the local files folder.

The main “con” to be aware of I think would be that, since your files are not local, some other plugins are more restricted in how they work. Things like re-generating thumbnails or extracting OCRd text from already-uploaded files are trivial with local storage, but not when the files are remote. This usually isn’t a big problem, as the plugins that act on uploaded files usually can do their work at the initial stage when the upload first happens, before the files get uploaded to S3, but it’s something to be aware of as a difference: there’s some lesser flexibility in changing things related to the files after they’re uploaded.

Wanted to leave a couple of notes here on my experience in case other folks find this thread in the future:

1 Like