I am trying to import big video files (mp4 with h264+AAC) with csvImport(plus). I imported the files via ftp to a /files/upload directory and I indicated the correct url to these files in my csv table.
The files are up to 5Go and are use with the HTML5Media plugin after.
With csvImport (not plus), a first file of 600Mo is imported but the import fails for a file of 5Go. Items are not created for the other files after the fail.
With csvImportPlus, items are created even for files of 5Go but I am not able to play them after (Error loading this resource in HTML5Media player).
I think it is not related to php limits (files are already uploaded). The first (correctly imported) file of 600Mo was copied during the import from /files/upload to /files/original and a thumbnail was created; I think these steps fails for bigger files.
Are the imports marking themselves as errored, stuck on in progress, or just marked as completed?
If they’re actually marking as encountering an error, it should be logged so you can see what’s happening. Certainly there could be a variety of issues with such large files.
I have no error with CSVImportPlus when the 5 items are created (but the videos are not readable):
Temps
Priorité
Message
19/07/2018 09:46:16
DEBUG
Import mis en file d’attente.
19/07/2018 09:46:16
DEBUG
Import démarré.
19/07/2018 09:46:16
DEBUG
Running item import loop. Memory usage: 19226768.
19/07/2018 09:46:17
DEBUG
Completed importing 5 items for the last batch part (total 5 records, updated 0 rows, skipped 0 rows) from “import films.csv” (5 rows).
But I have an error “stuck” (in French “arrêté”) with CSVImport. CSVImport says 1 item imported and 0 ignored. In fact, there is one item created (with the video not working) and 4 other lines ignored
Are you able to play the videos with your browser if you navigate directly to their source URL (that is, their location before they are loaded by the importer)?
With such large files, it’s possible that they were uploaded successfully but just can’t be streamed by the browser, particularly if they’re not set up to be web-optimized (with the metadata at the front, basically).
If they’re playing fine at the original URL then that metadata stuff shouldn’t be the problem.
If your suspicion is that the thumbnailer is what’s failing, you could try temporarily switching to the GD thumbnailer… since it doesn’t support videos, it won’t try to make thumbnails of them.
Have you tried using the File Sideload module to load the videos? Either one-by-one (which File Sideload can do by itself) or together with CSV Import? An advantage of trying one-by-one is you could isolate the file issues from possible background job issues.
Most users’ installations aren’t set up to log PHP errors in background scripts, so problems that would be easily solved by checking Apache or PHP logs are harder when they happen in the background. In this case, you could be hitting a PHP memory limit or something similar. Doing a single sideload would mean the process happens during the web request, which for most users will log errors somewhere without having to change any settings.
File Sideload seems to be a module only for Omeka S. I tried with dropbox plugin for Omeka classic and it works. I have to wait forever for the web request but I don’t have a timeout and I can read correctly my file afterward.
I upload the video files by ftp and then create item the dropbox plugin.