Parallelize media-library (re)indexing

When serviio begins indexing a newly added directory, ffmpeg process can be seen in top(1) chewing up the CPU for that still image 120 seconds into the video. Contacting the external sources of metadata can also take real-time...

I'd like to see these steps performed in parallel -- today's computers have at least two cores, and often more than that. How many of each to run, can be configurable through console (like the number of rendering threads is set now), or determined with the following logic:

• The CPU-intensive steps can simply be one per CPU.
• Those involving remote connections can be even bigger in number -- two per processor would be conservative...

To keep the machine otherwise usable during the indexation, ffmpeg can be spawned with a low nice-value (and some equivalent of that on Windows) so as to keep other processes at higher priority. Makes sense?

[mback2k]

I second this feature request. It would also be great if indexing a newly added folder and scanning for new or changed files were done in parallel.

Yesterday some friends came by to watch a TV show and we had to wait nearly an hour until the new folder showed up, because Serviio was busy checking all existing files. And I had to disable thumbnail generation.

It would be great if the folder structure could show up directly. Meta information and thumbnails are lower priority than the ability to at least start watching the video.

Thanks in advance!

[zip]

There are 2 parallel threads running, one checking for new files and one for updates to currently shared files. Checking for new files might take a while depending on how many files you're sharing as it has to go through all files in the shared folders and check if it's already in the DB or not. So if you have thousands of files it has to do thousands DB queries and Filesystem checks.

If you have more than a single shared folder and want to speed adding a particular file you can disable file checking in the other folders (temporarily).

There are 2 parallel threads running, one checking for new files and one for updates to currently shared files.

Processing of video-files is CPU bound. This means, that several video-files can be processed in parallel obtaining a (nearly) linear speed improvement. That's what I ask for

To top it off, would be nice, if Serviio monitored the host's overall load — so as to only use the idle CPU-capacity when (re)indexing.