Sun Jun 07, 2015 5:53 am
Customizable Metadata Extraction Settings
Hi All,
Love Serviio, been using it for just over 5 years now (0.3 was the first version I installed) and a Pro user since it was first released in version 1.0 and I've recommended it to friends and family a like. Overall I have no real complaints about serviio but I think the portions with metadata have a long way to go...
In the past, I have mostly relied on the/my file structure of my server to view and watch files. As I keep an orderly server with subdirectories for movies, TV shows (and seasons) it just made more sense to me at the time but as my media collection has grown, so has the complexity of using the file structure (I have some files 5-6 folders deep now) and I've started to rely heavily on the Metadata extractors and the advanced functionality the metadata provides (posters, episode naming, actor searches, etc).
Unfortunately, some of my naming conventions do not play well with the default extractors and further, several things that have been picked up, have been tagged incorrectly. I would love to see a fully customizable meta-data selection process where I can manually assign the metadata source (TV vs Movie) and identifier key, especially when a title returns as not found or a way to have a more customizable metadata extraction process. Failing that, a way to modify the default way that Serviio parses the filenames to search for Metadata would be helpful. Here are some file name examples and why this would be helpful:
For some of my directories, I have files sorted according to chronology and not alphabetical. To do this, I put the name of the movie after the year so the file is "1999 - MOVIE TITLE" I dont do this for all of my files but some like James Bond, Disney Animation or Disney/Pixar Animation I like to keep chronological. Unfortunately due to the way the metaextractor works, the search often times returns no results or incorrect results. I've tried renaming the files as "(1999) - MOVIE TITLE" to comply with the extractor process but that breaks it even further since the end result is a search for a blank title as followsTitle="<BLANK>,<FOLDERNAME>" and Year=1999 where it does not search for the Movie Title at all. In the case of James Bond, I was able to overcome this by numbering the films from 01 to 23 before the name such that the title looked lie "01 - Dr. No (1962)" but it only seems to work for James Bond.
There does not seem to be anyway to name the file such that I can keep the films sorted by chronological instead of alphabetical order (on the actual file system/server and in the manual folder) and have it pull down the correct Metadata. In fact, all of my films in the "Disney Pixar" directory either return with no tag or are incorrectly tagged as "Finding Nemo" because the search "1998 - A Bug's Life, Disney Pixar" returns no results for 1998 - A Bug's Life but does return Finding Nemo for Disney Pixar (the same goes for Cars, Monsters Inc, Brave, etc).
If I could set a manual Metadata extractor policy that says "for=<folder/filename>,search=themoviedb, extract=<Date> - <Title>.[mp4|avi], Search Date=<Date>, Title=<Title>" there would be no issue.
Another prime example where a metadata extractor policy would be nice are for TV shows. I have several TV shows now that I've had to rename the folder or individual files in order to pickup the correct metadata. For example "Shameless" and "The Office" both auto-select the UK version of the show and I've had to rename the individual episodes to "Shameless (US) s01e01" in order to get it to pull the correct Metadata. In addition to shows like that, I also have some like Archer which requires the folder to be name "Archer (2009)" in order to pull correct metadata. Again being able to set a manual extractor policy that says "search thetvdb, extract <Title> s<Season>e<Episode>, Search Title=<Title> + "(US)", Season=<Season>, Episode=<Episode>" would allow me to fix this without having to rename all my files.
The last 2 examples I have are where the files are stubbornly not found. For example, no matter how I title the name of the show "E-Ring" it stubbornly returns no results (though I can quickly and easily find it in thetvdb) and thus does not get placed in the "series" folders, in addition, various TV show specials or shows in which my files are 2-part files instead of 1 conjoined file (i.e. Scrubs Season 8 Episode 18-19) thetvdb has the season ending in a double length episode at 18 episodes but the episodes often air independently as e18 and e19 and I have them as 2 files, I recognize this is a DB error not a serviio error but as a result my "Series -> Scrubs -> Season 8 directory is missing a file. It'd be nice to be able to force the file to show in an auto-sort folder especially when no metadata is found.
Finally, the second part of my request would be better logging for the metadata collection processes. It can be difficult to parse through the logs and find which files are missing metadata or have incorrect data, especially since the log rotates with restart of the service and all the other information that gets logged there. I dont know that logging is quite the way to go since logging is more static but perhaps a better idea is a panel within the console or even a web panel that looks into the Serviio db as sort of a "true" admin panel that is capable of listing details about various files (mapping the filename to metadata name, source, id, metadata, etc). As noted, several of my files were tagged incorrectly and I've had files simply "disappear" (though I could find them in the manual folder search, they got tagged wrong and ended up somewhere totally unexpected in the auto-sort). In some cases, it was immediately apparent (5 entries of Finding Nemo each with different run times and only one file called Finding Nemo is a prime example of "immediately apparent") and in other cases it was much more subtle though no less drastic of a mismatched tag and those more subtle errors can be a royal pain to find and eradicate especially when the file name isnt even a close match to the displayed name. The fact is, alot of what happens behind the scenes with serviio is a black box which is great for the end user but not so great for the power user who wants to tweak things a bit more than the average user to achieve the "perfect" media db.
Love Serviio, been using it for just over 5 years now (0.3 was the first version I installed) and a Pro user since it was first released in version 1.0 and I've recommended it to friends and family a like. Overall I have no real complaints about serviio but I think the portions with metadata have a long way to go...
In the past, I have mostly relied on the/my file structure of my server to view and watch files. As I keep an orderly server with subdirectories for movies, TV shows (and seasons) it just made more sense to me at the time but as my media collection has grown, so has the complexity of using the file structure (I have some files 5-6 folders deep now) and I've started to rely heavily on the Metadata extractors and the advanced functionality the metadata provides (posters, episode naming, actor searches, etc).
Unfortunately, some of my naming conventions do not play well with the default extractors and further, several things that have been picked up, have been tagged incorrectly. I would love to see a fully customizable meta-data selection process where I can manually assign the metadata source (TV vs Movie) and identifier key, especially when a title returns as not found or a way to have a more customizable metadata extraction process. Failing that, a way to modify the default way that Serviio parses the filenames to search for Metadata would be helpful. Here are some file name examples and why this would be helpful:
For some of my directories, I have files sorted according to chronology and not alphabetical. To do this, I put the name of the movie after the year so the file is "1999 - MOVIE TITLE" I dont do this for all of my files but some like James Bond, Disney Animation or Disney/Pixar Animation I like to keep chronological. Unfortunately due to the way the metaextractor works, the search often times returns no results or incorrect results. I've tried renaming the files as "(1999) - MOVIE TITLE" to comply with the extractor process but that breaks it even further since the end result is a search for a blank title as followsTitle="<BLANK>,<FOLDERNAME>" and Year=1999 where it does not search for the Movie Title at all. In the case of James Bond, I was able to overcome this by numbering the films from 01 to 23 before the name such that the title looked lie "01 - Dr. No (1962)" but it only seems to work for James Bond.
There does not seem to be anyway to name the file such that I can keep the films sorted by chronological instead of alphabetical order (on the actual file system/server and in the manual folder) and have it pull down the correct Metadata. In fact, all of my films in the "Disney Pixar" directory either return with no tag or are incorrectly tagged as "Finding Nemo" because the search "1998 - A Bug's Life, Disney Pixar" returns no results for 1998 - A Bug's Life but does return Finding Nemo for Disney Pixar (the same goes for Cars, Monsters Inc, Brave, etc).
If I could set a manual Metadata extractor policy that says "for=<folder/filename>,search=themoviedb, extract=<Date> - <Title>.[mp4|avi], Search Date=<Date>, Title=<Title>" there would be no issue.
Another prime example where a metadata extractor policy would be nice are for TV shows. I have several TV shows now that I've had to rename the folder or individual files in order to pickup the correct metadata. For example "Shameless" and "The Office" both auto-select the UK version of the show and I've had to rename the individual episodes to "Shameless (US) s01e01" in order to get it to pull the correct Metadata. In addition to shows like that, I also have some like Archer which requires the folder to be name "Archer (2009)" in order to pull correct metadata. Again being able to set a manual extractor policy that says "search thetvdb, extract <Title> s<Season>e<Episode>, Search Title=<Title> + "(US)", Season=<Season>, Episode=<Episode>" would allow me to fix this without having to rename all my files.
The last 2 examples I have are where the files are stubbornly not found. For example, no matter how I title the name of the show "E-Ring" it stubbornly returns no results (though I can quickly and easily find it in thetvdb) and thus does not get placed in the "series" folders, in addition, various TV show specials or shows in which my files are 2-part files instead of 1 conjoined file (i.e. Scrubs Season 8 Episode 18-19) thetvdb has the season ending in a double length episode at 18 episodes but the episodes often air independently as e18 and e19 and I have them as 2 files, I recognize this is a DB error not a serviio error but as a result my "Series -> Scrubs -> Season 8 directory is missing a file. It'd be nice to be able to force the file to show in an auto-sort folder especially when no metadata is found.
Finally, the second part of my request would be better logging for the metadata collection processes. It can be difficult to parse through the logs and find which files are missing metadata or have incorrect data, especially since the log rotates with restart of the service and all the other information that gets logged there. I dont know that logging is quite the way to go since logging is more static but perhaps a better idea is a panel within the console or even a web panel that looks into the Serviio db as sort of a "true" admin panel that is capable of listing details about various files (mapping the filename to metadata name, source, id, metadata, etc). As noted, several of my files were tagged incorrectly and I've had files simply "disappear" (though I could find them in the manual folder search, they got tagged wrong and ended up somewhere totally unexpected in the auto-sort). In some cases, it was immediately apparent (5 entries of Finding Nemo each with different run times and only one file called Finding Nemo is a prime example of "immediately apparent") and in other cases it was much more subtle though no less drastic of a mismatched tag and those more subtle errors can be a royal pain to find and eradicate especially when the file name isnt even a close match to the displayed name. The fact is, alot of what happens behind the scenes with serviio is a black box which is great for the end user but not so great for the power user who wants to tweak things a bit more than the average user to achieve the "perfect" media db.
