Profile - transcode based on overall bit rate mode status

My Serviio & ffmpeg expertise is limited to what I have gleaned over the last 3 weeks using this forum as a guide.

I recenty purchased my 1st. "smart" TV - Panasonic Viera (TX-L42E30B 2011) and I've been exploring its DLNA capabilities.
I particulary wanted to be able to use my Panny to access a growing library of movies stored on a networked server (Intel i7 quad 12GB Windows 7 ultimate x64).
I eventualy came upon Serviio which appears to meet my needs far better than other software free/fee that I looked at.

Most of my movies are stored as MKV files of varying quality and size.
I'm very pleased to say that with the advice and guidance within this forum, I've been able to play most movies using native support on the TV using a very slightly tweaked version of cerberus' profile (viewtopic.php?f=13&t=5119).
I have previously remuxed each MKV file (using MKVMerge - painless and quick) to remove any header compression as I understand that DLNA does not support header compression.

I've even found that the Panny has native support for subtitles and plays them fine as long as I don't transcode them!

So what's my problem?
Normally the Panny happily handles these movies very well including pause, restart, fast forward and fast reverse. However, for just a few movies, these navigation functions do not work. For these movies, pause works, but resume, fast forward or fast reverse will result in restarting the movie from its beginning.
After inspecting these problem MKV files (with MediaInfo) I've concluded that it is only MKV files that were originally encoded with "Overall Bit Rate Mode=Variable" that cause a problem. This may be a specific problem with my Panny, but I think its just a transport stream restriction?

So what do I want?
I really like the fact that Serviio presents 95% of my library to play native on the TV - that helps my network and keeps my server cool!
Ideally
>I would like an option in the Serviio profile so that I can select transcoding just for MKV files with "Variable Overall Bit Rate Mode".
>Now I realise this would probably involve additional inspection of the MKV file.
>I threw together a batch script using the command line version of MediaInfo to run through my library and identify all my problem MKV files.
Or 2nd best
>A method of providing a "user exit" which would hand over to a user written script where the decision to re-render or not can be made.

ps - here's a tested quick and dirty native ffmpeg cmd that rerenders to a reasonable quality and removes "Variable Bit Rate Mode" ...
ffmpeg.exe -i "input_file_path\filename.mkv" -y -threads 6 -async 1 -vcodec libx264 -coder 1 -flags +loop -cmp +chroma -partitions -parti8x8+parti4x4+partp8x8+partp4x4-partb8x8 -me_method hex -subq 5 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -b_strategy 1 -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -wpredp 2 -vf "pad=max(iw\,ih*16/9):max(ih\,iw/16*9):(ow-iw)/2:(oh-ih)/2:black[out]" -acodec ac3 -ab 192k -ac 6 -copyts -scodec copy -map 0 "output_file_path\filename.MKV"

pps - above padding options provided by Illico within this forum.

[jhb50]

Use the same technique as I use for refresh and invoke a modified ffmpeg.bat See the refresh plugin and the wiki entry

Hi jhb50, thankyou for your update.

I did come across your excellent work when I was reviewing the forum and one of the early tasks I did was to take a leaf out of your book and write a small ffmpeg script wrapper - all it initially did was to write a log entry to a file with the parameters passed to it from Serviio, and then issue the ffmpeg.exe cmd on behalf of Serviio.

Well what I found was that for locally held files (ie my library sitting on the same server as the Serviio code), Serviio does not invoke ffmpeg to present the MKV file to the TV when they are natively supported by the TV. It only does so, when the active profile requests a remux/rerender. I don't know how Serviio presents these natively supported files to the TV, but it isn't using ffmpeg that I can see.

So I can't see how an ffmpeg wrapper can help - ffmpeg never gets called.
If I force Serviio to call ffmpeg (by using Serviio's profile interface) then I can happily handle the 5% of files that I want to rerender within an ffmpeg wrapper, but I'm then left with the 95% of files that I don't want to rerender 'cos the TV can handle them natively - and I'm not sure how to present these to the TV - ie how does Serviio do it?

Regards.

[jhb50]

Why not setup the profile to mux them anyway (which takes minimal resource) and that way everything goes to ffmpeg.

Hi jhb50, thanks again for your update.

Well I guess muxing all the good ones and rendering all the bad ones might be a pragmatic solution were it not for bugs in ffmpeg remux.

1> about 5% of my "good" MKV files won't remux using ffmpeg due to this ffmpeg bug ...
> ([matroska @ 0000000004768a10] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 2054 >= 2054)
> these files happly remux with MKVMerge but not with ffmpeg.

2> just remuxing with ffmpeg distorts the aspect ratio of the movie (reported elsewhere).
> so ffmpeg remux is not "benign" and over a large range of movies, a significant proportion get messed up.

3> I also "feel", and this may just be me being picky, that Serviio goes out of its way to not interfere with media that can be presented to the target device for
> native rendering on the device so ... I think it would be neater and in keeping and not to preprocess them with ffmpeg wherever possible.

Kind regards

[Illico]

1> about 5% of my "good" MKV files won't remux using ffmpeg due to this ffmpeg bug ...
> ([matroska @ 0000000004768a10] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 2054 >= 2054)
> these files happly remux with MKVMerge but not with ffmpeg.

Could you try adding this on the command line used ?
-copytb 1

Specify how to set the encoder timebase when stream copying.
1 : Use the demuxer timebase. The time base is copied to the output encoder from the corresponding input demuxer. This is sometimes required to avoid non monotonically increasing timestamps when copying video streams with variable frame rate

Could you try adding this on the command line used ?
-copytb 1

Hi Illico, kind of you to join the conversation.

I tried -copytb 1 and incidentally also tried -copytb 0 & -copytb -1
I wasn't sure where to place it on the cmd line so I tried before -i and after
and I'm sad to report that each failed in the identical way

Regards

[zip]

are you using the FFmpeg distributed with Serviio?

are you using the FFmpeg distributed with Serviio?

Hello zip,

In several places within this forum, people reported outstanding ffmpeg issues, so I decided to download and install the latest ffmpeg from the ffmpeg site.

So in answer to your question - No these ffmpeg mux errors were raised when using a later version of ffmpeg than the one shipped with Serviio (see ffmpeg header in console log 1 below).

However, I did see these errors before replacing ffmpeg. So just to make sure that I'm not leading anyone astray here, I have just replaced ffmpeg with the original ffmpeg.exe shipped with Serviio win 0.6.2 and reproduced the problem (see console log 2 below).

Remember, these errors only occur for about 5% of my MKV files - and that this is only an issue we are discussing here, because jhb50 suggested a strategy of remuxing all my mkv files as part of a possible solution to some real-time navigation errors.

And before I leave this post, it would be remiss of me not to thank you for this excellent project.

Thank you and kind regards

  Code:

--------Console log 1 start--------
C:\Program Files\Serviio\lib>ffmpeg
ffmpeg version N-40301-gc1fe2db Copyright (c) 2000-2012 the FFmpeg developers
  built on May  3 2012 11:44:06 with gcc 4.6.3
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-runtime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass --enable-libcelt --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      51. 49.100 / 51. 49.100
  libavcodec     54. 17.101 / 54. 17.101
  libavformat    54.  3.100 / 54.  3.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 72.104 /  2. 72.104
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 11.100 /  0. 11.100
  libpostproc    52.  0.100 / 52.  0.100
Hyper fast Audio and Video encoder
--------Console log 1 end--------


  Code:

--------Console log 2 start--------
Welcome to Convert.
Hit 1 "O:\DVD Imports\0 Unseen\********\********.mkv"
19:45:41 0:00:00.09 Cmd= C:\Program Files\Serviio\lib\ffmpeg.exe -i "O:\DVD Imports\0 Unseen\********\********.mkv" -copytb 1 -map 0 -codec copy  "O:\DVD Imports\0 Test\********\********.MKV"

ffmpeg version 0.9, Copyright (c) 2000-2011 the FFmpeg developers
  built on Dec 13 2011 20:46:11 with gcc 4.4.2
  configuration: --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memalign-hack --enable-libmp3lame --enable-librtmp --extra-libs='-lrtmp -lpolarssl -lws2_32 -lwinmm' --arch=x86 --enable-runtime-cpudetect --enable-pthreads --target-os=mingw32 --cross-prefix=i686-mingw32- --pkg-config=pkg-config
  libavutil    51. 32. 0 / 51. 32. 0
  libavcodec   53. 42. 0 / 53. 42. 0
  libavformat  53. 24. 0 / 53. 24. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 53. 0 /  2. 53. 0
  libswscale    2.  1. 0 /  2.  1. 0

Seems stream 0 codec frame rate differs from container frame rate: 48.00 (48/1) -> 48.00 (48/1)
Input #0, matroska,webm, from 'O:\DVD Imports\0 Unseen\********\********.mkv':
  Metadata:
    title           : ********
  Duration: 01:28:32.30, start: 0.000000, bitrate: 10608 kb/s
    Stream #0:0(fre): Video: h264 (High), yuv420p, 1920x1040, SAR 1:1 DAR 24:13, 23.98 fps, 48 tbr, 1k tbn, 48 tbc (default)
    Metadata:
      title           : Video
    Stream #0:1(fre): Audio: dts (DTS), 48000 Hz, 5.1(side), s16, 1536 kb/s (default)
    Metadata:
      title           : French DTS
    Stream #0:2(fre): Audio: dts (DTS), 48000 Hz, stereo, s16, 768 kb/s
    Metadata:
      title           : Director Commentary
    Stream #0:3(eng): Subtitle: text (default) (forced)
File 'O:\DVD Imports\0 Test\********\********.MKV' already exists. Overwrite ? [y/N] y
Output #0, matroska, to 'O:\DVD Imports\0 Test\********\********.MKV':
  Metadata:
    title           : ********
    encoder         : Lavf53.24.0
    Stream #0:0(fre): Video: h264, yuv420p, 1920x1040 [SAR 1:1 DAR 24:13], q=2-31, 23.98 fps, 1k tbn, 1k tbc (default)
    Metadata:
      title           : Video
    Stream #0:1(fre): Audio: dts, 48000 Hz, 5.1(side), 1536 kb/s (default)
    Metadata:
      title           : French DTS
    Stream #0:2(fre): Audio: dts, 48000 Hz, stereo, 768 kb/s
    Metadata:
      title           : Director Commentary
    Stream #0:3(eng): Subtitle: text (default) (forced)
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (copy)
  Stream #0:3 -> #0:3 (copy)
Press [q] to stop, [?] for help
[matroska @ 03d55680] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 11057 >= 11051
[matroska @ 03d55680] pts < dts in stream 1
av_interleaved_write_frame(): Invalid argument
19:45:50 0:00:09.24 Completed with rc=1
--------Console log 2 end--------


[zip]

FFmpeg distributed with Serviio has a patch for this FFmpeg bug. Obviously they haven't fixed it yet.

FFmpeg distributed with Serviio has a patch for this FFmpeg bug. Obviously they haven't fixed it yet.

Nor do they have any plans to fix it. The trouble is that the ffmpeg devs doesn't consider this a ffmpeg bug, but rather a case of a corrupt video, and as such they have no interrest in 'fixing' it.

My personal workaround is to just ignore the 'non monotonically increasing dts ...'. I've no idea if this may have any bad side-effects, but I haven't noticed any.

  Code:

gong:/usr/ports/multimedia/ffmpeg/work/ffmpeg-0.7.12# diff -c libavformat/utils.c.orig libavformat/utils.c
*** libavformat/utils.c.orig   2012-05-13 10:39:51.000000000 +0200
--- libavformat/utils.c   2012-05-09 11:25:56.000000000 +0200
***************
*** 3197,3203 ****
          av_log(s, AV_LOG_ERROR,
                 "Application provided invalid, non monotonically increasing dts to muxer in stream %d: %"PRId64" >= %"PRId64"\n",
                 st->index, st->cur_dts, pkt->dts);
!         return AVERROR(EINVAL);
      }
      if(pkt->dts != AV_NOPTS_VALUE && pkt->pts != AV_NOPTS_VALUE && pkt->pts < pkt->dts){
          av_log(s, AV_LOG_ERROR, "pts < dts in stream %d\n", st->index);
--- 3197,3203 ----
          av_log(s, AV_LOG_ERROR,
                 "Application provided invalid, non monotonically increasing dts to muxer in stream %d: %"PRId64" >= %"PRId64"\n",
                 st->index, st->cur_dts, pkt->dts);
! //        return AVERROR(EINVAL);
      }
      if(pkt->dts != AV_NOPTS_VALUE && pkt->pts != AV_NOPTS_VALUE && pkt->pts < pkt->dts){
          av_log(s, AV_LOG_ERROR, "pts < dts in stream %d\n", st->index);

[zip]

that's basically what I'm doing