Serviio

DLNA media server
https://serviio.org/forum/

[Patch] HD stream selection in HTTP Live Streaming (HLS)

https://serviio.org/forum/viewtopic.php?f=3&t=18897

Page 1 of 1

[Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Thu May 21, 2015 2:32 am
by stefana888
I am utilizing onlive live content in HLS where the manifest file contains multiple streams with different bandwiths (programs). (The url to a single stream can not be extracted by hand as far as I know from the current type of manifest, see example.) Serviio then defaults to the first video and audio streams it finds which isn't very flexible. In fact, it will probably be a "worst" approach always returning the streams with the lowest bandwith. A "best" approach would be returning the streams with the hightest assumed bandwith i.e. in HD-format. The following modification manages that;

In org.serviio.library.metadata.FFmpegMetadataRetriever

  Code:
private static Map<String, Object> getParametersMap(List<String> ffmpegMediaDescription)
          throws InvalidMediaFormatException
          {

 . . .

             List<EmbeddedSubtitles> subtitlesList = new ArrayList();

             String[] array = ffmpegMediaDescription.toArray(new String[ffmpegMediaDescription.size()]);
              int i = 0;
              while (i < array.length)
               {
                   String line = array[i].trim();
                   int inputPos = line.indexOf("Input #0");

 . . .

                           parameters.put("bitrate", bitrate);
                           }
                         }
                       }
                     }
                      else if (line.startsWith("Program"))
                      {
                        int j = i + 1;
                        while (j < array.length)
                        {
                           if (array[j].trim().startsWith("Program"))
                           {
                             i = j - 1;
                             break;
                           }
                           j++;
                        }
                      }
                      else if ((line.startsWith("Stream")) && (line.indexOf("Video:") > -1) && (parameters.get("video_codec") == null))
       
 . . .

               subtitlesList.add(new EmbeddedSubtitles((Integer)streamInfo.getValueA(), sCodec, (String)streamInfo.getValueB(), defaultLanguage));
               }
             }
           }
           i++;
         }
         parameters.put("embedded_subtitles", subtitlesList);
         return parameters;

. . .


(Example manifest)

  Code:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",FORCED=NO,AUTOSELECT=YES,URI="subtitlelist_leng_b110000.m3u8",LANGUAGE="eng"
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=110000,RESOLUTION=320x180,SUBTITLES="subs"
chunklist_b110000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=64000,CODECS="mp4a.40.2",SUBTITLES="subs"
chunklist_b64000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=268000,RESOLUTION=320x180,SUBTITLES="subs"
chunklist_b268000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=668000,RESOLUTION=640x360,SUBTITLES="subs"
chunklist_b668000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1000000,RESOLUTION=880x480,SUBTITLES="subs"
chunklist_b1000000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1800000,RESOLUTION=1024x576,SUBTITLES="subs"
chunklist_b1800000.m3u8

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Mon May 25, 2015 6:24 pm
by zip
I think there is no given order of the items in the m3u file, so picking first / last will not make it better.

In general, Serviio will picj the first video stream and audio stream that FFmpeg suggests. Supporting playback / transcode of any other video stream that the first would need further updates in the app, i think.

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Mon May 25, 2015 7:12 pm
by stefana888
Take a look at this ffmpeg output. As one can see higher bitrates here have higher 'Program #'. The suggested code takes the streams with the highest 'Program #'. The ffmpeg output corresponds to the m3u8 manifest stream order. Actually they seem standardized in order from low to high stream bitrates.

Who can live with Serviio producing a 320 x 180 video format when there is a 1024 x 576 video format available? The idea works for all my media, not only HLS online content.

  Code:
Input #0, hls,applehttp, from 'http://xxxxxxxxx.net/xxxxxxxxxx/c51eb4af0c90422baad58832447f31de/manifest.
m3u8':
  Duration: N/A, start: 13730.119344, bitrate: N/A
  Program 0
    Metadata:
      variant_bitrate : 110000
    Stream #0:0: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 110000
    Stream #0:1: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 320x180 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 9
0k tbn, 50 tbc
    Metadata:
      variant_bitrate : 110000
    Stream #0:2: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 67 kb/s
    Metadata:
      variant_bitrate : 110000
  Program 1
    Metadata:
      variant_bitrate : 64000
    Stream #0:3: Audio: aac (LC), 44100 Hz, stereo, fltp, 65 kb/s
    Metadata:
      variant_bitrate : 64000
  Program 2
    Metadata:
      variant_bitrate : 268000
    Stream #0:4: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 268000
    Stream #0:5: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 320x180 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 9
0k tbn, 50 tbc
    Metadata:
      variant_bitrate : 268000
    Stream #0:6: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 67 kb/s
    Metadata:
      variant_bitrate : 268000
  Program 3
    Metadata:
      variant_bitrate : 668000
    Stream #0:7: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 668000
    Stream #0:8: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 25 tbr, 90k tbn,
50 tbc
    Metadata:
      variant_bitrate : 668000
    Stream #0:9: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 59 kb/s
    Metadata:
      variant_bitrate : 668000
  Program 4
    Metadata:
      variant_bitrate : 1000000
    Stream #0:10: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 1000000
    Stream #0:11: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 880x480 [SAR 1:1 DAR 11:6], 25 tbr, 90k tbn,
 50 tbc
    Metadata:
      variant_bitrate : 1000000
    Stream #0:12: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 117 kb/s
    Metadata:
      variant_bitrate : 1000000
  Program 5
    Metadata:
      variant_bitrate : 1800000
    Stream #0:13: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 1800000
    Stream #0:14: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1024x576 [SAR 1:1 DAR 16:9], 25 tbr, 90k tbn, 50
 tbc
    Metadata:
      variant_bitrate : 1800000
    Stream #0:15: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 117 kb/s
    Metadata:
      variant_bitrate : 1800000
At least one output file must be specified

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Tue May 26, 2015 12:17 pm
by jhb50
I have always found the resolution in increasing order as well.

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Thu May 28, 2015 1:10 pm
by zip
Ok, created a ticket: https://bitbucket.org/xnejp03/serviio/i ... deo-stream

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Fri Jul 03, 2015 6:33 pm
by stefana888
@jhb50's bitbucket comment

https://developer.apple.com/library/ios ... MENDATIONS

"First bit rate should be one that most clients can sustain", for testing purposes.
Thus it is not likely that the first stream have the highest bitrate. An exception as in jhb50's bitbucket comment shouldn't decide the rule...

It is no problem for me, I can patch Serviio class files myself. But what about a potential customer with demands in live streaming...

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Sat Jul 04, 2015 12:01 am
by jhb50
Why not alter the patch to select the stream based on the "Preferred Online Quality" set in the console rather than the highest.
Until these options are expanded to separate 1080 and 720, high should select 720 since 1080 overwhelms many systems that are capable of 720.
Perhaps you can post a patch to allow XHigh,High,Medium and Low, or even better add the values, 1080,720,480,360,240 to the existing ones which would provide compatibility.

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Sat Jul 04, 2015 4:06 pm
by stefana888
jhb50 wrote:Why not alter the patch to select the stream based on the "Preferred Online Quality" set in the console rather than the highest.
Until these options are expanded to separate 1080 and 720, high should select 720 since 1080 overwhelms many systems that are capable of 720.
Perhaps you can post a patch to allow XHigh,High,Medium and Low, or even better add the values, 1080,720,480,360,240 to the existing ones which would provide compatibility.
It would no longer be a simple patch to introduce a complete new feature that needs an alternate FFmpegMetadataRetriever. But I agree something is definitely missing here when multiple resolutions are available. Only picking the first stream does not satisfy any reasonable needs in http live streaming.

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Sun Jul 05, 2015 4:27 am
by stefana888
(Changed my last post.)

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Sat Jul 11, 2015 10:18 am
by zip
The preferred quality is there for the outgoing quality / bitrate (to enable streaming for users with lower upstream bandwidth).

It might have to be a different config setting altogether, or use a -D system property for now.

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Sat Jul 11, 2015 12:55 pm
by jhb50
Am I missing something? I assumed you wanted to pick the hls stream based on the users preferred outgoing quality... so is not the console setting the value to use just like we pick the youtube stream quality? I don't understand why a new setting would be required or why this would be a complete new feature that needs an alternate FFmpegMetadataRetriever?

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Sun Jul 12, 2015 8:26 pm
by stefana888
A -D system property would be the easiest way, toggling if the patch functionality (test for 'Program') is applied. Else I don't see how completely rewrite FFMpegMetadataRetriever to tie it to the console is avoided.

I will deliver the source code to Zip for consideration.

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Sun Jul 12, 2015 11:11 pm
by jhb50
Here is a stream where all Programs are 640x360 but different bitrates. I don't understand why there are 2 Programs for each bitrate with the second not showing fps and a different audio bit rate. I guess these would need to be handled too but I don't know which Program to suggest be used.
NHK provides high,mid and low streams and uses the bitrates for adaptive streaming.
  Code:
Input #0, hls,applehttp, from 'http://www3.nhk.or.jp/nhkworld/app/tv/stream/global/live_high.m3u8':
  Duration: N/A, start: 78198.375189, bitrate: 423 kb/s
  Program 0
    Metadata:
      variant_bitrate : 1368400
    Stream #0:0: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1368400
    Stream #0:1: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 1368400
    Stream #0:2: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 1368400
 
 Program 1
    Metadata:
      variant_bitrate : 1368400
    Stream #0:3: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1368400
    Stream #0:4: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 1368400
    Stream #0:5: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 1368400
 
Program 2
    Metadata:
      variant_bitrate : 990000
    Stream #0:6: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 990000
    Stream #0:7: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 990000
    Stream #0:8: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 990000
 
Program 3
    Metadata:
      variant_bitrate : 990000
    Stream #0:9: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 990000
    Stream #0:10: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 990000
    Stream #0:11: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 990000
 
 Program 4
    Metadata:
      variant_bitrate : 660000
    Stream #0:12: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 660000
    Stream #0:13: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 660000
    Stream #0:14: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 660000
 
 Program 5
    Metadata:
      variant_bitrate : 660000
    Stream #0:15: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 660000
    Stream #0:16: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 660000
    Stream #0:17: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 660000

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

PostPosted: Mon Jul 13, 2015 5:45 am
by stefana888
jhb50 wrote:Here is a stream where all Programs are 640x360 but different bitrates. I don't understand why there are 2 Programs for each bitrate with the second not showing fps and a different audio bit rate. I guess these would need to be handled too but I don't know which Program to suggest be used.
NHK provides high,mid and low streams and uses the bitrates for adaptive streaming.
  Code:
Input #0, hls,applehttp, from 'http://www3.nhk.or.jp/nhkworld/app/tv/stream/global/live_high.m3u8':
  Duration: N/A, start: 78198.375189, bitrate: 423 kb/s
  Program 0
    Metadata:
      variant_bitrate : 1368400
    Stream #0:0: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1368400
    Stream #0:1: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 1368400
    Stream #0:2: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 1368400
 
 Program 1
    Metadata:
      variant_bitrate : 1368400
    Stream #0:3: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1368400
    Stream #0:4: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 1368400
    Stream #0:5: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 1368400
 
Program 2
    Metadata:
      variant_bitrate : 990000
    Stream #0:6: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 990000
    Stream #0:7: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 990000
    Stream #0:8: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 990000
 
Program 3
    Metadata:
      variant_bitrate : 990000
    Stream #0:9: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 990000
    Stream #0:10: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 990000
    Stream #0:11: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 990000
 
 Program 4
    Metadata:
      variant_bitrate : 660000
    Stream #0:12: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 660000
    Stream #0:13: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 660000
    Stream #0:14: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 660000
 
 Program 5
    Metadata:
      variant_bitrate : 660000
    Stream #0:15: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 660000
    Stream #0:16: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 660000
    Stream #0:17: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 660000


Sparate playlists for high, medium, low bitrates seem uncommon. They tend to bunch them all in a single ordered playlist. Often from low to high.
All times are UTC
Page 1 of 1
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/