iPhone, iPad, iPod – HTTP Live Streaming (HLS) with free tools on Windows

Apple HTTP Live Streaming (HLS) has been a nightmare to get working. Below, I’ll go through some of my trials and tribulations in getting HLS encoding for non-live streams working in Windows.   In summary, I couldn’t get the bitrate I wanted on my videos.  I’ll provide explanations (and rants) below, but first, for the impatient, here is the solution.

In the end, here is exactly what I did:Watch Justice League vs. Teen Titans (2016) Full Movie Online Streaming Online and Download

  • Download FFMPEG, latest stable release (0.6.1 in my case) Windows build from VideoHelp.com (FFMPEG.org doesn’t do their own builds for some reason?)
  • Download x264 Windows build from x264.nl (once again, the x264 team doesn’t do their own windows builds)
  • Download the Nero AAC Encoder from nero (this is NOT open source)
  • Download the open source segmenter from here (author’s site is here)

Here’s how I have created my streams in a Windows environment:

  1. Extract WAV file from original video:
    ffmpeg.exe -i INPUT.AVI -vn -acodec pcm_s16le AUDIO.WAV
  2. Produce an AAC file:
    neroAacEnc.exe -cbr 64000 -he -if AUDIO.WAV -of AUDIO.AAC
  3. Produce the required audio-only MPEG2-Transport Stream:
    ffmpeg.exe -i AUDIO.AAC -vn -f mpegts -acodec copy AUDIO.TS
  4. Go ahead and segment the audio-only MPEG2-Transport Stream (this can be done later):
    segmenter.exe AUDIO.TS 10 STREAMNAME/AUDIO STREAMNAME/AUDIO.M3U8 http://mydomain.com/virtual/path/to/STREAMNAME/
    Note that this will dump AUDIO-*.TS and AUDIO.M3U8 files into the .\STREAMNAME directory which must exist prior to running.
  5. Choose a bitrate, and encode the video for one of those bitrates. Apple recommends 96 kb/s, 256 kb/s, and 800 kb/s for video. This example will use 96 kb/s (“low” bitrate):
    x264.exe --level 30 --profile baseline --bitrate 96 --keyint 30 -o LOW.MP4 INPUT.AVI
  6. Mux the audio and video together to produce an MPEG2 Transport Stream:
    ffmpeg.exe -i LOW.MP4 -i AUDIO.AAC -f mpegts -vcodec copy -acodec copy -vbsf h264_mp4toannexb LOW.TS
  7. Segment the stream:
    segmenter.exe VIDEO-96KBPS.TS 10 STREAMNAME/LOW STREAMNAME/LOW-TMP.M3U8 http://mydomain.com/virtual/path/to/STREAMNAME/
  8. Optional: Make all URLs relative in STREAMNAME/LOW.M3U8… Apple’s mediastreamvalidator will issue warnings if you don’t do this. I use a sed script:
    sed.exe "s/http\:\/\/mydomain.com\/virtual\/path\/to\/STREAMNAME\///" STREAMNAME/LOW-TMP.M3U8 > LOW.M3U8
  9. Repeat steps 5-8 for Medium (256 kb/s) and High (800 kb/s) bitrates.
  10. Generate the final playlist:
    echo #EXTM3U> STREAMNAME\STREAMNAME.M3U8
    echo #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=320000>> STREAMNAME\STREAMNAME.M3U8
    echo med.m3u8>> STREAMNAME\STREAMNAME.M3U8
    echo #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=864000>> STREAMNAME\STREAMNAME.M3U8
    echo high.m3u8>> STREAMNAME\STREAMNAME.M3U8
    echo #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=160000>> STREAMNAME\STREAMNAME.M3U8
    echo low.m3u8>> STREAMNAME\STREAMNAME.M3U8
    echo #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=64000>> STREAMNAME\STREAMNAME.M3U8
    echo audio.m3u8>> STREAMNAME\STREAMNAME.M3U8

So what’s going on above?  What’s the gist of an HLS stream?  Briefly, setting up a static compliant HLS stream involves the following: Encode your video at varying bitrates. Then slice each of those encoded videos into even chunks (10 seconds recommended) and output a playlist. A master playlist (aka variant playlist) points to each bitrate-playlist. Let the player, based on bandwidth constraints, figure out what bitrate it wants on the fly. If it has to switch to a different bitrate-playlist, it merges segments by matching up the audio (which should be encoded identically between streams). This seems to fit Apple’s general philosophy: your video will stream; it will work regardless of your bandwidth limitations.

A couple of important notes:

  • One benefit: No streaming server is needed to make this work. Any HTTP server will work, including Apache and IIS.
  • As mentioned here, “If your app delivers video over cellular networks, and the video exceeds either 10 minutes duration or 5 MB of data in a five minute period, you are required to use HTTP Live Streaming.” (You have to wonder if any of this is to simply protect AT&T’s bandwidth?)
  • Apple has proposed HLS as a standard.
  • So far as I can tell, Apple doesn’t provide an encoder to produce the needed MPEG-2 Transport Stream, nor does any SINGLE Apple tool to encode and dump out the necessary files for HLS. They do give you command-line tools for segmenting, creating a variant playlist (aka master-playlist), and verifying streams. This seems like a glaring omission from QuickTime Pro honestly.  It does seem from forum posts that QuickTime Pro can output a format that the Apple’s  segmenter (a command line tool on MacOS) can deal with though.
  • It’s all standards based (x264, AAC/MP3, MPEG2-TS) with recommendations for settings provided in their FAQ. So we should be able to cobble the necessary files together from free tools that run on Windows. Almost all of them are open source…

So off I went, in an attempt to figure all this out from available, free tools. I came across a very helpful article at www.ioncannon.net, iPhone HTTP Streaming with FFMpeg and an Open Source Segmenter and initially got a stream up and running quite quickly.  But the bitrates of the resultant streams just didn’t come close to the bitrates I wanted.  (Knowing how HLS works, it seems it would be fairly important to have bitrates that come close to what is advertised in the main playlist.)  I spent the next few days trying my best to get FFMPEG to cooperate, but to no avail… on a 150 kbit/s stream, it produced 220 kbit/s… on a 800 kbit/s stream, it produced a 250 kbit/s stream. On top of this, if I told FFMPEG to use AAC, the audio sounded horrible (MP3 was much better at the same bitrate!)

After days of experimentation, I finally got the results I wanted from free tools for creating HTTP Streams for use with iPhone, iPad, iPod. The information out there on how to get these tools to do what you want is pretty horrible. Especially for the video processing layman like myself, who “just wants it to work!”

In summary, FFMPEG fails on two fronts here:

  • FFMPEG is horrible at AAC audio encoding. The resultant audio sounds horrible. The free NeroAacEnc does a much better job.  Yes, we could use MP3, but AAC supposedly provides better sound at similar compression rates versus MP3.
  • FFMPEG simply doesn’t (by any of my tests) come close to honoring a recommended bitrate for x264 video encoding.  I have tried suggestions from  the FFmpeg x264 encoding guide, including two pass encoding, but to no avail.  Video bitrates were simply way off.

So I am using a combination of executables: ffmpeg (extract audio for Nero), Nero AAC Encoder (audio encoding), x264 (video encoding), ffmpeg (mux audio/video to create TS file), and an open source segmenter (to create m3u8 for each bitrate). The resulting output looks great, and at least comes pretty close to desired bitrate… finally!

Hope this helps someone out there!

18 Comments on “iPhone, iPad, iPod – HTTP Live Streaming (HLS) with free tools on Windows

  1. What resolution/frame rate is your source? If it’s anything but something small (say, 320×240 or less), encoding _any_ video but the lowest resolution at 96kbps is difficult as you likely reach the limit of the codec’s compression. For any given clip, if you want to see the limit of compression with x264, set the quantiser to 51 and use that as the rate control method. Whatever output bit rate you observe is the maximum compression that x264 can achieve. Note that slower presets will improve compression at the cost of encoding speed.

    Also note that in FFmpeg, all bit rate values are in bits per second. You can express 96kbps as either ‘96000’ or ’96k’.

    The FFmpeg AAC encoder is known to be bad quality in the past releases. It is getting better but it’s still not good enough really and needs significant further development.

    If you have specific questions about x264, you can get support on IRC in #x264 on irc.freenode.net.

    • @Rob:
      Thanks for the pointers.

      I *could* get FFMPEG to come close to the requested bitrate by fiddling with the quantizer (qmin to be exact). And yes, with the quantizer, I was able to reach my target bitrate or lower…. so it appears that my video dimensions may not be a factor. Changes in movement within the video will increase bitrate, and unless I misunderstand, a quantizer setting isn’t going to take that into account.

      In summary, FFMPEG doesn’t appear to automatically adjust quality based on a bitrate target, on the fly, as good as using x264 directly. FFMPEG seemed to take a hint from the bitrate, as it did make some difference in the file size, but never really honored the bitrate as well as the x264 executable. I tried many different things, including two pass encoding (which I believe your encoding guide recommends for constant bitrate), as well as the ioncannon site mentioned above. I mostly specified bitrate as “-b 96k” as an example.

  2. Thought I’d add this as a comment to help others figure a few things out. (I used your commands above as a starting point).

    The above is great, except for one thing. Lots of intermediate files, lots of seperate commands.

    For the Audio Encode, instead of 2 steps.

    ffmpeg.exe -i INPUT.AVI -vn -f wav -acodec pcm_s16le – | neroAacEnc.exe -if – -cbr 64000 -he -ignorelength -of AUDIO.AAC

    Just a single step using Piped output from FFMPEG to NeroAAC
    (-ignorelength is used in the AAC encoder) – An alternative to Nero, is QAAC, which uses the quicktime encoder installed if you have quicktime installed. (if you have iTunes, you have this) https://sites.google.com/site/qaacpage/
    Command line parameters will be different.

    For the video,
    I personally needed to do things like skip seconds and frames and I wanted to do it consistently. (so same command to skip seconds for the audio and for the video)
    So I did piped output for x264 from ffmpeg. Again based off your command above.

    ffmpeg.exe -i INPUT.AVI -an -f yuv4mpegpipe – | x264.exe – –stdin y4m –level 30 –profile baseline –bitrate 96 –keyint 30 -o OUTPUT.MP4

    The required magic in the above:
    -f yuv4mpegpipe means you don’t need *much* extra in the x264 command (raw doesn’t include things like resolution etc, the pipe above does I think)
    for x264, –stdin y4m ensures it gets the right details in from the PIPE input.

    For people new to piped methods:
    At the end of ffmpeg I use “-” for the output file name,
    This tells it to pass the output to the next program.
    the “|” represents the next program.
    and in x264 instead of specifying the input file, I use “-” again to say: “Your input file comes from the previous program”.

    So don’t delete any random -‘s or the | from the above, or it won’t work.

    – Pete

    • More fun with pipes?

      2 commands. Audio, then video. You can run the video command as many times as you want.
      Make sure you change the segmenter output filenames!

      This means you only end up with the segmented TS files.
      One issue I had was the -vbsf mpeg4toannexb Had to drop that from the video encode command.
      (I haven’t tested the final pipe to the segmenter in this configuration, but it outputs the unsegmented TS files for me fine otherwise – and my past experience with the segmenter has worked with a pipe)

      Create the audio:

      ffmpeg.exe -i INPUT.AVI -vn -f wav -acodec pcm_s16le – | neroAacEnc.exe -if – -cbr 64000 -he -ignorelength -of AUDIO.AAC

      Then use the Audio, mux it with the video, output it all to the segmenter in segmented files. No intermediate files except the audio (which you needed in the first place for the lowest stream!)

      ffmpeg.exe -i INPUT.AVI -an -f yuv4mpegpipe –
      | x264.exe – –stdin y4m –level 30 –profile baseline –bitrate 96 –keyint 30 -o –
      | ffmpeg.exe -y -i – -i AUDIO.AAC -f mpegts -vcodec copy -acodec copy –
      | segmenter.exe – 10 STREAMNAME/LOW STREAMNAME/LOW-TMP.M3U8 http://mydomain.com/virtual/path/to/STREAMNAME/

      if you are really creative you can even use the “echo” command in a batch file and stdout redirection to create the overarching m3u8 file (the one that joins the different bitrate streams)

      You can automate a lot of the process using dos…

  3. This is an excellent article! I followed and it works. My videos are on-demand. I do not see end time in video player in iPhone and it shows as LIVE!

    Please help!

  4. Finally We figured! The end tag was missing from M3U8 file which made Apple player think it is LIVE!

    Thank you for your support.

    BTW, do you know how to set screen size (iphone: 480 by 320) so the HLS is set for iPhone and for iPad it is 680 by 320.

  5. This creates static TS and m3u8 files for Apple HLS. If I want to streamline the work from encoder to a storage, say, ingest the ts/m3u8 while they are being generated by the pipeline process, to an internet storage. Is there anyway to do it now?

    • I believe FFMPEG can read from a steam, and yes, there is a way to get these to work using pipes. But you may end up relying on FFMPEG’s encoding, rather than using x264 / NeroAAC… FFMPEG will definitely work, but it doesn’t do a great job giving you the bitrate you ask for. See above comments from PeterA on using pipes.

      I’m pretty sure Wowza 3, which just came out, supports this out of the box, but it isn’t free.

      Good luck!

  6. step number 10:
    echo #EXTM3U> STREAMNAME\STREAMNAME.M3U8
    echo #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=320000>> STREAMNAME\STREAMNAME.M3U8
    echo med.m3u8>> STREAMNAME\STREAMNAME.M3U8
    echo #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=864000>> STREAMNAME\STREAMNAME.M3U8
    echo high.m3u8>> STREAMNAME\STREAMNAME.M3U8
    echo #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=160000>> STREAMNAME\STREAMNAME.M3U8
    echo low.m3u8>> STREAMNAME\STREAMNAME.M3U8
    echo #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=64000>> STREAMNAME\STREAMNAME.M3U8
    echo audio.m3u8>> STREAMNAME\STREAMNAME.M3U8
    How i can apply it?
    ‘#EXTM3U’ is not recognized as an internal or external command,
    operable program or batch file.

  7. Anyone have a lead on an automated process for this? While doable using the command line, I have loads of video that will need to be processed, and would greatly like to save some time.

  8. Thank you very much for this post, which helped me a lot.

    I don’t know whether you ran across it, but qtaacenc.exe, which is a command-line encoder, is a workable option. It relies on an installation of QuickTime and/or iTunes, with Apple Application Support. I ran across this option, with a user’s opinion that Apple’s aac encoding is equal or superior to Nero, here:

    http://ubuntuforums.org/showpost.php?p=12228107&postcount=6

    I got piped output from a .mp4 file, via ffmpeg, to encode with the following command: (adapting PeterA’s approach–by the way, PeterA, I’ve run across many explanations of piping, which have all always been difficult to understand, and yours is the *only* clear explanation I’ve ever read about it–thank you!)

    ffmpeg -i ep2c1Logo.mp4 -f wav – | qtaacenc – test.m4a

    I assume that could be further adapted via mp4box etc. to mux the m4a stream (either through an intermediate file, or through another pipe?) with a video stream into an .mp4 container.

    I don’t know the licensing terms for Apple’s aac encoder, but I hope it’s more liberal than Nero’s. The latter’s license agreement says you may not use Nero aac for commercial purposes. I suppose I’ll look up the Apple aac encoder’s license terms . . .

Leave a Reply to Robert Swain Cancel reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.