Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support MPEG2-TS media #33

Open
nedmundwong opened this issue Feb 1, 2019 · 10 comments
Open

Support MPEG2-TS media #33

nedmundwong opened this issue Feb 1, 2019 · 10 comments
Labels
type: enhancement New feature or request
Milestone
v1.1

Comments

@nedmundwong
Copy link

I'm trying to use the sample_xcode_project.xcodeproj to play some HLS videos with TS fragment. But after trying with many HLS testing videos (e.g. https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8
), the player keep showing : Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()

However, when using the browser version of Shaka Player

(https://shaka-player-demo.appspot.com/demo/#asset=https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8;lang=en-US;videoRobustness=http://192.168.18.193:8111/playlist.m3u8;audioRobustness=http://192.168.18.193:8111/playlist.m3u8;build=uncompiled)

The HLS video can be played. Hence I'm not sure if the problem is related to the player or that particular m3u8 file format is not supported. If the problem is related to the latter one, could you please provide some m3u8 files that can be played for this player. Thanks in advance!
@TheModMaker
Copy link
Collaborator

TheModMaker commented Feb 1, 2019 •
edited

We currently only support MP4 and WebM (WebM isn't added by default on iOS). We usually just forward things to FFmpeg, so there is a slim chance of it working somewhat by just adding a flag to configure:

./configure --containers mov,mpegts

You'll need to edit the build.sh file in the Xcode project and delete the ShakaPlayerEmbedded.framework folder to get it work in the sample project.

The HLS+MP4 Angel One clips should work fine: https://storage.googleapis.com/shaka-demo-assets/angel-one-hls/hls.m3u8

@TheModMaker TheModMaker added the type: enhancement New feature or request label Feb 1, 2019
@nedmundwong
Copy link
Author

Many thanks for your prompt reply! The clip that you provide works fine.

I've tried your method by adding flag to configure , and changed the framework in the .xcodeproj
Although the error Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER () is still shown, but by checking the log, it seems something different:

When loading (http://184.72.239.149/vod/smil:BigBuckBunny.smil/playlist.m3u8)

[Log]: "Unload latency:" 0.004
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="560000",RESOLUTION="560x320",URI="chunklist_w847761226_b560000.m3u8""
[Log]: "Guessing multiplexed audio+video."
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="280000",RESOLUTION="280x160",URI="chunklist_w847761226_b280000.m3u8""
[Log]: "Guessing multiplexed audio+video."
[Log]: "First segment" "media_w847761226_b560000_0.ts" "starts at" 0
[Log]: "First segment" "media_w847761226_b280000_0.ts" "starts at" 0
[Log]: "Dropping variant - video not compatible with platform" "type=video codecs=avc1.42E01E,mp4a.40.2 bandwidth=undefined frameRate=undefined width=560 height=320"
[Log]: "Dropping variant - video not compatible with platform" "type=video codecs=avc1.42E01E,mp4a.40.2 bandwidth=undefined frameRate=undefined width=280 height=160"
[Log]: "load() failed:" {category:4, code:4032, data:[...], handled:false, message:"Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()", severity:2, stack:"Error@ ...

When using previous framework, in the Log file, the player cannot even find the first ts segment. After changing the framework, it can detect the ts and try to load it. However, when the player loading the ts, it always prompt the error "Dropping variant - video not compatible with platform" "type=video codecs=avc1.42E01E,mp4a.40.2 bandwidth=undefined frameRate=undefined width=560 height=320" where only the width and height is loaded correctly.

Does it mean there is no chance to play TS on this embedded player?

@TheModMaker
Copy link
Collaborator

That content is multiplexed, meaning the media files contain both audio and video streams. Shaka Player doesn't officially support that since browser support is inconsistent and not really aligned with streaming content where audio and video are usually separate. But Shaka Player will just add the segments and leave it to the browser.

Shaka Player Embedded doesn't support multiplexed content at all. You'll need to convert your content to have audio and video in separate streams. We could add support for multiplexed content at some point, but it breaks a lot of assumptions in the media pipeline now.

@TheModMaker TheModMaker mentioned this issue Feb 1, 2019
Open
@TheModMaker TheModMaker changed the title Cannot play HLS with TS fragment Support MPEG2-TS media Feb 1, 2019
@nedmundwong
Copy link
Author

Thank you @TheModMaker for the explanation. And I've tried to separate the content to have audio and video in separate streams. But seems the player still got into some problems:

I've followed the tutorial, and tried to use the shaka packager to encode the video content. Here are the basic information:

2 single track TS streams (1 video, 1 audio)

Shaka Packager Command:

packager \
  'in=small.mp4,stream=audio,segment_template=$Number$a.ts,playlist_name=maina.m3u8,hls_group_id=audio,hls_name=ENGLISH' \
  'in=small.mp4,stream=video,segment_template=$Number$v.ts,playlist_name=mainv.m3u8' \
  --hls_master_playlist_output h264_master.m3u8

Result: Error

[Log]: "Unload latency:" 0.004
[Log]: "First segment" "1a.ts" "starts at" 0.1
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="646562",AVERAGE-BANDWIDTH="646562",CODECS="avc1.42c01e,mp4a.40.2",RESOLUTION="560x320",AUDIO="audio",URI="mainv.m3u8""
[Log]: "Guessing video."
[Log]: "First segment" "1v.ts" "starts at" 0.1
[Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=1"
[Log]: "load() failed:" {category:4, code:4032, data:[...], handled:false, message:"Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()", severity:2, stack:"Error@...

1 single track TS stream (1 video)

packager \
  'in=small.mp4,stream=video,segment_template=$Number$v.ts,playlist_name=mainv.m3u8' \
  --hls_master_playlist_output h264_master.m3u8

Result: Error

[Log]: "Unload latency:" 0.005
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="505562",AVERAGE-BANDWIDTH="505562",CODECS="avc1.42c01e",RESOLUTION="560x320",URI="mainv.m3u8""
[Log]: "Guessing video-only."
[Log]: "First segment" "1v.ts" "starts at" 0.1
[Log]: "Dropping variant - video not compatible with platform" "type=video codecs=avc1.42c01e bandwidth=undefined frameRate=undefined width=560 height=320"
[Log]: "load() failed:" {category:4, code:4032, data:[...], handled:false, message:"Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()", severity:2, stack:"Error@...

2 groups of streams (each with an init segment and a series of media segments) (MP4 Output)

Shaka Packager Command:

packager \
  'in=small.mp4,stream=audio,init_segment=audio/init.mp4,segment_template=audio/$Number$.m4s,playlist_name=audio/main.m3u8,hls_group_id=audio,hls_name=ENGLISH' \
  'in=small.mp4,stream=video,init_segment=h264_480p/init.mp4,segment_template=h264_480p/$Number$.m4s,playlist_name=h264_480p/main.m3u8,iframe_playlist_name=h264_480p/iframe.m3u8' \
  --hls_master_playlist_output h264_master.m3u8

Result: Played Successfully

2 single file MP4 streams (1 video, 1 audio)

packager \
  in=small.mp4,stream=audio,output=audio.mp4,playlist_name=audio.m3u8,hls_group_id=audio,hls_name=ENGLISH \
  in=small.mp4,stream=video,output=h264_480p.mp4,playlist_name=h264_480p.m3u8 \
  --hls_master_playlist_output h264_master.m3u8

Result: Played Successfully

From the above results, we can see that TS streams still could not be played after separating audio and video or only playing the video stream. The contents that could be played are m4s in m3u8 and the single mp4 streams. However, when I tried to play the above streams from the shaka player demo website, all streams could be played successfully. I'm not sure if I used the shaka packager with the wrong command or if the player could not support separated audio and video streams with TS as segments.

@TheModMaker
Copy link
Collaborator

One thing I can think of is the video is too small to get hardware decoding. By default on iOS, we'll only play videos we can get hardware decoding on. (I also just realized there isn't a flag to disable force hardware decoding, but there is a flag to enable it) So try adding the --gn-args force_hardware_decode=false flag. I think you can also set the environment variable GLOG_v=1 when running to get verbose logging.

shaka-bot pushed a commit that referenced this issue Feb 5, 2019
@TheModMaker
Add flag to not force hardware decoding.
dc1b921 
On iOS we default to force hardware decoding.  We have a flag to enable
forcing hardware decoding, but no flag to not force hardware decoding.

Issue #33

Change-Id: I7999fabbbb9d563799cc31a3651d8b38458ff5b8
@nedmundwong
Copy link
Author

I've used the flag to configure and build the framework again. After added the environment variable, more information was provided for the above issue:

When playing this HLS (https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8), still prompted the same error Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()
The Log is as follow:

[Log]: "Unload latency:" 0.004
[Log]: "First segment" "segment_0.ts" "starts at" 0
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="628000",CODECS="avc1.42c00d,mp4a.40.2",RESOLUTION="320x180",AUDIO="audio",URI="f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_180_250000.m3u8""
[Log]: "Guessing video."
[Log]: "First segment" "segment_0.ts" "starts at" 0
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="1728000",CODECS="avc1.42c00d,mp4a.40.2",RESOLUTION="640x360",AUDIO="audio",URI="f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_360_800000.m3u8""
[Log]: "Guessing video."
[Log]: "First segment" "segment_0.ts" "starts at" 0
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="928000",CODECS="avc1.42c00d,mp4a.40.2",RESOLUTION="480x270",AUDIO="audio",URI="f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_270_400000.m3u8""
[Log]: "Guessing video."
[Log]: "First segment" "segment_0.ts" "starts at" 0
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="2528000",CODECS="avc1.42c00d,mp4a.40.2",RESOLUTION="960x540",AUDIO="audio",URI="f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_540_1200000.m3u8""
[Log]: "Guessing video."
[Log]: "First segment" "segment_0.ts" "starts at" 0
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="4928000",CODECS="avc1.42c00d,mp4a.40.2",RESOLUTION="1280x720",AUDIO="audio",URI="f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_720_2400000.m3u8""
[Log]: "Guessing video."
[Log]: "First segment" "segment_0.ts" "starts at" 0
[Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH="9728000",CODECS="avc1.42c00d,mp4a.40.2",RESOLUTION="1920x1080",AUDIO="audio",URI="f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_1080_4800000.m3u8""
[Log]: "Guessing video."
[Log]: "First segment" "segment_0.ts" "starts at" 0.08
[Log]: "First segment" "segment_0.ts" "starts at" 0.08
[Log]: "First segment" "segment_0.ts" "starts at" 0.08
[Log]: "First segment" "segment_0.ts" "starts at" 0.08
[Log]: "First segment" "segment_0.ts" "starts at" 0.08
[Log]: "First segment" "segment_0.ts" "starts at" 0.08
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0208 11:15:07.781924 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported
[Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null"
I0208 11:15:07.782481 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported
[Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null"
I0208 11:15:07.782693 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported
[Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null"
I0208 11:15:07.782980 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported
[Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null"
I0208 11:15:07.783174 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported
[Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null"
I0208 11:15:07.783499 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported
[Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null"
[Log]: "load() failed:" {category:4, code:4032, data:[...], handled:false, message:"Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()", severity:2, stack:"Error@/Users/...

After several trials with different HLS sources, it keeps showing: media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported, after some Googling in the Internet, it seems related to media_utils.cc. Not sure if the shaka player requires to check browser compatibility with the ts files before playing.

@TheModMaker
Copy link
Collaborator

Note that there is no browser here (or more accurately Shaka Player Embedded is the browser). You are correct about a "media_utils" file, but it is actually here. I looked a bit at this and we definately will need to make some changes for this to work. There is some weirdness about segment appending that we'll need to fix.

@TheModMaker TheModMaker added this to the v1.1 milestone Sep 26, 2019
@Romantic-LiXuefeng
Copy link

@TheModMaker Is there any plan to support this issue?

@TheModMaker
Copy link
Collaborator

We do plan on supporting it eventually, but we're not working on it yet. This is in the v1.1 milestone, so we'll start working on it after the v1.0 release is done.

@YutaMoriJP
Copy link

YutaMoriJP commented Dec 26, 2023 •
edited

@TheModMaker I was having issues with MPEG2-TS and wasn't sure whether Shaka Player now supports it or not. Is mux.js needed or can it be used without it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: enhancement New feature or request
Projects
None yet
Milestone
v1.1
Development

No branches or pull requests
4 participants
@TheModMaker @YutaMoriJP @Romantic-LiXuefeng @nedmundwong