Proposed Encoders / Decoders Libraries

[jeromewu]

So far only following encoders / decoders libraries are supported:

Update at 2020/11/24

Video

• mp4 (x264)
• mp4 (x265)
• webm (libvpx)
• ogv (theora)
• mpeg-1 (native)
• mpeg-2 (native)
• aom (av1 / avif) (only in next version, see comments below)

Audio

• mp3 (lame)
• aac (fdk-aac)
• wav/wv (wavpack)
• ogg (vorbis)
• opus (opus)
• flac (native)

Image

• gif (native)
• webp (libwebp)

Others

• font files (freetype)
• *.srt, *.ass subtitle files (libass) (sample code HERE)

---

If you there is any library you would like to add:

• If it already exists in the replies, simply give an emoji to upvote
• If it doesn't exist, you can reply this issue to propose.

I will check this issue periodically to decide which library to integrate next, or identify which is NOT possible to integrate at the moment.

[jeromewu]

RTSP

[jeromewu]

x265 / h265

[cyberquarks]

mpeg-1

[grkblood13]

mpeg-2

[CrypticSignal]

AAC and WAV

• AAC is a common audio codec for videos, and probably the next widest supported audio codec after MP3.
• WAV is a common lossless audio format.

I don't know how WASM works, but here's how you usually use encode to AAC or WAV with FFmpeg:
AAC:
-c:a libdfdk_aac (requires FFmpeg to be compiled with fdk-aac) or -c:a aac (the native FFmpeg AAC encoder). lidfdk_aac is preferred as it's a higher quality AAC encoder.
This may be useful: https://trac.ffmpeg.org/wiki/Encode/AAC
WAV:
-c:a pcm_s16le, -c:a pcm_s24le or -c:a pcm_s32le, where 16, 24 and 32 denote the output audio bit depth.

[polytropoi]

Ogg please! It's important for game audio, because mp3 doesn't loop properly.

[samwatkinson1]

AV1

[Cobertos]

gif

[smashah]

libwebp

It would be helpful if you could provide some instructions on how to implement these libs so we can make some PRs

[jeromewu]

Hi all, for the latest version (v0.9.3), more libraries supported has been added. Please check if it helps in your project.

@smashah It would be great if you can send PRs, for integrating new libraries, you can check this repo: https://github.com/ffmpegwasm/ffmpeg.wasm-core.

Also you can check this series of posts to learn more foundation:

• https://jeromewu.github.io/build-ffmpeg-webassembly-version-part-1-preparation/
• https://jeromewu.github.io/build-ffmpeg-webassembly-version-part-2-compile-with-emscripten/
• https://jeromewu.github.io/build-ffmpeg-webassembly-version-part-3-v0.1/
• https://jeromewu.github.io/build-ffmpeg-webassembly-version-part-4-v0.2/

[mathiasrw]

Thank you so much for creating this!

AVIF please! The AV1 standard is gaining support in the industry and is also used as a single frame "film" as an image format for the web: AVIF. Chrome is supporting it, cloudflare is supporting it and on https://blog.cloudflare.com/generate-avif-images-with-image-resizing/ Cloudflare gives a list of good reasons why both Intel and Apple is jumping on the AVIF (single frame AV1) wagon.

[jeromewu]

For AV1, in fact I have successfully integrated into ffmpeg.wasm, but there are two main issues:

1. It takes more than a minute to transcode an one second video
2. It adds 2 extra MB to ffmpeg-core.wasm

So right now I disable it as it is not usable for now.
For AVIF, I will check if it is slow as well.

[knaik]

I would love more focus on audio side of things, I am not sure if it's covered with theora or vorbis. So Opus and Flac. I mentioned vorbis ogg because Opus can be in an ogg container.

[mathiasrw]

@jeromewu 60x to transcode - aw. Hard one. But epic that you have already looked into it.

Is there any chance to obtain the version with AV1 included even it not being part of the main release?

[jeromewu]

@mathiasrw

I have published the version with AV1 in the next / 0.9.0-alpha.2 version, you can use it with following ways:

Node

$ npm install @ffmpeg/core@next
# or
$ npm install @ffmpeg/[email protected]

Browser

const ffmpeg = createFFmpeg({
  corePath: 'https://unpkg.com/@ffmpeg/[email protected]/dist/ffmpeg-core.js',
});

[jeromewu]

Hey guys, new / verified libraries are updated, please check and don't forget to upgrade to latest version of ffmpeg.wasm to use them.

[sejbr]

Any chance for h264_nvenc and hevc_nvenc (https://developer.nvidia.com/blog/nvidia-ffmpeg-transcoding-guide/)?
Both of these are great and fast combined with nvidia gpus.

[jeromewu]

@sejbr From what I know, it is impossible to use GPU capabilities right now, these two flags are impossible at the moment.

[sejbr]

You mean in web assembly? That's a shame :/.

[EstebanFuentealba]

Thank you for your excelent work @jeromewu 👏👏! if possible add libwebp plis 🙏

[jeromewu]

@smashah and @EstebanFuentealba
libwebp is added in v0.9.6, please check. 😄

[pbl4845]

SRT & RIST
That would be even more awesome!!

[sminodonte]

mjpeg

[owenthereal]

Great work on the library. Can we support RTMP? It would be interesting to see if it's possible to stream from the browser to YouTube or Twitch.

[tudalex]

Jpeg2000

[mpuz]

audio - ALAW, MULAW, RTP
Multicast, if not yet

[acschristoph]

Rtsp please

[hsbtr]

Does the new version support rtsp?

[ChuLiaoLiao]

Does it support GPU now?

[mattiasottosson]

Libvmaf
https://github.com/Netflix/vmaf/tree/master/libvmaf

[bittere]

Hi guys!
First off, awesome work on the project! Love it a lot.
I was wondering if it would be possible for SVG rendering support through librsvg or resvg.
Thanks!

[paivaric]

Rtsp please

[salivity]

could do with mutiple variations based on different tasks to save bandwidth and store the wasm's on a storage service for people to use.

[ddtamn]

how to slowdown a video ?
im using this command : await ffmpeg.run('-i video.mp4 -filter:v "setpts=2.0PTS" output.mp4');
but im get this error : [fferr] Unrecognized option 'i video.mp4 -filter:v "setpts=2.0PTS" output.mp4'.
createFFmpeg.js:43 [fferr] Error splitting the argument list: Option not found
createFFmpeg.js:43 [ffout] FFMPEG_END

[andfaulkner]

How hard would it be to handle .apng file inputs?

[Bec-k]

RTSP please

[salivity]

h265 is not working

[evilArsh]

webgpu has been released recently, RTSP please,god bless you

[elshnkhll]

I am using FFPLAY.EXE
my CL: ffplay -protocol_whitelist pipe,file,udp,rtp -i play.sdp

play/sdp file:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=-
c=IN IP4 127.0.0.1
m=audio 5153 RTP/AVP 97
a=rtpmap:97 opus/48000/2

Can you help?

Will donate! Thanks!

[Somebi]

I guess it is becoming a tradition to ask for RTSP each year. :)

[elshnkhll]

LIVE555?

[cjfswd]

.webm ?

[Rados51]

.webm ?

@cjfswd This works fine already.

[constlhq]

PCMA wanted

[CrypticSignal]

@jeromewu You should update aac (fdk-aac) in the OP to aac as it seems like ffmpeg.wasm doesn't support libfdk_aac.

[manfwh]

amv

[thipperz]

webp decoding

[salivity]

webp decoding

You can use ImageMagick for webp

[HappyPedestrian]

rtsp?

[fenicento]

Is there an updated / reliable list of supported formats and codecs for the latest release? Shouldn't this be in the README / FAQ?