The VideoVerse

TVV EP 05 - Jean-Baptiste Kempf - The VideoLAN origin story

October 14, 2022 Jean-Baptiste Kempf Season 1 Episode 5
The VideoVerse
TVV EP 05 - Jean-Baptiste Kempf - The VideoLAN origin story
Show Notes Transcript Chapter Markers

Most people have heard of VLC, but not everyone knows about VideoLAN, the parent company behind it.   Founder Jean-Baptiste Kemp shares humble beginnings by a group of university students as a non-cross-platform and non-opensource solution. 

Then we discuss other equally exciting projects that VideoLAN is currently working on.

Watch the full video version.

Learn more about Visionular and get more information on AV1.

-My name is Jean-Baptiste, and I'm the president of the VideoLAN non-profit. 

 - JB, it's so awesome to have you on the show. We are very familiar, of course, with your work, and through VLC and all of the projects you're involved in. So I am, you know, really honored, actually, to be able to talk with you, and why don't you start by, I think everyone's familiar with VideoLAN and with VLC, in particular, but give us a really quick history of, you know, how it came to be, and yeah. 

- So people know about VLC, but they don't really know about VideoLAN, and a lot of people mix them, right? And that's okay, right? Because VLC is the most popular part of VideoLAN, but there are other projects of VideoLAN 'cause VideoLAN is a non-profit and like H.264 or dav1d, but a lot of other software, which are libraries, and people don't know about them. So it all comes down with the history of VideoLAN. The story of VideoLAN is a really weird one because like there is no one that decided to do VLC. It's a process that started in the late '80s that did like a few projects that arrived at the thing that was VLC. So how did it need start? So at Ecole Centrale Paris, which is one of the university in south of Paris, the network was managed by the students since the '80s, and the students wanted to have a faster network because they wanted to play video games in 1993, 1994, and because the students managed networks, the university didn't want to pay for a network upgrade. So they were like a bit blocked about that. How can we have a new network? And they went to see like a few... Because the university didn't want to pay for that, they went to see like a few partners, and they were one partner, which was a TV station, a French TV station called TF1, and they say, "Well, the future of video is satellite." Okay, well, now, we know that it's not. 

  • But in 1995, you know, why not, right? 
  • And what they wanted is that to be able to stream only with like one satellite dish and one satellite decoder for the 2,000 people who are on the campus and not have 2,000 dishes and 2,000 satellite decoders, and they wanted to use the network because, well, it was digital. So it should work on the network, and they had this Token Ring 10-megabit network, which was, of course, not enough, but they say, "Well, if you do that, we'll pay for the network upgrade," and the student who wanted to play video games say, "Okay, sure, we'll do that." It seems obvious today, but at the time, like the faster was Pentium 90 and most people, most students had the 486 DX at 33 megahertz. So doing MPEG-2 decoding, even as the real time, like you had those huge card that costed like 100s of francs or dollars. And that was absolutely not possible, not doable, right? 

  • So they discussed about like decoding only once and send raw video with a very small codec and so on, but so this project was like a bit insane, right? 1995, 1996, but they did it, and basically, they did this really cool demo for this project, which was called Network 2000, and the project could have stopped there. Actually, it did stop there, but after, there was a demo, everything worked. 30 seconds, everything was crushing. Everyone was happy, but one year after in 1998, there was a few students who said, "Well, you know what? Like maybe people could use that to stream video on a network," and that's how it became VideoLAN. This was not open source. This was not cross-platform. It took them two years to convince the university to change the license of the software to GPL, and they did it in 2001, and based on that, some people who are not at the school, not at the university, one was in England and one was in Netherlands, one reported it to Windows and the other reported it to MapQuest, but this was just a client, right? The solution was a lot more. There was a server. There was some networking. They did some weird multicast thing on the ATM network and something. There was lots of part of the VideoLAN solution, and one of it was the VideoLAN client, which was not the most complex at that time, but because it was able to play MPG-2, someone integrated like video playback, and then someone else, well, added, like DivX support, and because it was open source, people started adding new features and so on, and this VideoLAN the client became VLC and like no one cared that much about the whole complete solution, and because it was done at the beginning on Linux where you have the packaging altogether, and it was done by porting a Linux packaging, you didn't use those codec packs, right? So everything was inside the binary on Windows and Mac, and that's how VLC became very popular, and also, because VLC was a network player, I mean, it is a network player. It was very resistant to errors and broken files, and at that time, when we were downloading things in peer to peer, that was very important.

  • So what what's interesting is that no one designed VLC to be like, "Oh, I'm going to do a new player that is going to play everything, and it's going to be cross-platform," because VLCs a small part of the VideoLAN project, which is a small part, the continuation, open source continuation of a project which was like you do a new network because like in the 1980s, they had a Token Ring network that was very slow, and they wanted to play video games, and because of that, VideoLAN became like first, to student group, and in 2003, there was H.264 that was started over there, but it's a weird beginning, but then, it became non-profit, but VideoLAN stores a lot of multimedia open source projects. 
  • Mmm, interesting. Now, at what point did you get involved? 
  • So, actually, I arrived at Ecole Centrale Paris in 2003, and I was a kind of VP of the network, of the student network, and VideoLAN was actually a part of this non-profit managing the network, and so I started by, basically, doing broadcasting mostly on the server side, which was, at the beginning, VideoLAN server then VideoLAN client, which was screening files, and that's how I started with VLC, but I actually spent more time starting in 2005 and '06 when I was actually leaving the school, and I was in a extremely boring internships where I had nothing to do, and I was just like, "Hey, what can I do?" And so because most of my friends were working on VLC, I said, "Okay, let's do that." 
  • Yeah, let's work on that. Wow, that's amazing. So you started almost 20 years ago. 19 years ago or so? 
  • So after that, I created the VideoLAN non-profit in 2007, 2008, where I took most of the assets around all the VideoLAN project, VLC being one, but X264 and libdvdcss and a lot of things, and I started a lot of other projects around the VideoLAN non-profit, and the VideoLAN non-profit is doing a lot more than VLC today. 
  • And bootstrapping new codec standards, tell us about dav1d and, you know, how that project came to be and what it is. I think our audience knows, but anyway, tell us about dav1d. 
  • So dav1d is so this very fast AV1 decoder, but it's one of the most impressive software I know, and people don't realize it, right? It's, basically, a collaboration with people from VideoLAN, people from FFmpeg, and people from Two Orioles, which is a company doing a VP9 AV1 encoder, and we were absolutely not happy with the state of the open source format decoders. The VP9 decoder was not as good, and one became two FFmpeg, it was a lot faster, and there is a good reason for that because it's extremely difficult to define a new standard and have something that actually works and then something that is prediction ready, especially because when you look at the energy spent, a lot of the energy spent is on all those machines, right? With your screen and the decoders and so on. So we wanted to have something that was extremely efficient because we thought it was important, especially because at the beginning, peoples were start fighting against AV1 saying, "Oh, it's too complex to decode. No one can do that on real time," blah, blah, blah, blah, blah, things that we heard every time, right? When H.264 was out, I remember, "Yeah, it's going to be too slow," blah, blah, blah, blah.  When it's going to be too slow. 
  • Yeah. (laughs) 
  • Blah, blah, blah. "Oh, and AV1 is out, it's going to be slow," like, come on, guys, like, you've seen that five times. You know how it's going. 
  • Yes. - It's not. 
  • But our goal was to say that we wanted to have something that was very quickly there and like absolutely, like every cycle counts, and because you need to bootstrap somewhere, and it's a chicken and egg problem, right? You don't have hardware decoders so people don't encode in these formats and because people don't encode in this format, well, they don't because they have no one that is able to play, right? So we thought that having something that was integrated very quickly in VLC and FFmpeg was just like sure, it's not going to be amazing compared to hardware decoders, but it's going to be quite good, and the result we see today was that, in many cases, even for software decoding, sometimes the software decoders are faster and use less power than the hardware decoders. 
  • For a lot of reasons, right? So the bet, I think, is won, and yeah, so we launched this project. We had a lot of monetary help from the Alliance for Open Media, gave the initial money, and then because they thought that we arrived at something that was decent, then we got like extra funding from a few companies directly, but people don't realize how insane dav1d is, right? So let me explain that. dav1d is around 30,000 lines of T. 30,000 lines of T, it's tricky, right? But we are not talking about something so out of line, right? 
  • But it's 190,000 line of handwritten assembly, handwritten assembly, right? 
  • Wow, wow. - In 2020, 2021. I didn't know that. 
  • People are writing assembly by hand. And it's To give you, there is more assembly in dav1d than in the whole FFmpeg for the whole other formats, right? 
  • Wow. 
  • - It's insane. Well, why is that so big? Because we support five architectures, V7, V8. We use Intel 32-bit, and which is SSE3, and Intel 64-bit, basically, AVX2. 
  • Yeah. But we also support SSE3 with 64 bit because most of the console don't support AVX2, but so that's five architectures, and you need to code for them, but then at the same time, you need to support 8-bit decoding, but today, right? Like it needs to have 10-bit or 12-bit. 
  • And so, well we have like everything that we could accelerate with it. So it it's like, I don't know any software, open source or non-open source. 
  • That in the last few years. 
  • Yeah. I've written and has so much assembly. 
  • Because like even FFmpeg has less. But also, that most of it was written in less than two years. 
  • By couple of people, right? 
  • I was gonna ask, how big was the team that developed it? 
  • Five, 10. Something like that, right? 
  • And, of course, it runs on iOS, android. 
  • Yeah, absolutely. 
  • Well, because like VideoLAN... And because, well, I need it for VLC, right? I need 4Gs on like platforms. 
  • Yeah, it has to support all platforms. 
  • You can't say, "Oh, no, you know, this one is small. I don't care." No, we do care, right? VLC is on more platform than Como.
  • Than Office, right? 
  • It's in our DNA, right? So we do that. 
  • I had no idea about all that handwritten assembly, no idea. 
  • And the thing is like, when you talk about handwritten, every time you speak about handwritten assembly, people are saying, "Yeah, but you know, you shouldn't do that, right? Because compilers are faster, right?" Every single time, you hear that, right? And so it's like, "Maybe, so let's try." In average, on many functions, we are four to five times faster. Than anything else even when it's like, "Oh, we use-" "No, choose this version of Clang." So we use this version of, "This is your Clang or ICC," and "Oh, you need to use all those options." We took all those options, right? We are faster. 
  • So compared to the original decoder from AOM, which was developed during the process, and this is absolutely normal, right? We are often three to four times faster, right?  We not talking about 30%. It just say plus 200%. And the AOM decoder was absolutely not ridiculous, right? It was pretty good compared to what was done on VP9, and people learn about it, but once you have a fixed target, right? Not everything is moving. You can try to get exactly the perfect architecture, and like win every cycle. You can cheat because you know what specs allows, right? You know, that like this one is not going to saturate because, else, everyone's screen is invalid. So you can cheat there. Et cetera, et cetera, et cetera.And there is also the fact that the threading on dav1d is extremely complex 'cause you have slice threading frame threading. There is the post filter threading, and, of course, the film grain threading, and most people don't... A lot of people who use dav1d directly in the software had lots of difficulty setting all those because it depends on the input and so on. So that was done so that you can actually, if you know what you're doing, you can have the maximum power. 
  • Yeah. - For the software. 
  • Yeah, that's great. You know, I think I love it that you're pointing out the fact that dav1d actually outperforms or, you know, you made the comment early on that software can outperform even hardware and
  • Yeah, so that's a good
  • Let's talk about that because I think that's counterintuitive, and I think even some of our audience, our audience are mostly deep experts, but I don't think everybody either believes that or knows it. (laughs) 
  • Well, it's because it depends, right? It depends on the hardware, right? But especially for like... The problem is that some complex CPUs, because most of the formats, right? They're on the CPU on DPU, but they are not CPU on GPU, right? It's a DSP, right? It's exactly for that. And sometimes, you need to turn it on and turning on this part of the CPU or the GPU is going to have like a bigger power while when you can decode 240P or 360P. Maybe, you know, like it's going to be similar sometimes.It's truer. It's rarely smaller. But it happens, right? We had some tests. And then you're going say, "Oh, well, it's a bug in the CPU." But the CPU is out, right? It's on the phone. What can you do, right? 
  • Yeah, exactly. (laughs) 
  • Like there was a bug, I think it was on Samsung devices where the VP9 decoder on hardware is mismanaging one of the features of VP9. So what are you going to do? Well, you disable it, right? Because there is hundreds of million people who have these bug, right? So we had this case where when you compare, and that was mostly true for lower resolutions that sometimes, well, the hardware decoding was drawing more power because it was turning on another part of the chip or because like it's for lot of reasons and that can happen, yes. 
  • - Yeah, yeah, Amazing. 
  • But it's also more important than that is that realize that you can achieve, and actually it's mostly on lower resolution. Similar conception there, and that's pretty cool because then, what costing you is the backlight of the screen. -
  •  Yeah, yeah, that's right, and this is really interesting, of course. I spend most of my time professionally day to day on the encoding side, but I'm always happy to talk about decoding because, you know, it's the chicken and the egg, right? So I can create the bit stream, but if you can't play it back anywhere, it doesn't matter, and I know that, you know, we get in conversations all the time around AV1 or any new standard, and it always goes like this, you know, with a customer. "Yes, we know it's really great. We know it has an advantage. Maybe they've tested, maybe not," but you know, generally, there's acknowledgement like, "Yes, we're really excited, but come back and talk to us when you've got this ecosystem and that ecosystem and when Qualcomm is supported and when Apple's supported." You know, they have their long list, right? And we always come back with AV1, what's beautiful, and with dav1d is we can always come back and say, "Hey, guys, did you know that this player, dav1d, is so powerful and so efficient that AV1 is being deployed on some of the lowest end devices using software decoding in markets that you would think that'd be the last place, you know, that it's being deployed, and that's counterintuitive. 
  • It's true. Well, there are cases and markets where like having a lower bandwidth is so much more important, right? Because infrastructure is not there. Or because there is a lot of data cap and because when you have a cell phone, like the biggest consumption of power is your screen and your 4G chip, right? And when you play back video, right? So that's important, and also, like I envision, and that what I said at the beginning is that, you will always need a software decoder as a backup, right? So let's do one that is efficient when we can do it, but what is true is that the amount of time spent on the software decoder is huge, and so it's a lot of resources, in my opinion. But I think that the ecosystem needs to do that for every new format. 
  • It does, it does. 
  • If you don't have
  • Right, and it needs to be on a very liberal license, right? MIT, you want to modify it? Modify it, like do it. My goal is that the formats is out, right? And I think that to speak about something else in dav1d, like the VVC project actually understood that for once, right? So the VVC decoder and encoder that was out of the MPEG VVC consortium is actually something that is good, right? Not like in HVC times where you had like something that, well, you couldn't do anything about it, right? So coming fast on production is important, especially since, well, 85% of the stream are still H.264. 
  • That's right, that's right, absolutely. Wow, amazing. Well, so is there anything coming next for dav1d and what can you tell us? 
  • So we released dav1d 1.0 not so long ago. We are fixing a few stuff. There are a couple of functions that could be faster, and we're working on that, but basically, the main work is done. What we could do, and in some things I would like to do, but currently, I don't have the resources to do it, is to start to do a hybrid decoder, which means CPU and GPU, especially like on dav1d, like if you activate film grain, we give you the shades the pixel shaders that you can run inside your application so that the pixel shader is applied. The filter is applied on the GPU, and a lot of phones have like very good GPUs or they're not that good to do complex video games. They're good enough to apply in a pixel, and when you see everyone, there are lots of loop restoration, post filters with the film grain, as you already know, and so on, and some of those could be uploaded to the GPU. It's absolutely not going to increase the number of FPS. I might think that, on many devices, it's going to lower the number of FPS, but as soon as you eat 60 FPSs, you don't care, right? 
  • Yeah, that's right. 
  • But what you could improve is the number of FPS per watt, right? So especially because the filter, which are the last part of the AV1 decoding process are very similar on all those pixels. So like GPUs are pretty good at doing that. So you could expect that the first part, which is like entropic coding, motion compensation, and so on is done mostly on the CPU. And at some point in the mid decoding, you will move too the GPU. This is extremely rare because most of the time, you do either a full GPU like the one that I have done for (indistinct) full GPU decoders or there was also a AV one, which was doing everything on the GPU. Having a hybrid decoder as open source, I don't think it's ever done. The problem is that here, we are not talking about just implementation. We're talking about actually R&D. And we need to have someone, and I don't have the resources on VideoLAN to have someone that. Hey, you know what? I'm going to spend six months to one year having
  • But I think it's necessary for the ecosystem. Because I think people are changing phone less and less. So the renewal of the machines is going to go slower. Like when I watch my mom with a five-year-old phone and six-years-old PC and say, "Well, you should change it." She's like, "No, it's fine." So having those kind of like hybrid decoding so that you can decrease the power consumption is important. 
  • Yeah, yeah, amazing. 
  • So I can't tell you it's next because, like, currently, I'm not sure I'm going to do it, but I think that the next big thing, we could do around dav1d. 
  • Yeah, yeah. So would you go out and get sponsorship? I mean, it sounds like dav1d was, like you said, it was initially funded from the AOM, and then a handful of other companies have
  • Sure, but the founding was mostly like something that is sure, right? They need something, right? 
  • Sure, they have. Yeah, that one's easy, yeah. 
  • Like I want this, right? And when they have, they say, "You know what? Like, ah, I would try GPU decoding. Would it work?" "I think so." "How do you know?" “I think so, trust me." 
  • - We have to build it. 
  • Right, so that's why it's a bit more difficult to find a sponsorship and resources
  • Got it, got it, got it. Now, so your commercial model, I know you're a non-profit. So do you literally just, you know, you raise enough money to pay your team and pay the team working on a project, or what does that look like? How does that work? 
  • VideoLAN has no money. Right, we pay no one. Okay, so there is no business model around VideoLAN. Yeah, and so you're an administrator of these projects. And that allows full independence, right? So that you can do. 
  • Yeah, that's right. 
  • Right, and what you see around that, FFmpeg is the same, right? But what you see around is a few companies, which are basically consulting company, around that, right? And I created one from Video Labs, around (indistinct) one, around FFmpeg lately, which is called FF Labs, and there are a few other people in the community who are the consultant. And so on, and because we have those commercial entities, those are able to take contracts. And dav1d was one of the contracts. But it's mostly like, it's the outlier. Most of the time, it doesn't happen like that. But yeah, of course, if you want to have something that is like extremely focused, like dav1d, and like in one year and a half have a decoder, and two years, have the fastest decoder ever. Then, of course, you need resources, and then you use those kind of very common thing. You see on open source, which is commercial company who works. And there is a of people around the VideoLAN, FFmpeg community. Who actually do that. 
  • Got it, got it. Wow, amazing. Well, thank you for giving all the details behind dav1d. You know, I started out by saying that, you know, I wanted to talk about dav1d. I also wanted to talk, maybe have a little more sort of philosophical type conversation, not really philosophical, but to say that it's so important for a new codec standard that this role of having an open source, highly performant decoder, you know, that really works because without that, you know, you can have the greatest encoders, it can be a really great standard but it's ultimately gonna fail because you can't play back the bit stream, you know? 
  • And you need, at the beginning, to have a decoder that works everywhere. Because it's like you have like an ecosystems and like not everyone has the same type of platforms. But like if you want to reach out, you need to have everything. Especially since like, let's be honest, H.264 was amazing, right? When you move from MPEG-2 and then what, right? Oh, what is it good with HVC? Well, it's great for 4k. I don't have a 4k stream. I don't care, right? So you need to... It's like when you move from VHS to DVD, right? Like the change was in usage. Was huge, right? When you change from DVD from Bluerays, what is a Blueray? Why it's a DVD extra HD, right? That's it. That's how you define it. But having DVDs where you have chapters like multiple videos, multiple angles, you don't have to rewind your VHS. And that's the same what that happened with H.264. H.264 was like a huge leap. And then you need to be a ton better, right? And for example, HEVC completely failed, completely failed. It's completely, it's a narrow, and, of course, people from MPEG are, "No, it's great, it's great." 
  • Yeah. (laughs) 
  • It's a complete failure. It's a complete failure compared to the amount of money that was invested, right? 
  • That's right, that's right.
  • Even in open source, right? VP8 or VP9 did not get attraction they should have had, right? Everyone is, in my opinion, the continuation and learned about the issues. That they were but it's important because you need everyone to be able to play your streams because then, it costs money on networking on encoding time, on storage, and so on. Right, and today, it's quite funny, right? Because there is VVC, AV1, sometimes in a few years, AV2. You have impact MPEG, EVC. 
  • EVC, yeah. 
  • Which is like the low version of VVC, but it's also baseline and not baseline, which is confusing. And then you get the MPEG-5, LCEVC. Which is not really a codec. 
  • But it's a codec that is actually an intimate layer of all the codecs. And so like, are you going to build all those inside your DSPs, on your chips? Are you going to waste one billion transistor on each of those formats? What about encoding, right? When are you going to drop MPEG-2 decoding in hardware or an encoder in hardware, right? So those trade-offs are important and you have to fix your licensing, right? Sure, if you don't, then you get the HEVC problem, but you need to have very good decors, and for example, on HEVC, it's fun, but the FFmpeg HEVC decoder is, in many case, lower and less good than the AV1 one because we spent less time on it, right? 
  • That's right, that's right. 
  • So if you want your format to be popular and yeah, you don't care about piracy. You don't care about UGC content. You don't care, but those people are the ones who are going to buy your devices and make it out, right? And you are not alone, right? There are many formats, so you need to compete. 
  • Thank you so much for coming on, and I know that we're gonna do a part two. So I'm really super excited to hear about the projects that you have coming up, but this concludes the dav1d discussion, and thank you again. Thank you, JB. 
  • Thanks a lot.
How VideoLAN got started
JBK gets involved with VLC
dav1d The fastest AV1 decoder?
What’s next for dav1d?
The Funding & Sponsorship behind dav1d