Android Video Playback Tutorial (part2) — The structure of MP4 file
In the last section, I went through some basic terms in video playback development.
In this section we will focus on container level, and use MP4 as an example to illustrate what does a container file looks like.
Here’s the checklist of today’s tutorial :
1. Structure of MP4 file
2. Metadata of MP4 file
3. Initialisation of a standard video player
4. Online video streaming1.Structure of MP4 file
In the last section we go through the structure of the container file :
But this may still looks a bit abstract to you, then lets go deeper.
1.1 What is MP4 format
Generally speaking, MP4 is actually just a standard which gets verified by ISO, which means that as long as you generates a MP4 file, it must follow the standard and definition from the ISO, just like a protocol that everyone follow.
Click this link->[ISO standard for Mp4 file]
As you can see in the definition, we define a lot of metadata's position, size. Lets take a look at the first few lines:
With that being said, any container file includes MP4 are just structural files, the only differences are their data’s positions and headers and stuffs like that according to ISO standard. If you are interested you can check whats the definition and specification for other format like rmvb, mkv, AVI, or why some of them are suitable for online streaming while some don’t.
2 .The MetaData of a MP4 file
About the metadata of a mp4 file, Apple has specified the details
(this is another version of mp4 which is aiming for QuickTime, but its almost the same so we can ignore the difference now)
Feel free to checkout the specification of the QuickTime MP4 [Movie Atom]
Get Qing Zhong’s stories in your inbox
Join Medium for free to get updates from this writer.
First of all, in the meta data, we called them Atom Header. Atom Header has two different types: 1. Leaf Atom 2. Container Atom. The former one contains a lot of information while the later one contains other Leaf or Container Atom. There's a hierarchy between container and leaf atom.
It looks very complicated ，but for a player a lot of information are not really needed. The most important information is the Sample Table Atoms, which is the stbl atom header. It stores a table which maps the vide time and the sample’s position in the file(for example, which byte to which byte is the data for that), including the size of the sample as well.
3 .How Player works
Usually a player consists of 4 parts
1. Extractor
2. TrackRenderer
3. Load Controller
4. SourceExtractor will load the data from the source file(via HTTPS/HTTP), Load control module will control the strategy for buffering ,TrackRenderer will decode the data read from Source and then display them to the screen on your device.
Well with Android APIs contexts, we can simplify the process as the below animation:
Before the player handles the data to renderer/decoder, extractor will need to read all the atom headers into the ram, for example the sample table we mentioned. Usually three tables will be initialised, one is for the time index, another one is the starting position of each sample, the last one is the size of each sample
If the player needs to start the playback from the 1 millesec.
from TABLE 1 we knew for the first milli sec, which sample do we need, from the second and third table we know whats the range of that sample inside the container (0–300th byte). Then our player will download the data and feed them to the renderer.
So when we seek to different position of the video, we knew exactly what parts of the data from the container we want to start downloading/buffering.
From everything above we know that before the video playback starts, we need to read and parse those atom headers(which is a time consuming work), which is also a bottleneck for online streaming’s waiting time. In the next section im gonna go through the Adaptive Streaming, which make use of the Fragmented MP4 technology to reduce the waiting time.
4 .Online Video Streaming
The only difference between online streaming and local video playback is the Source, one is local another one needs to be transferred via network. Obviously we don’t need to download the entire file into our ram, so we need to control the buffering and download strategy.
For example, while user is watching the 10th second, we may just need to buffer all the way till 20th second.
And the so called Progressive Downloading solves the problem.
It sounds like some very deep and complicated technology~
while its actually just HTTP. In Http 1.1 it defines something called Range in the headers.
With this tag you can specify the starting and the end byte you want to download from the source file. This is also the foundation of the break point downloading in a lot of browsers. Every time we disconnect and reconnect, we calculate the already downloaded file’s size and specify **Range** == current size+1.
