hadoop_watermarking
.pdf
Hadoop video watermarking
Components
•Hadoop 1.2.1
•FFMPEG static build for x86_64
•Savanna 0.3
•OpenStack Grizzly
OpenStack and Savanna
•Provisioning layer
•VMs are deployed by Savanna on top of OpenStack installation
•Savanna configures Hadoop nodes
FFMPEG
•Split video into pieces
•Apply watermark
•Concat pieces
Hadoop
•MapReduce Framework
•HDFS - Distributed File System
Split 1
Split 1
…
Split N
MapReduce
Input Format |
Map |
||||
|
|
|
K1 V1 |
|
|
|
|
|
|
||
|
|
|
|
||
|
|
|
|
|
|
K2 V2
K3 V1
K2 V1
K3 V2
Kx Vy
Ky Vz
Kx Vo
K1 V1
K2 V2
K3 V1
K2 V1
K3 V2
Kx Vy
Ky Vz
Kx Vo
Reduce
K1 (V1)
K2 (V2, V1)
K3 (V1, V2)
Kx (Vy, Vo)
Ky (Vz)
Workflow
Input data
Input video file
Watermark
Local File System on Name Node
Input video file
Watermark
Local File System on Name Node
ffmpeg -i input.mp4 -ss 00:00:0x -t 00:01:00 Part_x
Part_0 |
Part_1 |
Part_2 |
Part_3 |
Part_4 |
Watermark
Local File System on Name Node
Part_0 |
Part_1 |
Part_2 |
Part_3 |
Part_4 |
Watermark
Local File System on Name Node
hadoop dfs -copyFromLocal Part_x /watermarking/jobUUID/in/in_x
hadoop dfs -copyFromLocal Watermark /watermarking/jobUUID/watermark
Part_0 |
Part_1 |
Part_2 |
Part_3 |
Part_4 |
Watermark
HDFS
Part_0 |
Part_1 |
Part_2 |
Part_3 |
Part_4 |
Watermark
HDFS
local_Part_X |
local_Part_Y |
hadoop dfs -copyToLocal /watermarking/jobUUID/ in/in_x local_Part_x
Watermark Watermark
!
!
Same on all workers
TaskTracker local FS |
TaskTracker local FS |