You are on page 1of 6

A Survey on Different Video Watermarking

Techniques and Comparative Analysis with


Reference to H.264/AVC
Sourav Bhattacharya, T. Chattopadhyay and Arpan Pal

Abstract — Last few years have witnessed rapid growth in enhancement and database linking etc. [2].
video coding technology. Among various standards, Growing popularity of video based applications such as
H.264/Advanced Video Codec (AVC) is found to be of Internet multimedia, wireless video, personal video recorders,
significant importance regarding reduced bandwidth, better video-on-demand, set-top box, videophone and
image quality and network friendliness. One of the current videoconferencing have a demand for much higher
fields of interest is to develop a system with authentication and compression to meet bandwidth criteria and best video quality
copyright protection methodology embedded within an as possible. Different video Encoder Decoders (CODECs)
efficient video codec. In this paper we first perform a survey have evolved to meet the current requirements of video
on available video watermarking techniques, feasibility study application based products. Among various available
on watermarking techniques meeting application specific standards H.264 / Advanced Video Codec (AVC) is becoming
criteria for H.264/AVC and then we perform a comparative an important alternative regarding reduced band width, better
analysis based on robustness and computational complexity of
image quality in terms of peak-signal-to-noise-ratio (PSNR)
different watermarking algorithms. 1
and network friendliness [26], but it requires higher
computational complexity.
Index Terms — Video Watermarking, H.264/AVC.
Different watermarking techniques have been proposed for
I. INTRODUCTION different video CODECs, but only a few works on
H.264/AVC can be found in the literature. H.264/AVC uses
High speed computer networks, the Internet and the World different transformation and block sizes than MPEG series, so
Wide Web have revolutionized the way in which digital data development of new algorithms is required to integrate robust
is distributed. The widespread and easy accesses to watermarking techniques for different profiles of H.264/AVC.
multimedia contents and possibility to make unlimited copy In section II we review the basics of digital watermarking,
without loss of considerable fidelity have motivated the need video watermarking terminologies and techniques. In Section
for digital rights management. Digital watermarking is a III we briefly discuss common video watermarking
technology that can serve this purpose. A large number of techniques. Comparative analyses between different
watermarking schemes have been proposed to hide copyright watermarking techniques are described in section IV. Finally
marks and other information in digital images, video, audio basics of H.264/AVC encoder are explained and applicability
and other multimedia objects [1, and references there in]. of different watermarking techniques in H.264/AVC is drawn
A watermark is a digital data embedded in multimedia in section V.
objects such that the watermark can be detected or extracted at
later times in order to make an assertion about the object. The
main purpose of digital watermarking is to embed information II. VIDEO WATERMARKING
imperceptibly and robustly in the host data. Typically the
watermark contains information about the origin, ownership, A. Digital watermarking:
destination, copy control, transaction etc. Potential Digital watermarking also known as watermark insertion or
applications of digital watermarking include transaction watermark embedding, represents the method of inserting
tracking, copy control, authentication, legacy system information into multimedia data also called original media or
cover media e.g. text, audio, image, video. The embedded
1 information or watermark can be a serial number or random
Sourav Bhattacharya, T. Chattopadhyay and Arpan Pal are with
Convergence Solutions Practice, Tata Consultancy Services Limited, Kolkata, number sequence, ownership identifiers, copyright messages,
India. control signals, transaction dates, information about the
E-mail: {sourav.bhattacharya, tanushyam.chattopadhyay, arpan.pal} creators of the work, bi-level or gray level images, text or
@tcs.com
other digital data formats. In the literature large number of text

1-4244-0216-6/06/$20.00 ©2006 IEEE


[3]-[5], image [6]-[9], audio [10] and video [11]-[15] {Yes or No} = d ( X ′′, [ X ],W , [ K ]) (3)
watermarking algorithms can be found. These algorithms
modify the original media to generate the watermarked media. B Video watermarking terminologies:
There may be no or little perceptible differences between the Video watermarking describes the process of embedding
original media and the watermarked media. information in video data. Different data hiding terminologies
Fig.1 gives an overview of different types of watermarking are given in [16]. The important terminologies pertaining to
methodologies depending on their working domains, cover digital video watermarking are:
media, perceptibility and application areas. Digital Video: Video sequence is a collection of
After embedding watermark, the watermarked media are consecutive and equally time spaced still images.
sent over Internet or some other transmission channels. Payload: It is the amount of information that can be stored
Whenever the copyright of the digital media is under question, in a watermark. An important concept regarding the video-
the embedded information is decoded to identify copyright watermarking payload is watermark granularity. Watermark
owner. The decoding process can extract the watermark from granularity can be defined as how much data is required for
the watermarked media (watermark extraction) or can detect embedding one unit of watermark information.
the existence of watermark in it (watermark detection). Perceptibility: video watermarking methodology is called
imperceptible if humans cannot distinguish between the
Watermarking original video from the video with inserted watermark.
Robustness: a fragile watermark should not be robust
Domain Document Perception Application
against intentional modification techniques, as failure to detect
the watermark signifies that the received data is no longer
Spatial Frequency authentic. In case of application such as copyright protection,
Source Destination
Based Based it is desirable that watermark always remains in the video data,
even if the video data is subjected to intentional and
Text Image Audio Video unintentional signal processing attacks. Hence, depending on
Invisible
the requirements of the application the watermark is
Visible embedded in a robust, semi-fragile or fragile manner.
Robust
Security: the security of the watermarking algorithm is
Fragile
ensured in the same way as in encryption methodology.
According to the Kerckhoff’s assumption, the algorithm for
Private
watermark embedding can be considered to be public, where
Public
as the security depend solely on the choice of a key from a
Non-invertible Nonquasi-invertible
large key space.
Quasi-invertible C Video watermarking techniques:
Invertible
Fig.1 Different types of watermarking methodologies. Apparently any image watermarking technique can be
extended to watermark videos, but in reality video
The embedding or encoding process can be viewed as a watermarking techniques need to meet other challenges than
function or mapping that maps the input X (original media), that in image watermarking schemes such as large volume of
inherently redundant data between frames, the unbalance
W (watermark) and/or K (key) to output X ′ (watermarked
between the motion and motionless regions, real-time
media). Mathematically it can be expressed as
requirements in the video broadcasting etc. Watermarked
video sequences are very much susceptible to pirate attacks
X ′ = E ( X , W , [ K ])
(1) such as frame averaging, frame swapping, statistical analysis,
where E (⋅) denotes the embedding process and [⋅] digital-analog (AD/DA) conversion, and lossy compressions.
represents optional argument. Similarly the decoding or Video watermarking applications can be grouped as security
extraction process D (⋅) can be expressed formally as related like Copy control [18], fingerprinting, ownership
identification, authentication, taper resistance etc. or value
added applications like legacy system enhancement, database
W ′ = D ( X ′′, [ X ], [ K ]) (2) linking [1], video tagging, digital video broadcast monitoring
and the detection process d (⋅) can be expressed as [19], Media Bridge [20] etc.
Apart from robustness, reliability, imperceptibility,
practicality, video watermarking algorithms should also amplitude factor). But increases in cr reduces the data rate
address issues such as localized detection, real time of the scheme, where as increases in σ 2p or µ α results in
algorithm complexity, synchronization recovery, effects of
floating point representation, power dissipation etc [17]. perceptibility of the watermark.
According to the working domain, video watermarking As DCT is a linear transformation and watermark is
techniques are classified in pixel domain and transform independent of the picture, the watermark can be added in
domain techniques. In pixel domain the watermark is the DCT domain. The 1D watermark vector is rearranged
embedded in the source video by simple addition or bit into frame structure and by transforming it to 8 × 8 DCT
replacement of selected pixel positions. The main domain; the watermark can be added directly to a partially
advantages of using pixel domain techniques are that they decoded video stream. Since the size and transfer rate of
are conceptually simple to understand and the time watermarked video should be identical to the original video,
complexity of these techniques are low which favours real DCT coefficients of watermark and video frame are
time implementations. But these techniques generally lacks combined only if the resulting VLC code is of same length
in providing adequate robustness and imperceptibility of the original one. Again drift compensation is required to
requirements. cancel out watermark components from P and B frames, as
In transform domain methods, the host signal is motion compensated prediction or interpolation from other
transformed into a different domain and watermark is frames are added by the decoder to construct the P and B
embedded in selective coefficients. Commonly used frames.
transform methodologies are discrete cosine transformation A 2D spread spectrum method for video watermarking
(DCT) and discrete wavelet transformation (DWT). (just another watermarking system, JAWS) was proposed in
Detection is generally performed by transforming the [19], which is used for monitoring video data transmitted
received signal into appropriate domain and searching for over different broadcast links. This pixel domain
the watermarking patterns or attributes. The main advantage watermarking scheme is distinctive for its enhanced
of the transformed domain watermarking is the easy payload capabilities and shift invariance.
applicability of special transformed domain properties. For A novel collusion resistant (CR) video watermarking
example, working in the frequency domain enables us to approach is proposed in [21]. This is a practical frame by
apply more advanced properties of the human visual system frame video watermarking technique. Here a basic
(HVS) to ensure better robustness and imperceptibility s × s watermark pattern is first created and this pattern is
criteria. repeatedly embedded so that it is centred around a fixed
number of selected points known as anchors in every video
III. SURVEY ON VIDEO WATERMARKING frame. The part of the video frame where the basic
watermark is embedded is called the footprint. Anchor
Watermark can be either directly inserted in the raw points are calculated using feature extraction algorithm. As
video data or integrated during encoding process or the content of the video frames changes, so do the selected
implemented after compressing the video data. Now we feature points. As a result of that watermark footprints
shall briefly discuss some common video watermarking evolves with the video. After generating these watermark
techniques. frames with in a given host frame, spatial masking is
Spread spectrum (SS) based watermarking technique was applied on it to ensue robustness and imperceptibility
proposed in [11]. In the basic algorithm each bit of criteria. Then the scaled watermark is embedded in the host
data using addition.
watermark a j , a j ∈ {−1,1} is spread over a large number
Watermarking using CDMA modulation was proposed
of chips (cr ) and modulated by a binary pseudo-noise in [22]. In this proposed methodology one of the four least
significant bitplanes are replaced by watermark planes. The
sequence pi , pi ∈ {−1,1} . The video and watermark are
bitplanes to be replaced are selected according to a random
represented as vectors and scaled addition is carried out for periodic quaternary sequence. The watermark plane is
watermark insertion. The retrieval of the watermark is generated using 1D spread spectrum methodology. For
carried out by high-pass filtering followed by correlation- detection of the watermark, the author proposed a two-level
based method. The robustness of the algorithm can be hierarchical correlation methodology.
increased by increasing cr , σ 2p (variance of pseudo One of the prime motivations for integrating
watermarking into video coding structures such as MPEG-
random sequence), or µα (mean of locally adjustable 2, H.264 etc is to reduce the overall real-time video
processing complexity. The reader is referred to [26] for an proposed.
exposition on H.264/AVC. Watermarking algorithm based on group of frames
In [23] a watermarking method using variable length (GOF) has few important benefits as they utilize temporal
code (VLC) swapping was proposed. This methodology properties of the video. This consideration helps to maintain
was based on the observation that in the MPEG-2, H.26l temporal imperceptibility.
VLC tables there are pairs of code words ( r , l ) 6 c0 and
IV. COMPARATIVE ANALYSIS OF DIFFERENT VIDEO
(r , l ± 1) 6 c1 such that length(c 0 ) = length(c1 ) , WATERMARKING TECHNIQUES

lsb(c0 ) ≠ lsb(c1 ) . Such level-adjacent pairs are called In this section we shall discuss the comparative analysis
of different video watermarking techniques in Table 1. We
label-carrying VLC (lc-VLC). A covert data bit U i is shall use the following terminology in the table:
embedded into a frame by extracting eligible lc-VLC, R: Robustness; Rl: Reliability; I: Imperceptibility; P:
ci ∈ {c 0 } ∪ {c1} , and swapping a codeword, if necessary practicality; T: Time complexity; S: synchronization
recovery. We shall denote the measure of goodness using
such that to ensure lsb( ci ) = U i . This process does not the quantifiers Good (G), Acceptable (A), and Poor (P).
use any random key based component as a result of that this
TABLE 1
method is not robust against attacks.
In [24], Darmstaedter et al. proposed a data hiding Technique R Rl I P T
method (region based energy modification, RBEM), where SS A A G G G
data were embedded by manipulating the average energy or JAWS A A G G G
luminance intensities in sub-regions of each frame. This CR G G G P P
method achieves a high data capacity by embedding one bit CDMA A A G A A
into every 8 × 8 block, and error control coding is used to VLC P P G G G
ensure robustness. Here the data sequence U is directly RBEM A A G A A
embedded in the cover data. The concept of block TDC G G G A A
classification was introduced here. With the classification PW A A G A G
of blocks, this scheme can take the advantage of local 3D DFT G G G A A
spatial characteristics and adjust its embedding strategy to GOF G A G A G
improve imperceptibility and robustness criteria.
One of the first transformed domain video watermarking V. WATERMARKING TECHNIQUES APPLICABLE TO H.264
methods (TDC) was proposed by Cox et al. in [25]. The / AVC
authors proposed and stressed on the importance of H.264/AVC is becoming a popular video codec for its
embedding the watermark into perceptually significant better compression, picture quality and applicability to
components to increase robustness against signal processing portable electronic devices. So a H.264 video CODEC with
and lossy compression techniques. The watermark of length a suitable watermarking embedded in it is a good consumer
n was populated from a standard normal distribution apart electronics product in current scenario. We have already
from a binary PN sequence in order to enhance robustness. discussed about different watermarking techniques and
This method uses a non-blind approach for watermark compared their performances. Now we shall discuss about
detention. Detection is performed by transforming the an overview of H.264/AVC and then we shall discuss about
original and test frame in the DCT domain and correlating the applicability of different watermarking technology in
the difference vector with the expected watermark pattern. H.264/AVC.
A perceptual watermarking (PW) method explicitly We shall discuss the overview of H.264 CODEC with
model masking properties of the HVS and utilizes these help of a block diagram shown in Fig. 2 [27].
models to analyse video sequence or frames to embed The H.264 video encoder works as follows:
watermark in the optimal way. The five main properties of • Input image is captured.
the HSV namely, frequency sensitivity, luminance • Prediction cost is computed by exploiting temporal (P)
sensitivity, contrast masking, edge masking and temporal redundancy and spatial (I) redundancy.
masking can be exploited by video watermarking • Best prediction mode (temporal (P) or spatial (I)) is
techniques [29], [30]. selected by defining a minimizing function on the costs.
In [31] a 3D DFT based robust watermarking scheme • Residue is computed for the best prediction mode.
+ Entropy
Input video T Q Reorder
(YUV 4:2:0 Encoder
format) Best
Predictio +
n Mode
and Inter (P) NAL
block
size
F/ n-1 selection
(Reference)
Intra (I)

+
/
F n
(Reconstruct
De-blocking T-1 Q-1
Filter +
ed)

Fig 2 Block diagram of H.264/AVC Encoder

• Residue is gone through a Integer transformation on SS Pixel/Transf Algebraic/ Y


4x4 sub-blocks followed by quantization. ormed DCT
• Using inverse quantization and inverse transformation
residue part is reconstructed. JAWS Pixel Algebraic Y
• Reconstructed image is filtered with de-blocking filter CR Pixel Algebraic Y
to remove blocking artefacts.
• Quantized coefficients are reordered and entropy CDMA Pixel Algebraic N
coded.
• Reconstructed and deblocked image is used as the VLC During Algebraic Y
reference for next frame prediction. Compression
Watermarking can be implemented in motion vectors or in RBEM Pixel Algebraic Y
integer transformation [28]. But H.264 differs from other
video CODECs like MPEG in the following basic units: TDC Transformed DCT N
• All transformations are performed on 4x4 block instead
of 8x8 or 16x16 block. PW Pixel/Transf Algebraic/ Y
• Integer transformation is used which differs from the ormed DFT
DCT used in other CODECs. /DCT
But most of the reliable and robust watermarking 3D DFT Transformed DFT N
techniques are applied in transformed domain only, so some
modifications in the existing algorithms are required to
implement watermarking in H.264 system. REFERENCES
Moreover one of the target applications of H.264 is [1] F. Hartung and M. Kutter, “Multimedia watermarking techniques”,
videophone and video conferencing. These require criteria Proceedings of the IEEE, vol. 87, no. 7, July 1999.
[2] I. J. Cox and M. L. Miller, “Electronic watermarking: the first 50
that watermarking needs to be performed in real time. So years”. Fourth, IEEE Workshop on Multimedia Signal Processing,
now we shall discuss the applicability of different 2001, pp. 225-230.
watermarking technology in H.264 in a table 2. [3] J. Brassil, S. Low, N. Maxemchuk, and L. O’Gorman, “Electronic
marking and identification techniques to discourage document
copying,” IEEE J. Select. Areas Commun., vol. 13, pp. 1495–1504,
TABLE 2 Oct. 1995.
[4] S. Low and N. Maxemchuk, “Performance comparison of two text
Technique Domain Basic Tool Applicability marking methods,” IEEE J. Select. Areas Commun.(Special Issue on
Copyright and Privacy Protection), vol. 16, pp. 561–572, May 1998.
in H.264
[5] S. Low, N. Maxemchuk, J. Brassil, and L. O’Gorman, “Document [25] I. J. Cox, J. Kilian. F. T. Leighton and T. Shamoon, “Secure spread
marking and identification using both line and word shifting,” in spectrum watermarking for multimedia”, IEEE transactions on image
Proc. Infocom ’95, Boston, MA, Apr. 1995. processing, vol. 6, pp. 1673-1687, (1997).
[6] F. M. Boland, J. J. K. Ó Ruanaidh, and W. J. Dowling, [26] T. Wiegand, G. Sullivan, G. Bjøntegaard, and A. Luthra, Overview of
“Watermarking digital images for copyright protection,” in Proc. Int. the H.264/AVCvideo coding standard, IEEE Trans. Circuits Syst.
Conf. Image Processing and Its Applications, vol. 410, Edinburgh, Video Technol., vol. 13, pp. 560–576, July 2003.
U.K., July 1995. [27] Iain E. G. Richardson, H.264 and MPEG-4 Video Compression,
[7] M. S. Kankanhalli, Rajmohan, and K. R. Ramakrishnan, “Content- ISBN 0-470-84837-5
based watermarking of images,” in Proc. ACM Multimedia ’98, [28] Gang Qiu, Pina Marziliano, Anthony T.S. Ho, Dajun He, and Qibin
Bristol, U.K., Sept. 1998. Sun, “A hybrid watermarking scheme for H.264/AVC video” ,
[8] I. Pitas, “A method for signature casting on digital images,” in Proc. Proceedings of the 17th International Conference on Pattern
Int. Conf. Image Processing (ICIP), Lausanne, Switzerland, Sept. Recognition (ICPR’04)
1996. [29] R. B. Wolfgang, C. I. Podilchuk and E. J. Delp, “Perceptual
[9] E. Koch and J. Zhao, “Toward robust and hidden image copyright watermarks for digital images and video”, Proceedings of the IEEE,
labeling,” in Proc. Workshop Nonlinear Signal and Image vol. 87, pp. 1108-1126, (1999).
Processing, Marmaros, Greece, June 1995. [30] M. M. Reid, R. J. Millar and N. D. Black, “Second-generation image
[10] L. Boney, A. H. Tewfik, and K. H. Hamdy, “Digital watermarks for coding: An overview”, ACM Computing Surveys, vol. 29, pp. 3-29.
audio signals,” in Proc. EUSIPCO 1996, Trieste, Italy, Sept. 1996. [31] F. Deguillaume, G. Csurka, J. O’Ruanaidh and T. Pun, “Robust 3D
[11] F. Hartung and B. Girod, “Digital watermarking of raw and DFT video watermarking”, Proceeding of the SPIE, vol. 3657, pp.
compressed video,” in Proc. SPIE Digital Compression Technologies 113-124.
and Systems for Video Commun., vol. 2952, Oct. 1996, pp. 205–213.
[12] F. Jordan, M. Kutter, and T. Ebrahimi, “Proposal of a watermarking Sourav Bhattacharya was born in Kolkata, India, in
technique for hiding/retrieving data in compressed and 1982. He received the B.Tech. in Computer Science
decompressed video,” ISO/IEC Doc. JTC1/SC29/WG11 and Engineering from Institute of Engineering and
MPEG97/M2281, July 1997. Management under West Bengal University of
[13] I. Cox, J. Kilian, T. Leighton, and T. Shamoon, “Secure spread Technology in 2005. Currently he is associated with
spectrum watermarking for images, audio and video,” in Proc. IEEE Research and Development Section of Embedded
Int. Conf. Image Processing (ICIP 96), Lausanne, Switzerland, Sept. Systems group of Tata Consultancy Services Limited,
1996. Kolkata, India. His research interests include digital image processing,
[14] -----, “Digital watermarking of uncompressed and compressed video compression, digital watermarking.
video,” Signal Processing (Special Issue on Copyright Protection and
Access Control for Multimedia Services), vol. 66, no. 3, pp. 283– Tanushyam Chattopadhyay was born in Suri,
301, 1998. India, in 1976. He received the BSc in Physics from
[15] G. C. Langelaar, R. L. Lagendijk, and J. Biemond, “Realtime Visva Bharati and completed his MCA from Bengal
labeling methods for MPEG compressed video,” in Proc. 18th Symp. Engineering College, Shibpur, India, in 1998 and
Information Theory in the Benelux, Veldhoven, The Netherlands, 2002, respectively. He was awarded with the
May 1997. University Gold medal in MCA. He started his career
[16] B.Pfitzmann, ”Information Hiding Terminology”, Proc. of First Int. as research personnel in Indian Statistical Institute,
Workshop on Information Hiding, Cambridge, UK, May30-June1, Kolkata, and later on, joined the software
1996, Lecture notes in Computer Science, Vol.1174, Ross professional in Research and development section of Embedded Systems
Anderson(Ed.), pp.347-350. group of Tata Consultancy Services Limited, as an Assistant system
[17] J. S. Pan, H. C. Huang, L. C. Jain, “Intelligent Watermarking Engineer. He is one of the key programmers involved in the development
Techniques”. of a H.264 based video conferencing and video telephony system. His
[18] J. A. Bloom, I. J. Cox, T. Kalker, J. –P. M. G. Linnartz, M. L. Miller, areas of interest include video compression, digital watermarking and
and C. B. S. Traw, “Copy protection of DVD video”, Proceeding of encryption, video segmentation and summarization.
the IEEE, vol. 87, pp. 1267-1276, (1999).
[19] T. Kalker, G. Depovere, J. Haitsma, M. Maes, “A video Arpan Pal received both B.Tech. in Electronics and
watermarking system for broadcast monitoring”, proceedings of the Electrical Communication Engineering and M.Tech
SPIE, vol. 3657, pp. 103-112, (1999). degree in Telecommunication Systems Engineering
[20] Digimarc Company Website: http://www.digimarc.com from Indian Institute of Technology, Kharagpur,
[21] K. Su, D. Kundur and D. Hatzinakos, “A novel approach to India in 1990 and 1993 respectively. From 1993 to
collusion-resistant video watermarking”, Proceedings of the SPIE, 1997, he was a scientist with Research Center Imarat
vol. 4675, pp. 491-502. (RCI), a Defence Research and Development
[22] B. G. Mobasseri, “Exploring CDMA for watermarking of digital Organization laboratory at Hyderabad, India, From 1997 to 2002, he was
video”, (1999) proceedings of of the SPIE, vol. 3675, pp. 96-102. with Macmet Interactive Technologies Pvt. Ltd., Kolkata, Since 2002, he
[23] G. C. Langelaar, R. L. Lagendijk, and J. Biemond, “Realtime is with Tata Consultancy Services, Kolkata, where he is leading the
labeling of MPEG-2 compressed video,” (1998) journal of visual Convergence Solutions Practice. His areas of interests include Digital
communication and image representation, vol. 9, pp. 256-270. Signal Processing, Kalman Filters, Wireless Networks, Wireless Radio
[24] V. Darmstaedter, J. –F, Delaigle, D. Nicholson and B. Macq, “A Transceivers, Audio/Video Compression and Wireless/Multimedia
block based watermarking technique for MPEG-2 signals: Security. He has filed for some patents in the area of Wireless Security and
Optimization and validation on real digital TV distribution links”, Wireless Baseband Communication and is also involved in next generation
Proceedings 3rd European Conference on Multimedia Applications, Wireless Standardization efforts like European Union MAGNET.
Services and Techniques, pp. 190-206, 1998.

You might also like