J. Vis. Commun. Image R.

38 (2016) 2944

Contents lists available at ScienceDirect

J. Vis. Commun. Image R.

journal homepage: www.elsevier.com/locate/jvci

A robust watermarking framework for High Efficiency Video Coding

(HEVC) Encoded video with blind extraction process q
Tanima Dutta , Hari Prabhat Gupta
Dept. of Computer Science and Engineering, Indian Institute of Technology (BHU) Varanasi, India

a r t i c l e i n f o a b s t r a c t

Article history: Digital watermarking is an efficient and promising means for copyright protection of multimedia objects.
Received 7 March 2015 Digital videos are stored and transmitted in a compressed format, which has drawn a great deal of atten-
Accepted 9 December 2015 tion in compressed domain watermarking for video. The embedding and extraction of watermark bits in
Available online 2 March 2016
compressed domain, therefore does not require complete decoding and re-encoding of the compressed
video. There have been several compression standards for video. High Efficiency Video Coding (HEVC),
Keywords: a successor to H.264 Advanced Video Coding (AVC), is the latest standard for video compression with high
Blind extraction
compression efficiency. In this paper, we propose a robust watermarking framework with a blind extrac-
Compressed domain
Copyright protection
tion process for HEVC encoded video. A readable watermark is embedded invisibly in 4
4 intra pre-
Digital video dicted blocks of the HEVC encoded video. Our watermarking framework enforces security by exploring
High Efficiency Video Coding (HEVC) the spatio-temporal characteristics of the compressed video and a random key for selection of embedding
Key extraction regions. We also analyze the strengths of different compressed domain features of HEVC encoded video
Robust watermarking for implementing the embedding algorithm. Experimental results demonstrate that the proposed work
Security restrict the increase in video bit rate and degradation of perceptual quality. The proposed framework
can also survive filtering, compressions, and noise additions maintaining good quality and robustness.
2016 Elsevier Inc. All rights reserved.

1. Introduction many applications, complete decoding of video sequences is not

In the digital world of today, humans can arbitrarily and easily
access or distribute digital media from networks due to the rapid
advancement in network technology. The protection of intellectual
property becomes important for the society and needs more atten-
tion. This has been a serious threat to digital media producers and
copyright owners to protect media from a potential intruder to
avoid loss in business [1]. Digital watermarking is a favorable
method for copyright protection, where a digital code (generically
called watermark) is embedded in the digital media and typically
contains the information about the copyright owner, the creator
of the media, or the authorized customer.
Introducing watermarks in digital videos can be useful to safe-
guard copyright. The watermark can be inserted either in uncom-
pressed (raw) video [2] or compressed video [3,4]. Video signals
are often stored and transmitted in a compressed format. Applica-
tion of uncompressed watermarking techniques for compressed
video sequences, however, needs complete decoding and re-
encoding of the video for watermark embedding or detection. In
q
This paper has been recommended for acceptance by M.T. Sun.
Corresponding author.
always suitable. Consequently, compressed video watermarking
[3,4] has gained more attention. In addition, the embedding and
extraction of watermark in compressed domain has less computa-
tional overhead [3] since complete decoding and re-encoding of
the video stream is not required for both embedding and extrac-
tion of watermark bits. Several video compression standards have
been emerging in the last few years. The aim of each standard is to
provide more compressed data with better perceptual quality (or
simply quality). High Efficiency Video Coding (HEVC), also known
as H.265, is the most efficient and latest compression standard
used in a wide range of applications [5]. HEVC has been designed
to address essentially all existing applications of H.264/MPEG-4
Advanced Video Codec (AVC). HEVC mainly focus on increasing
video resolution and the use of parallel processing architectures.
HEVC provides improved compression efficiency and support for
differentiated and premium 4 K content and multi-view encoding
for broadcast and online vendors to bundle more channels/ content
on existing delivery mediums.
To minimize computational overheads and visual impacts, a
solution for encryption and watermarking compliant with H.264/
AVC and HEVC video coding standards is designed in [6]. The com-
pressed domain watermarking algorithms for H.264/AVC encoded
video are not very suitable for watermarking in HEVC encoded

http://dx.doi.org/10.1016/j.jvcir.2015.12.007
1047-3203/ 2016 Elsevier Inc. All rights reserved.
30 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
video since unlike H.264/AVC, (1) HEVC has coding tree blocks
(CTBs) of size 16
16, 32
32, and 64
64; (2) HEVC has 35 intra
modes in total with 1/32th sample accuracy (bilinear) and adaptive
smoothing filter for most reference samples; (3) Symmetric predic-
tion block (PB) partitioning (like intra) and asymmetric PB parti-
tioning; (4) In merge mode, one derived motion vector candidate
is selected based on neighbors of temporal and spatial motion vec-
tors to save bits (allows direct-like modes); (5) HEVC has more
transform sizes, such as, 4
4, 8
8, 16
16, and 32
32; and
(6) out of 35 intra prediction modes, 1, 2, and 434 are angular
modes.
In this paper, we propose a novel video watermarking frame-
work for the HEVC encoded video that has the following salient
features:
1. Quality: The degradation of the quality of the watermarked
video due to embedding distortion is minimized by appropri-
ately selecting intra predicted blocks for embedding of water-
mark bits.
2. Robustness: We propose a robust algorithm for selecting blocks
for watermark embedding based on spatio-temporal features of
the compressed video. The algorithm minimizes the synchro-
nization error. The watermark embedding and extraction algo-
rithms are also robust in nature. We also propose a framework
consisting of a public key and a private key to prevent from the
self collusion attack. The public key is extracted from spatio-
temporal characteristics of the compressed video.
3. Security: We secure the watermarking framework by using a
random key to select candidate blocks from a set of blocks
selected previously based on spatio-temporal characteristics
of the compressed video.
4. Blindness: A blind watermarking framework does not require
the original video at the decoder to extract watermark bits.
We propose a blind watermarking framework where a com-
pressed and encrypted private key is sent to the authorized cli-
ent through a secured channel to extract the watermark. This
minimizes the transmission overhead of sending a location
map of huge size.
5. Bit rate: Our watermarking framework allows a limited increase
in video bit rate by embedding watermark bits only in nonzero
quantized coefficients.
6. Complexity: We use compressed domain features to extract the
public key, select blocks for embedding, and embed the water-
mark. This decreases the computational overhead.
The rest of the paper is organized as follows: In the next section,
we briefly discuss the literature on video watermarking for HEVC
and H.264/AVC encoded video. Section 3 proposes the framework
for compressed domain watermarking in the HEVC encoded video.
We present the experimental results to evaluate the performance of
the proposed work in Section 4 and conclude the paper in Section 5.
2. Related work and motivation
A very limited literature exists on watermarking in HEVC
encoded video and thus, watermarking in HEVC encoded video is
still in its rudimentary phase. Swati et al. has proposed a fragile
watermarking scheme, where the watermark is embedded in the
least significant bit (LSB) of the nonzero quantized coefficients in
the HEVC encoded video [7]. Ogawa et al. propose a watermarking
method for HEVC/H.265 video streams that embeds information
during the encoding phase of the video [8]. Many literatures exist
on compressed domain watermarking for H.264/AVC encoded
video. In this paper, we also focus on review of literature on com-
pressed domain watermarking in H.264 encoded video. Zhang and
Ho [9] proposed an embedding scheme for video watermarking,
where the authentication information is represented by a prepro-
cessed binary sequence and embedded into mid frequency coeffi-
cients in I frame. To improve the security in watermark, the sign
of the coefficients is changed based on the watermark. The water-
marking scheme presented in [9] improved in [10] by focusing on
gray scale patterns and characters. Qiu et al. [11] proposed embed-
ding the robust watermark onto quantized discrete cosine trans-
form (DCT) coefficients of intra frames and the fragile watermark
into motion vectors of inter frames. Kuo and Lo [12] improved
the video watermarking technique proposed in [11] by choosing
more appropriate locations for embedding both robust and fragile
watermarks in H.264 video during video encoding.
In [13], the watermark is embedded by replacing some bits in
the bit-stream directly, but the pre-embedding analysis is compu-
tationally complex. The same authors proposed a robust and non-
blind watermarking technique by using the Watson Visual Model
[14] for embedding in I frame [15]. The proposed non-blind tech-
nique [15] extended for the P frame in [16], where watermark bits
are embedded in all nonzero coefficients of P frame.
A blind method to embed in I frames of the compressed video is
proposed in [3]. Block selection is performed based on the value of
prediction modes and the number of nonzero quantized coeffi-
cients (NNZ) of a 4
4 block. The nonzero coefficients are changed
to zero value for hiding watermark bits. A data hiding framework is
designed in [17] to select the embedding region using the concepts
of forbidden-zone-data-hiding. The parity of sign of the coefficients
and the value of mid frequency coefficients are changed for water-
marking in I frame [18]. In [19], the watermark embedded in non-
zero coefficients of P frame in compressed domain to have a better
perceptual quality of the watermarked video and a limited increase
in video bit rate. In [20], a non-blind H.264/CAVLC structure-
preserving substitution watermarking algorithm is proposed that
embeds watermark by bit substitutions in the motion vector differ-
ences of non-reference frames. Su et al. [21] proposed a non-blind
method to embed watermark in I frames and P frames. A water-
mark is embedded using spread spectrum and coefficients are
selected using the Watson Visual Model [14].
 Motivation: The paper is motivated by the following limitations
observed in the literature on video watermarking for HEVC
encoded video. In [7], the scheme is highly sensitive to the quan-
tization parameter (QP) value since the watermark is embedded
in the LSB of DCT coefficients. In [8], a watermark is embedded
during the encoding phase of the HEVC/H.265 video. HEVC is
built on the platform of H.264/AVC, therefore, the paper is also
motivated by the following limitations observed in the existing
literature for watermarking in H.264/AVC encoded video. The
methods proposed in [9,10,18] are fragile to re-encoding and
recompression attacks and the preprocessing phase is computa-
tionally expensive. Moreover, the work in [9] simply applied the
MPEG-style algorithm to H.264/AVC and did not tailor to the
characteristics of H.264/AVC. The schemes described in
[15,16,22] are non-blind in nature, so original video is required
to extract the watermark at the decoder. In addition, complete
decompression and recompression of the video is required in
[15,16] for watermarking that restricted the scope of applica-
tions. The watermarking technique presented in [15] requires
a computationally expensive prediction process for watermark
embedding and has higher increase in video bit rate. The blind
algorithm in [3] has changed the nonzero coefficients to zero
value for hiding the watermark that significantly degrades the
quality. In [17], the error correction capability features of
the Reed-Solomon codes are exploited to provide robustness.
The method proposed in [21] is non-blind that embed water-
mark in I frames and P frames of H.264/AVC encoded video.
 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944 31

+
Partially

Rate of change in Modes

Selected Encoded Random 0.04
Watermark HEVC Key
Video
0.03 +
Selection of Public Private Compessed
different Key Key Watermarked 0.02
compressed
Public domain features HEVC +
Key Video 0.01 +
Extraction Palette
Generation + +
Block Selection +
0
+ +
Blind Extraction 0 1 2 3 4 5 6 7 8
Candidate Block
Private Number of NonZero Quantized Residuals (NNZ)
Key Selection
Generation HEVC (a) Change in Prediction Modes with NNZ
Embedding Extracted Uncompessed
Feature Selection Watermark Watermarked
HEVC Video

Rate of change in Modes

Compessed 0.4
Watermarked Blind
Embedding +
Video 0.3 +
+
(a) Embedding Framework (b) Extraction Framework 0.2 +
+
Fig. 1. Block diagram of the proposed watermarking framework for HEVC encoded +
video.
0.1 +
+ +
+ +
The compressed domain watermarking methods proposed in 0
0 2 4 6 8 10 12 14 16 18 20
[19,23] have the following limitations:
Motion Vector Field
(1) The watermark is embedded only in inter-predicted 4
4
blocks of P frames in H.264/AVC encoded video, which limits
(b) Change in Prediction Modes with Motion Vector
the embedding capacity; (2) the robustness threshold is selected Fig. 2. Change in Prediction Modes with NNZ and Motion Vector. The change in
randomly and not optimized; (3) the method is fragile to syn- luminance intra prediction modes decreases with the increase in the value of NNZ
chronization error; and (4) the size of the location map is huge, in a block. Similarly, the change in luminance intra prediction modes is higher for
no motion (static) and high motion blocks.
which adds transmission overhead. In [20], the quality of the
watermarked video is poor due to change in motion vectors of
the blocks. therefore, if any attacker wants to manipulate I frames, the percep-
In summary, there exist only few contributions in the literature tual quality of the video degrades highly [3,17,15]. Next, we select
on compressed domain watermarking for HEVC encoded video. appropriate luminance component of 4
4 intra predicted blocks
Considering the limitations of HEVC and H.264/AVC watermarking in I frames for invisibly embedding a bipolar watermark sequence
in the literature, we propose a robust watermarking framework 1; 1. If a smooth region of a frame is selected as embedding
with a blind extraction process for HEVC encoded video. The region for watermark embedding, the framework may cause visible
increase in video bit rate is also minimized. The compressed artifacts [3]. An efficient algorithm for selecting a suitable embed-
domain features are only explored for selecting embedding region, ding region of a frame uses the video content to get appropriate
embedding watermark, and extracting the public key. The frame- features, which would provide a bound on the degradation of per-
work is made secured by use of a random key for selecting the ceptual quality and increase in video bit rate. However, these fea-
blocks for watermark embedding. tures need to be robust enough to avoid the possibility of changes
after watermark insertion or re-encoding [3]. In other words, the
appropriate features would be robust to prevent desynchroniza-
3. Proposed framework tion during watermark extraction in the HEVC encoded video.

In this section, we propose a robust watermarking framework
for HEVC encoded video. The proposed framework first selects can-
didate blocks in I frames, known as embedding region, for embed-
ding a watermark in HEVC encoded video. Next, we have
presented the watermark embedding and extraction algorithms.
We also illustrate the extraction of the public key and the estima-
tion of embedding thresholds. The block diagram of the proposed
framework is shown in Fig. 1.
Synchronization error in intra coded blocks in HEVC encoded video:
The increase in desynchronization in the watermark detection
process due to embedding distortion in compressed domain
watermarking is known as synchronization (re-encoding) error.
The intra prediction modes have block sizes of 32
32 down to
4
4 in HEVC encoded video. There are multiple prediction
modes in I frames so it is very hard to track the change in pre-
diction modes.

3.1. Embedding region We consider the following criteria for selecting the 4
4 intra
predicted blocks in I frames of HEVC encoded video.
The efficiency of a watermarking framework depends on the
selection of an embedding region. For selecting the embedding  Smooth regions of a frame: In I frame of the HEVC encoded
region, we first select appropriate I frames where watermark can video, a block is said to be a SMOOTH mode block, if it has
be embedded. We work on I frames since such frames convey a 32
32, 32
16, 16
32, or 16
16 prediction mode. The
wide range of information compared to P and B frames and blocks which have such prediction modes are not used for
32 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
watermark embedding. These blocks are used for smooth
regions of a frame and watermark embedding in a smooth
region may cause visible artifacts. In addition, due to embed-
ding distortion, the blocks in the smooth region have a tendency
to change into different mode blocks. This increases desynchro-
nization in the watermark extraction process at the decoder.
Therefore, our framework avoids the SMOOTH mode blocks
for watermark embedding.
 Number of nonzero quantized coefficients: The number of
nonzero quantized coefficients (NNZ) plays an important role
in selecting blocks for embedding that can minimize synchro-
nization error [3]. Part (a) of Fig. 2 shows the relationship
between the rate of change in prediction modes and NNZ in I
frames. The result presented in the figure is with 80% confi-
dence level though the error bars are not visible in the plots.
Fig. 2(a) is based on the nature of CIF and QCIF videos. The figure
illustrates that the synchronization error decreases as the value
of the NNZ is increased from 0 to 8. It can be observed from
the figure that the blocks with higher values of NNZ may be
selected for watermark embedding. With the increase in the
value of NNZ, the number of blocks available for watermark
embedding decreases. However, the exact number of blocks
for embedding depends on the spatial characteristics of the
video [3]. An appropriate threshold, denoted by N t , would be
therefore selected based on the spatial characteristics of each
sequence. For example, Fig. 2(a) shows that a block having
NNZ greater than 5 has less synchronization error, i.e., N t 5.
 Motion information: Embedding uncorrelated watermarks in
successive frames usually results in annoying temporal flicker
and visual artifacts. Motion information is necessary for avoid-
ing such temporal flicker. The degradation of quality of water-
marked video due to any perturbation in a low motion region
is less than that of a higher motion region or static region (no
motion). The value of motion vectors in I frames are zero since
such frames are intra coded. The extraction of pseudo motion
vectors of I frames based on motion vectors at the same location
in the nearest P frame of the previous group of pictures (GOP) is
described in [24] as follows:
1. Divide P frames into non-overlapping blocks.
2. Calculate motion vectors for all blocks in P frames of previ-
ous GOP.
3. Smooth all motion vectors by using a 3
3 median filter.
4. Motion vectors of intra-coded blocks have zero values in P
frames, so estimate motion vector for intra-coded blocks
from neighboring blocks in the respective largest coding
unit (LCU).
5. Assign pseudo motion vector to each block in I frames by
interpolating motion vectors at the same location in the
nearest P frame.
Procedure 1. Block Selection
Finally, the pseudo motion vector for every block in the I frame
is obtained. If the value of pseudo motion vector of a block is
greater than zero, then the block can be selected for watermarking.
Part (b) of Fig. 2 shows the relationship between the rate of change
in prediction modes and pseudo motion vectors. It illustrates that
the embedding in low motion regions provides an acceptable
robustness against synchronization error due to embedding. An
appropriate threshold, denoted by Mt , would therefore be selected
for embedding the watermark, where M t > 0. All blocks are
selected for watermark embedding, where the value of nonzero
pseudo motion vector is less than Mt . The value of threshold M t
depends on the relationship between the rate of change in
 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
prediction modes and pseudo motion vector. In this paper, we con-
sider the value of M t as 20 based on the Fig. 2(b). The result pre-
sented in the figure is with 80% confidence level though the error
bars are not visible in the plots. Fig. 2(b) is based on the nature
of CIF and QCIF videos.
Procedure 2. Function MagRound
 Quality: The proposed procedure for embedding region selec-
tion incorporates a visual threshold, denoted by V t , for main-
taining acceptable perceptual quality of the watermarked
video. The coefficients obtained from DCT transform are divided
into DC coefficient and AC coefficients in a 4
4 block. We work
on AC coefficients since such coefficients are less sensitive to
embedding error compared to DC coefficients. Let AC 1 and AC 2
are first two nonzero AC coefficients in a DCT transformed
block. If the absolute difference between AC 1 and AC 2 of a block
is greater than the threshold V t , then that block may be used for
watermark embedding. In Section 3.5, we estimate the value of
visual threshold V t for HEVC encoded videos.
The complete process of selecting 4
4 intra predicted blocks
that are suitable for watermark embedding is given by Procedure
1, where the function MagRound() is described in Procedure 2.
Procedure 3. Candidate Block Selection
33
3.2. Key extraction
In each Iframe, candidate blocks are selected for embedding
using a random key from the blocks that already selected in Proce-
dure 1. The candidate blocks are those blocks where the watermark
is actually embedded. The selection of candidate blocks using a
pseudo random key enhances the security of the proposed frame-
work. Different frames in a video will have different keys, which
are generated randomly using a pseudo random number generator
to avoid self-collusion attack [25]. The security of the algorithm is
imposed by randomly selecting candidate blocks using Procedure 3
from the set of blocks selected in Procedure 1.
If the watermark is embedded in the same location in correlated
I frames then the embedding algorithm becomes vulnerable to the
self-collusion attack [25]. Thus, a very long sequence is required.
Transmitting a long sequence, however, would make the algorithm
impractical. Such problem can be solved by generating the
sequence from a combination of a public key, which is extracted
from some features of the LCU, i.e., a block of size 64
64 and a
private key that is possessed by the copyright owner. The public
key is extracted from each LCU and pass to a cryptographic system
with the private key.
The compressed domain features available from HEVC codec
information are used for generating the public key without any fur-
ther decoding. The public key would be extracted from some fea-
tures of the LCU that cannot be changed by the attacker without
degrading the perceptual quality of the video. To make the
extracted public key robust, some features of the LCU, which are
sensitive to human eyes, can be used. One such feature is DC coef-
ficients of the LCU. If DC coefficients themselves are used for public
key extraction, then the attacker could change DC coefficients by
the same amount, which will make watermark detection impossi-
ble. In I frames, changing a nonzero DC value to zero or zero DC
value to nonzero will degrade the quality of the watermarked
video and even create blocking artifacts [3]. Another feature is
the chrominance prediction modes of the LCU.
A 256 bits long public key extracted from each LCU is based on
DC components and chrominance prediction (chroma) modes in
the LCU as shown in Fig. 3. The first 64 bits give the DC coefficient
of every 4
4 block in the LCU. The last 192 bits show chroma
modes of 4
4 blocks in each LCU. Each chroma mode is repre-
sented by 3 bits.
The private key is estimated from the candidate block locations
and the public key. The security of the watermarking systems are
generally not made as complex as cryptographic systems. In this
34 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
work, public key and private key are encrypted using modulo 2
arithmetic. The encrypted keys are compressed losslessly using
run length coding. The public key is available to all clients, whereas
the private is sent to the authorized client through secure channel.
The complete process of key extraction is elaborated in Procedure 4.
The number of candidate blocks selected in Procedure 3 and the
number of blocks selected in Procedure 1 are denoted by m and n,
respectively. A random key is used to select m blocks out of n
blocks, i.e., candidate blocks. In the proposed framework, the cryp-
tographic space for security is n Cm . As the value of m increases, the
proposed watermarking framework will be more robust.
Procedure 4. Key Extraction
3.3. Watermark embedding
A bipolar watermark sequence 1; 1 is embedded invisibly in
candidate blocks in I frames of HEVC encoded video using com-
pressed domain features. In the proposed watermarking frame-
work, we embed a watermark bit in a candidate block by
changing the value of first two nonzero AC coefficients, i.e. AC 1
and AC 2 .
To increase robustness of our watermarking framework, we use a
robustness threshold, denoted by Rt , where Rt > 0. The proposed
watermark embedding procedure is described as follows: If the
value of watermark bit is 1 and the magnitude value of AC 1 is less
than or equal to the magnitude value of AC 2 , then the value of mod-
ified coefficients are denoted by AC 01 AC 1 D signD
Rt and
AC 02 AC 2  D  signD
Rt , where the value of D is
MagRoundAC 1 ; AC 2 . Similarly, if the value of watermark bit is 1
and the magnitude value of AC 1 is greater than the magnitude value
of AC 2 , then the value of modified coefficients are denoted by AC 01
AC 1  D  signD
Rt and AC 02 AC 2 D signD
Rt . If the value
of watermark bit is 1 and the magnitude value of AC 1 is greater
than AC 2 or value of watermark bit is 1 and the magnitude value
of AC 1 is less than or equal to AC 2 , then no changes will be made.
If the value of Rt is large, the absolute difference between AC 1
and AC 2 will be large, which increases the robustness, but the
perceptual quality of the watermarked video will degrade signifi-
cantly. In this section, the watermark embedding algorithm pro-
posed in [19,23] is modified and the formal algorithm of
proposed watermark embedding in HEVC encoded video in com-
pressed domain is facilitated by Procedure 5. In Section 3.5, we
estimate the robustness threshold Rt for different video sequences.
Procedure 5. Watermark Embedding
If watermark is not embedded in appropriate coefficients, then
embedding algorithm may not robust and the watermarked video
may have visual artifacts. Therefore, the selection of appropriate
 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944 35

Table 1
Robustness threshold and visual threshold.

Video Rt D Vt H.264/AVC HEVC

PSNR eD PSNR eD
Coastguard 1 15 5 29.73 0.5 35.03 0.6
610 10 29.68 0.7 34.24 0.8
1115 15 28.92 1 34.02 1
3 15 5 29.71 0.8 34.69 0.7
610 10 29.04 1 33.88 0.9
1115 15 28.85 0.7 33.21 1
5 15 5 30.01 1 34.83 0.8
610 10 29.59 0.8 34.02 1
1115 15 29.49 0.7 32.57 0.7
Garden 1 15 5 28.53 0.7 35.44 0.7
610 10 27.45 1 34.87 0.9
1115 15 26.67 0.8 34.11 1
3 15 5 28.49 0.7 34.24 0.5
610 10 28.3 0.9 33.61 0.7
1115 15 28.28 1 33.14 1
5 15 5 28.7 1 33.85 0.8
610 10 28.3 0.8 32.98 1
1115 15 28.21 0.7 31.94 0.7
Husky 1 15 5 29.71 0.6 36.02 0.6
610 10 29.67 0.8 35.74 0.8
1115 15 28.22 1 35.04 1
3 15 5 29.69 0.8 35.64 0.7
610 10 29.04 1 34.61 1
1115 15 28.08 0.6 33.86 0.8
5 15 5 29.63 0.7 35.08 0.8
610 10 28.19 1 34.24 1
1115 15 27.44 0.9 33.37 0.9
Soccer 1 15 5 29.8 0.5 35.29 0.8
610 10 29.1 0.7 33.98 1
1115 15 28.92 1 32.89 0.7
3 15 5 29.09 0.7 34.91 0.8
610 10 28.78 0.9 33.79 1
1115 15 28.11 1 32.94 0.6
5 15 5 29.03 0.8 34.43 0.7
610 10 28.12 1 33.86 1
1115 15 27.94 0.7 32.42 0.6

coefficients or features for watermark embedding is very crucial. In

a candidate block, one or more compressed domain features, such Zero or Nonzero DC Value
as NNZ, sign and value of the quantized coefficients, and sign and
value of motion vectors, may be used for watermark embedding. 0 1 ... 63 64 65 ... 255
We use the following criteria for selection of appropriate features:
Chroma Prediction Mode
 Magnitude of low frequency nonzero quantized AC coeffi-
cients: Low frequency coefficients are more stable to synchro- Fig. 3. Extraction of public key from DC coefficients and chroma prediction modes
nization error [15]. Such coefficients can be changed to embed of each LCU.

watermark bits in a block [15,19]. It can maintain the degrada-

tion of quality and increase in video bit rate and synchroniza-
tion error.
 Sign of AC coefficients: In [9,10,18], the sign of AC coefficients
 NNZ value: In [3], the number of nonzero quantized AC coeffi-
is changed to embed watermark bits, which degraded the qual-
cients (NNZ) is decreased by changing the nonzero coefficients
ity of the watermarked video highly. Therefore, the sign of the
to zero value to embed a watermark bit, which increases syn-
coefficients is not changed to embed watermark.
chronization error. A block, which is actually predicted from
 High frequency coefficients: High frequency AC coefficients
such a watermarked block, may have a better match with a
are mostly reset to zero due to synchronization error or by
neighbor of that watermarked block at the decoder. Moreover,
the use of filters [26].
changing nonzero coefficients to zero degrades quality signifi-
cantly. Similarly, changing zero coefficients to nonzero will also
In summary, only low frequency nonzero quantized AC coeffi-
increase desynchronization at the decoder. Furthermore, chang-
cients in 4
4 blocks of I frames are suitable for watermark
ing zero coefficients to nonzero will result in significant
embedding. To reduce complexity, the first two nonzero quantized
increase in video bit rate. Therefore, the value of NNZ of a block
AC coefficients, namely AC 1 and AC 2 , in zigzag sequence order in a
is not changed to embed watermark.
4
4 block are changed for embedding a watermark bit. Water-
 DC coefficients: DC coefficients are not used for watermark
mark embedding in such coefficients will not create significant
embedding because the quality of the watermarked video will
visual artifacts. The values of such coefficients do not change much
degrade significantly and may even cause blocking artifacts
on desynchronization due watermarking noise.
[16].
36 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
3.4. Watermark extraction
The watermark extraction is performed at the decoder after
entropy decoding. The public key is scrambled using the private
key to generate the resultant sequence (palette) which specifies
the candidate block locations used for embedding. A palette is a
binary file sequence that contains the location formation of candi-
date block [3]. It is used in the decoder for watermark extraction. In
the proposed framework, a palette is generated from the combina-
tion of a private key and a public key at the decoder. In most liter-
ature [18,15,3,17], the palette is sent to the decoder through
secured channels. In the proposed work, only a compressed private
key is sent securely to the authorized client for extraction of water-
mark bits. In absence of palette at the decoder, the candidate
blocks will be selected using Procedure 1 and the random key,
which is used in Procedure 2. However, in absence of palette, the
robustness of the watermarking framework decreases. Due to syn-
chronization error the location of the blocks selected at the deco-
der will not be exactly same as that of the locations selected at
the encoder.
In an I frame, the first two nonzero coefficients of a candidate
block of the watermarked video are denoted by AC 01 and AC 02 . If
the magnitude value of AC 01 is greater than AC 02 , the extracted
watermark bit will be 1, otherwise the value of watermark bit
is 1. The formal procedure for watermark extraction is facilitated
by Procedure 6.
Procedure 6. Extract and Authenticate
3.5. Visual threshold and robustness threshold
The visual threshold (denoted by V t in Section 3.1) is used to
control the degradation of perceptual quality of the watermarked
video. If the value of V t is small, then degradation of quality will
be minimized, but the number of blocks selected for embedding
will be reduced. Furthermore, the quality of the watermarked
video also partially depends on robustness threshold (denoted by
Rt in Section 3.3). If the value of Rt is small, then robustness will
decrease, but the degradation of quality will be reduced. With
the increase in the value of Rt , the absolute value of the modified
coefficient (i.e., AC 01 and AC 02 ) will be high. Therefore, higher value
of Rt increases the robustness of the proposed framework, but
may significantly degrade the quality. In other words, if the value
of D is small, then higher value of Rt can be used. If the value of
D in a block is large, then smaller value of Rt may be used. In such
a scenario, the robustness of the proposed framework will be high
and degradation of perceptual quality will also reduced. However,
the maximum value of D can be V t . Higher value of Rt , therefore,
implies smaller value of V t . Intuitively, embedding capacity is pro-
portional to the visual threshold V t . If the value of V t is small, then
the embedding capacity is reduced.
Based on the desired quality, capacity, and robustness of the
watermarking framework, a percentage of 4
4 blocks, namely k,
containing a high value of D, is selected for embedding. In this case,
by choosing a higher value for k, the embedding capacity and
robustness will increase but the quality will decrease. For a lower
value of k, the embedding capacity and robustness will decrease
while the quality will increase. To obtain a target value of s, the
endurance function of D distribution, denoted by eD , is utilized as
follows:
X
eD s PD < s pD < k; 1
D<s
where the endurance function depends on the probability distribu-
tion function. Based on Eq. (1), for the same k, different values of s
will be achieved depending on the spatial activity of the sequences.
The achieved value for s is utilized in order to establish V t and Rt .
The estimated values of V t and Rt are used to select suitable 4
4
blocks for embedding. Table 1 shows the values of V t and Rt for dif-
ferent video sequences. We have changed the value of D; Rt , and V t ,
to get the value of eD . In each video sequence, for each value of Rt ,
the value of D and V t are chosen, where the value of eD is maximum.
The value of average PSNR over payload 2 f100; 200; 300g is esti-
mated for both HEVC and H.264/AVC videos. The value of D is taken
in the range of 15. Similarly, the value of Rt 2 f1; 3; 5g and
V t 2 f5; 10; 15g. The results in the table are shown for four different
CIF and QCIF video sequences, such as Coastguard, Garden, Husky,
and Soccer. As the resolution of the compressed video change, the
spatial characteristics as well as temporal characteristics per 4
4
block changes. As a result, the value of number of nonzero quan-
tized coefficients and motion vector of a block may change. There-
fore, the value of V t and Rt will also change for High Definition (HD)
videos. Similarly, if the configuration parameters of HECV or H.264/
AVC codec changes, the spatial characteristics as well as temporal
characteristics per block will also change and as a result the value
of V t and Rt will change. In Table 1, the corresponding values of
PSNR are highlighted in bold whenever the value of eD is 1.
4. Experimental results
The proposed watermark framework is implemented using
H.265/HEVC reference software HM9.0 [27]. The advanced tools
in anchor HM9.0 have already been thoroughly studied in JCT-
VC. Simulations were performed under common conditions set
by the JCT-VC without any optimization. The All-intra Main config-
uration is used in the experiments, where the RDQ and RDQTS are
disabled. Common conditions are intended to reflect the typical
 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944 37

Fig. 4. Embedding capacity using Procedure 1 for (a) Husky (b) Soccer (c) Flower and (d) Mobile videos (HEVC and H.264/AVC).

Table 2 bit-stream and is the configuration used under which coding effi-
PSNR of unwatermarked encoded video sequences for HEVC and H.264/AVC of QCIF, ciency of proposals are evaluated. We select the configuration of
CIF, and HD format. random-access-main with closed-GOP. The value of intra refresh
Sequence H.264/AVC HEVC type is set to 2.
PSNR (dB) PSNR (dB) The video sequences in common interchange format (CIF) and
Flower 28.55 34.93
quarter CIF (QCIF) [28,29] at payload = {100, 150, 200, 250, 300}
Ice 27.63 34.67 on an average of 100 frames are coded using a dyadic high-delay
Coastguard 28.49 34.58 hierarchical prediction structure with a GOP having intra frame
Garden 27.02 33.54 period value as 8 and 4 active reference pictures, where payload
Paris 26.35 35.71
is the amount of watermark bits actually embedded in a video
Trevor 28.21 33.89
Husky 28.38 33.27 sequence. The input videos in our experiments have different prop-
Football 28.91 35.75 erties, such as highly textured or less textured, low motion or high
Soccer 28.31 34.56 motion, with or without change in the shot, and uniform or noisy
Mobile 28.41 35.01
background. The quantization parameter (QP) is set to 28 [22].
Old town cross 54.09 64.91
Park joy 53.11 61.11
The values of V t and Rt are determined by using Eq. (1). The values
Crowd run 56.92 68.28 of M t and N t are determined by using Fig. 2. The proposed water-
Ducks take off 56.32 65.82 marking framework is also implemented using H.264/AVC refer-
In to tree 57.01 68.61 ence software JM17.2 and compared with the existing literature
38 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944

Table 3
Results for PSNR, VQM, and BIR using the proposed watermarking framework.

Sequence Payload H.264/AVC HEVC

2 3
PSNR (dB) VQM
10 BIR
10 PSNR (dB) VQM
102 BIR
103
Flower 100 28.09 2.99 1.24 34.45 1.84 0.76
150 28.08 3.39 1.92 33.96 2.15 1.44
200 27.99 3.63 1.49 33.45 2.38 1.01
250 27.92 3.78 1.87 33.32 2.93 1.39
300 27.88 5.06 1.42 32.69 3.31 0.94
Ice 100 27.13 3.58 1.58 34.07 2.03 1.1
150 27.12 3.62 1.66 34 2.47 1.18
200 27.08 4.79 1.27 33.36 3.28 0.79
250 27.06 5.53 2.69 33.15 3.44 2.21
300 27.01 4.03 1 31.67 3.68 0.52
Coastguard 100 28.06 3.27 1.97 34.01 2.28 1.49
150 28 3.36 1.44 33.83 2.37 0.96
200 27.95 3.42 1.06 32.86 2.51 0.58
250 27.83 3.69 1.38 32.02 2.86 0.9
300 27.81 4.07 1.6 31.85 3.04 1.12
Garden 100 26.86 5.18 1.23 33.27 2.52 0.75
150 26.83 5.27 2.19 33.02 2.73 1.71
200 26.82 5.33 1.99 32.63 3.51 1.51
250 26.74 5.38 2.09 31.55 3.99 1.61
300 26.34 5.31 1.27 30.94 4.23 0.79
Paris 100 26.06 4.32 1.51 35.37 2.48 1.03
150 25.95 4.3 1.07 34.08 2.63 0.59
200 25.74 4.28 1.88 34.76 2.72 1.4
250 25.62 4.08 1.89 33.27 3.52 1.41
300 25.34 3.87 1.89 32.72 3.9 1.41
Trevor 100 28.1 4.6 1.65 33.41 2.25 1.17
150 27.38 4.26 0.92 32.92 2.84 0.44
200 27.31 3.68 2.19 32.22 3.22 1.71
250 27.17 3.77 1.81 31.23 3.43 1.33
300 27 3.47 1.72 30.99 3.5 1.24
Husky 100 28.16 5.38 1.93 35.27 2.77 1.45
150 27.83 5.28 2.09 35.03 3.13 1.61
200 27.22 5.23 2.19 34.62 3.51 1.71
250 26.94 5.09 2.11 33.76 3.92 1.63
300 26.68 4.81 1.2 32.95 4.26 0.72
Football 100 28.56 4.23 1.49 35.37 2.48 1.01
150 28.24 4.13 1.37 34.08 2.03 0.89
200 28.05 3.98 2.03 34.76 2.46 1.55
250 27.92 3.91 1.96 33.52 2.8 1.48
300 27.81 3.87 1.89 32.65 3.4 1.41
Soccer 100 28 4.46 1.75 34.21 2.15 1.27
150 27.78 4.22 1.53 34.02 2.34 1.05
200 27.52 3.98 2.22 33.32 2.75 1.74
250 27.27 3.75 2.01 32.76 3.23 1.53
300 27.08 3.66 1.82 32.35 3.32 1.34
Mobile 100 28.01 3.16 1.68 34.91 1.41 1.2
150 27.68 3.32 1.55 34.72 2.55 1.07
200 27.32 3.91 2.23 33.75 2.99 1.75
250 27.07 4.05 2.11 32.96 3.13 1.63
300 26.78 4.16 1.97 32.55 3.36 1.49

[3,18,15,17], which are based on H.264/AVC and an appropriate
similar configuration of HEVC is used for coding video sequence
using H.264. We kept the payload same for every sequence
encoded in HEVC and H.264.
4.1. Embedding capacity
The number of blocks suitable for watermark embedding is the
embedding capacity of the video. In this work, we select the blocks,
which are suitable for watermark embedding, based on the spatio-
temporal characteristics of a video. Thus, the embedding capacity
of the proposed work depends on block selection criteria of Proce-
dure 1. Total number of blocks suitable for embedding selected in a
GOP in different HEVC and H.264/AVC video sequences are
depicted in Fig. 4.
In the HEVC encoded video, NNZ is comparatively less than
H.264/AVC video in most of the frames. Therefore, most of the
H.264/AVC encoded video sequences have higher embedding
capacity than HEVC encoded video. Embedding capacity is inver-
sely proportional to the motion threshold M t . Furthermore, embed-
ding capacity is proportional to the visual threshold. Therefore,
embedding capacity is a function of k, denoted by Fk. The embed-
ding capacity, denoted by C E , is given by
C E Fk;
where the value of k / Nt
M
1
t
, k > eD s / V t , and eD is given in Eq.
(1). The embedding capacity of a video depends on the spatial and
temporal characteristics of that video per 4
4 block. Therefore, if
most blocks of a video have higher texture and are in motion, then
that video will have higher embedding capacity.
 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944 39

Fig. 5. Perceptual quality is measured using average PSNR and VQM as shown in part (a) and part(b) of figure, respectively. Part (c) of figure shows BIR to estimate the video
bit rate.

4.2. Visual quality, peak signal-to-noise ratio, and bit increase rate
In this section, we measure the perceptual quality of the water-
marked video using Peak Signal-to-Noise Ratio (PSNR) and Visual
Quality Metric (VQM) at payload = {100, 150, 200, 250, 300}. We
also measure the increase in video bit rate using the metric Bit
Increase Rate (BIR). Bit Increase Rate (BIR) is defined as the percent-
age of bit rate increase per embedded bit [19], i.e.,
 into a single metric. It shows a high correlation with subjective
number of bits in watermarked video
BIR
payload
number of bits in original video
 of quality. When the payload is very less, the perceptual quality of
number of bits in original video

100;
payload
number of bits in original video
where payload is the amount of watermark bits actually embedded
in a video sequence.
PSNR, VQM, and BIR of the proposed framework for HEVC and
H.264/AVC encoded video are shown in Table 3. The values of PSNR
of unwatermarked encoded video sequences for HEVC and H.264/
AVC are shown in Table 2. VQM results in Table 3 are in the order
of 102 , which indicate that the watermarked videos have accept-
able quality. Video quality is a characteristic of a video passed
through a video transmission/processing system, a formal or infor-
mal measure of perceived video degradation (typically, compared
to the original video). The application VQM creates an objective
comparison of video codecs and measures the perceptual effects
of video impairments, including blurring, jerky motion, global
noise, block distortion and color distortion, and combines them
video quality assessment. Higher value implies higher degradation
the watermarked video is almost same as the unwatermarked one.
2
As the payload increases, the amount of degradation of quality also
increases. However, the degradation is quality also depends on the
embedding region, where watermark is embedded. PSNR also con-
firms that the proposed embedding scheme does not sacrifice the
quality very much. The degradation of the quality in terms of PSNR
due to watermark embedding is about or less than 1 dB for all
videos. This confirms that the degradation will not be perceived
40 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
Fig. 6. Change in quality and robustness of the watermarked video, which is
quantized with QP = 28, due to re-encoding by changing QP in a range from 20 to
36.
by human eyes. In case of BIR, the increase in video rate is in the
order of 103 which is nominal. HEVC shows better quality and less
increase in video bit rate for most of the videos due to its coding
Fig. 7. Change in quality and robustness by changing parameters for four different image processing attacks.
efficiency. The perceptual quality of the HEVC encoded water-
marked video is better than that of the H.264/AVC encoded video
for same payload. Similarly, increase in video bit rate in HEVC
encoded video is less than that of the H.264/AVC encoded water-
marked video.
The perceptual quality of the watermarked video is compared
with recent existing schemes based on the Peak Signal-to-Noise
Ratio (PSNR) and Visual Quality Metric (VQM). Increase in video
bit rate is also calculated compared with existing schemes. Fig. 5
(a) and (b) shows the average PSNR and average VQM in the case
of the proposed framework and the existing work [3,18,15,17].
The results are the average over five different payloads, i.e., {100,
150, 200, 250, 300}. The results for HEVC encoded video confirm
that the degradation of quality of the watermarked video will not
be perceived by human eyes. The result also shows that the pro-
posed framework for H.264/AVC encoded video outperforms other
schemes in perceptual quality for most of the videos. However, the
number of nonzero coefficients in Coastguard and Paris videos is
very less as the frames are highly correlated. Therefore, the syn-
chronization error is high in such blocks as shown in Fig. 2(a). This
causes higher perceptual error. In Trevor video, the number of
change in shots is high. Therefore, different thresholds will be more
appropriate for different shots. Thus, watermark embedding in
such frames by using a single threshold for visual quality increases
the degradation of the quality. The perceptual quality of the water-
marked video for HEVC is much better than H.264/AVC encoded
 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
Fig. 8. Average robustness against re-encoding attacks. Part (a) shows the robustness against recompression error. Part (b) shows the robustness against decreases of QP from
28 to 26. Part (c) shows the robustness against increases QP from 28 to 30.
video. The proposed watermark embedding algorithm and embed-
ding region selection algorithm are designed in such a way that the
degradation of quality of the watermarked video will be minimum.
In case of BIR, Fig. 5(c) shows that the increase in the video bit rate
is negligible. It also compares the proposed framework with the
existing literature [3,18,15,17]. If the zero coefficients are changed
to nonzero or the motion vectors are changed to embed water-
mark, then the increase in video bit rate is significant. In the pro-
posed watermarking framework, only two nonzero coefficients in
a block are perturbed to embed a watermark bit. Neither the value
of NNZ, motion vectors or zero coefficients of a block is changed. In
HEVC encoded videos, increase in the video bit rate is reduced
since HEVC provides appropriately 4050% reduction in the bit rate
at the same visual quality compared to H.264.
4.3. Robustness against attacks
In this section, we measure robustness of the proposed water-
marking framework against different attacks. Robustness can be
defined as how efficiently a watermarking system can resist differ-
41
ent innocent or malicious attacks on the video. The robustness of a
watermarking method [19] is given by
Robustness 1  BER
100; 3
where Bit Error Rate (BER) is defined as the frequency of bit errors
when detecting a multi-bit watermark message [30], i.e.,
number of error bits
BER : 4
total number of bits sent
BER is used for evaluating the robustness of the proposed work
against different attacks and compared with the existing water-
marking schemes [18,15,3,17].
The information about the embedding locations is saved as a
palette during the embedding process [3,18,15,17]. This palette is
provided to the decoder for the extraction process. In the absence
of palette, if a watermarked location is not detected or a non-
watermarked position is selected incorrectly, the synchronization
in the watermark sequence will be lost. This will decrease the
robustness of the a watermarking scheme. Palettes are used to
know about the embedding locations during the watermark
extraction process for both the proposed framework and the
42 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944

Table 4
Robustness against four different kinds of attacks for different video sequences.

Attacks Parameter Sequence Schemes

H.264/AVC HEVC [18] [15] [3] [17]
Salt and pepper Salt and pepper noise density = 0.001 Flower 83 88 65 67 81 75
Ice 82 86 67 66 77 72
Coastguard 76 80 61 63 75 72
Garden 82 86 65 68 77 78
Paris 83 87 69 69 82 72
Trevor 80 86 66 68 87 74
Husky 81 86 67 69 85 75
Football 82 86 67 70 76 76
Soccer 80 85 68 68 81 74
Mobile 79 84 69 67 80 73
Gaussian noise Gaussian noise density = 0.001 Flower 76 80 66 63 77 77
Ice 78 82 65 64 74 76
Coastguard 77 82 62 65 78 74
Garden 82 86 63 67 74 76
Paris 75 81 66 66 76 74
Trevor 77 81 61 64 81 73
Husky 76 83 65 65 81 74
Football 79 85 67 69 80 77
Soccer 82 84 61 67 78 75
Mobile 80 85 62 68 79 76
Circular averaging filter Circular averaging filter r = 0.06 Flower 73 81 61 61 79 74
Ice 78 83 63 63 75 70
Coastguard 78 84 61 64 79 72
Garden 81 86 66 66 78 73
Paris 80 85 63 67 79 75
Trevor 79 83 65 65 83 74
Husky 78 86 66 68 83 71
Football 80 87 68 67 82 70
Soccer 81 86 69 68 79 73
Mobile 82 88 68 67 80 74
Gaussian filter Gaussian filter r = 0.05 Flower 74 80 62 61 78 73
Ice 76 82 64 64 77 72
Coastguard 77 83 60 63 79 72
Garden 82 87 65 66 78 75
Paris 81 85 64 65 80 70
Trevor 80 84 66 66 82 74
Husky 75 87 67 69 81 71
Football 82 86 66 64 83 72
Soccer 80 84 68 65 80 74
Mobile 81 85 69 68 79 73

existing literature [3,18,15,17]. Thus, presence of palette increases
robustness in watermark detection.
In [22], it is shown that by increasing the quantization param-
eter (QP), the compression efficiency increases but the quality of
the watermarked video decreases. To get acceptable quality with
highly compressed video, the QP of 28 is chosen [22]. The robust-
ness of the proposed framework are evaluated against re-encoding
attack [3], such as recompression error and changing the value of
quantization parameter. The proposed watermarking framework
is also tested against different noise additions and filtering attacks,
such as salt and pepper noise, gaussian noise, circular averaging fil-
ter, and gaussian filter.
We first depicted the robustness results of the proposed frame-
work by changing the value of QP in Fig. 6. Initially, the watermark
is embedded when the value of QP is 28. The watermarked video is
re-encoded by changing the value of QP from 28 to a range of 20
36. We also check the change in quality using PSNR by changing
the value of QP. It is clear from the result that the HEVC encode
video sequences have better quality compare to H.264/AVC encode
video sequences when QP is changed. The results in the figure are
the mean over all sequences which are mentioned in the Table 3.
HEVC encoded videos are more robust to re-encoding attack.
Fig. 6 shows the results where the quality and robustness are plot-
ted for watermarked video with respect to unwatermarked video
for both H.264/AVC and HEVC. Similarly, changing the variance
for salt and pepper noise, standard deviation of gaussian noise,
and radius of gaussian filter and circular averaging filter are shown
in Fig. 7. HEVC encoded video mostly provides better quality and
robustness against different noise additions with respect to
H.264 video. Furthermore, HEVC video has better quality and
robustness when blurred lightly using averaging filter and poor
quality when blurred significantly in comparison to H.264/AVC
encoded video. Similarly, HEVC video has better quality and
robustness when blurred lightly using a gaussian filter in compar-
ison to H.264/AVC encoded video.
Next, Fig. 8 compares average robustness results for re-
encoding attacks, i.e., recompression error and changing QP from
28 to 26 and 28 to 30, respectively, with the existing literature
[3,18,15,17] providing palette to the decoder for extraction of
watermark bits. In case of recompression error, neither the value
of QP nor any other parameter is changed. The watermarked video
is re-encoded with similar parameters and changes in synchroniza-
tion in watermark bits are noted. HEVC encoded videos provides
higher robustness than H.264/AVC encoded videos in case of re-
encoding attacks. The proposed framework for H.264/AVC outper-
forms in most of the videos. The shot change in Trevor video is fre-
quent. A fixed threshold value for different shots does not give a
good result. However, our results for H.264 videos are comparable
with the existing literature. We can achieve better results by
changing the values of thresholds M t , N t , V t , and Rt when a change
in the shot is encountered. It is clear from the figure that the pro-
posed framework is highly robust against recompression errors.
 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
Table 5
Average PSNR, average BIR, and average robustness against recompression error (re-encoding attack) for five different HEVC encoded HD videos with two different resolutions.
HD video 1080p Resolution
3
PSNR (dB) BIR
10
Old town cross 53.36 0.35
Park joy 52.22 0.34
Crowd run 56.23 0.28
Ducks take off 55.86 0.32
In to tree 56.81 0.29
The proposed watermarking framework for both HEVC and H.264/
AVC encoded video outperforms the existing literature.
Finally, Table 4 illustrates the average robustness of the pro-
posed watermarking framework against noise additions, such as,
salt and pepper noise and gaussian noise and blurring using circu-
lar averaging filter and gaussian filter, at the decoder for extraction
of the watermark and compared with the existing literature
[3,18,15,17]. A palette is used for the proposed framework as well
as existing literature. HEVC encoded video outperforms against dif-
ferent noise additions and filtering attacks in comparison to H.264/
AVC encoded video. It is observed from the result that the proposed
work for H.264/AVC encoded video shows better results than exist-
ing literature for most of the videos. An appropriate selection of
embedding region and changing low frequency nonzero coefficient
to embed watermark make the proposed framework robust. In
short, HEVC encoded video provides better robustness against
image processing attacks using the proposed watermarking
framework.
4.4. Performance on High Definition (HD) videos
HEVC has adopted many advanced encoding tools. The size of
high definition (HD) video is a massive. There is no way for its con-
venient transmission. Instead, the video is compressed to reduce
the amount of data into a more manageable form. A high lossy
compression technique will minimize the size of the video, but
degrade the quality significantly. HEVC is an efficient compression
technique for compressing HD videos. It can significantly improve
the compression performance of HD videos and save half of the bit
rate compared to the H.264/AVC for the same perceptual video
quality. Thus, the benefit of HEVC in the streaming space is that
it can relate the encoding of existing standard-definition (SD)
and HD content better than that of H.264 by enabling cost savings
and the ability to stream higher quality video to lower bit-rate
connections.
In this section, we implement the proposed watermarking
framework on HD videos. We evaluate the results for quality,
increase in video bit rate using average PSNR and average BIR,
respectively using HEVC encoding for different HD videos [28] at
payload = {100, 150, 200} as shown in Table 5. We also evaluate
the average robustness against recompression error (re-encoding
attack) for HD videos using HEVC encoding and are shown in
Table 5. The resolution of HD videos are 1080p and 2160p. The
results show that the degradation of the quality of the water-
marked video and the increase in video bit rate are reduced. It is
clear from the results that the proposed watermarking framework
for HD videos is robust against recompression error. As the resolu-
tion of a compressed video changes, the number of nonzero quan-
tized coefficients and the value of motion vector of a block will also
change. In Section 3, the values of M t and N t in Fig. 2 and the values
of V t and Rt in the proposed algorithm are estimated based on the
nature of CIF and QCIF videos. The existing literature [16,31,22,19]
used for comparison also have used CIF and QCIF videos. The
threshold values in the existing literature are based on CIF and
QCIF videos. Therefore, for the sake of a fair comparison, the results
43
2160p Resolution
Robustness PSNR (dB) BIR
103 Robustness
91 64.26 0.23 93
95 60.69 0.22 95
94 67.73 0.21 92
93 65.21 0.19 91
92 68.04 0.18 90
for PSNR, BIR and robustness against recompression error for HD
videos are not compared with any existing literature.
5. Conclusion
In this paper, we proposed a novel watermarking framework
with a blind extraction process that embeds a readable watermark
invisibly into the I frames of HEVC encoded video. A comprehen-
sive description of the proposed video watermarking framework,
including block selection, public key extraction, watermark
embedding, and watermark extraction, is given. The experiments
are conducted to demonstrate that the proposed watermarking
framework is robust against image processing attacks and resistant
against re-encoding attacks. The watermarking security is imposed
by randomly selecting the candidate blocks. We conducted a series
of experiments to prove its effectiveness with respect to perceptual
quality and increase in video bit rate. The use of compressed
domain features in selecting blocks for watermark embedding,
the embedding and extraction algorithms, and the extraction of
public key reduces the computational complexity. The proposed
framework is implemented on different video resolutions. In
future, the proposed framework can be extended to provide a com-
mon platform to compressed domain watermarking for a video
encoded using different codecs, i.e., codec independent platform
and may address more classes of attacks.
References
[1] Y. Tew, K. Wong, An overview of information hiding in H.264/AVC compressed
video, IEEE Trans. Circ. Syst. Video Technol. 24 (2) (2014) 305319.
[2] F. Hartung, B. Girod, Watermarking of uncompressed and compressed video,
Sig. Process. 66 (3) (1998) 283301.
[3] A. Mansouri, A. Aznaveh, F. Torkamani, F. Kurugollu, A low complexity video
watermarking in H.264 compressed domain, IEEE Trans. Inf. Forens. Sec. 5 (4)
(2010) 649657.
[4] C. Lu, C. Hsu, Near-optimal watermark estimation and its countermeasure:
antidisclosure watermark for multiple watermark embedding, IEEE Trans. Circ.
Syst. Video Technol. 17 (4) (2007) 454467.
[5] G. Sullivan, J. Ohm, W.-J. Han, T. Wiegand, Overview of the high efficiency
video coding (HEVC) standard, IEEE Trans. Circ. Syst. Video Technol. 22 (12)
(2012) 16491668.
[6] A. Boho, G. Van Wallendael, A. Dooms, J. De Cock, G. Braeckman, P. Schelkens,
B. Preneel, R. Van De Walle, End-to-end security for video distribution: the
combination of encryption, watermarking, and video adaptation, IEEE Sig.
Process. Magaz. 30 (2) (2013) 97107.
[7] S. Swati, K. Hayat, Z. Shahid, A watermarking scheme for high efficiency video
coding (HEVC), PLoS One 9 (08) (2014) e105613.
[8] K. Ogawa, G. Ohtake, Watermarking for HEVC/H.265 stream, in: Proceedings of
ICCE, 2015, pp. 102103.
[9] J. Zhang, A.T. Ho, Efficient video authentication for H.264/AVC, in: Proceedings
of ICICIC 3, 2006, pp. 4649.
[10] J. Zhang, A. Ho, G. Qiu, P. Marziliano, Robust video watermarking of H.264/AVC,
IEEE Trans. Circ. Syst. II: Exp. Briefs 54 (2) (2007) 205209.
[11] G. Qiu, P. Marziliano, A. Ho, D. He, Q. Sun, A hybrid watermarking scheme for
H.264/AVC video, in: Proceedings of ICPR 04, 2004, pp. 865869.
[12] T. Kuo, Y. Lo, A hybrid scheme of robust and fragile watermarking for H.264/
AVC video, in: Proceedings of BMSB, 2010, pp. 16.
[13] M. Noorkami, R. Mersereau, Compressed-domain video watermarking for
H.264, in: Proceedings of ICIP 2, 2005, pp. 890893.
[14] A.B. Watson, DCT quantization matrices visually optimized for individual
images 1913, 1993, pp. 202216.
44 T. Dutta, H.P. Gupta / J. Vis. Commun. Image R. 38 (2016) 2944
[15] M. Noorkami, R.M. Mersereau, A framework for robust watermarking of
H.264-encoded video with controllable detection performance, IEEE Trans Inf
Forens Sec 2 (1) (2007) 1423.
[16] M. Noorkami, R. Mersereau, Digital video watermarking in P-frames with
controlled video bit-rate increase, IEEE Trans. Inf. Forens. Sec. 3 (3) (2008)
441455.
[17] E. Esen, A. Alatan, Robust video data hiding using forbidden zone data hiding
and selective embedding, IEEE Trans. Circ. Syst. Video Technol. 21 (8) (2011)
11301138.
[18] D. Xu, R. Wang, J. Wang, A novel watermarking scheme for H.264/AVC
video authentication, Sig. Process.: Image Commun. 26 (6) (2011) 267
279.
[19] T. Dutta, A. Sur, S. Nandi, A robust compressed domain video watermarking in
P-frames with controlled bit rate increase, in Proceedings of NCC 2013, 2013,
pp. 15.
[20] T. Stutz, F. Autrusseau, A. Uhl, Non-blind structure-preserving substitution
watermarking of H.264/CAVLC inter-frames, IEEE Trans. Multim. 16 (5) (2014)
13371349.
[21] P. Su, C. Wu, I. Chen, C. Wu, Y. Wu, A practical design of digital video
watermarking in H.264/AVC for content authentication, Sig. Process.: Image
Commun. 26 (89) (2011) 413426.
[22] J. Li, H. Liu, J. Huang, Y.Q. Shi, Reference Index-based H.264 video
watermarking scheme, ACM Trans. Multim. Comput. Commun. Appl. 8 (2S)
(2012) 33:133:22.
[23] T. Dutta, Robust Compressed Domain Video Watermarking for H.264/AVC, Ph.
D. Thesis, Indian Institute of Technology Guwahati, 2014.
[24] T. Dutta, A. Sur, S.K. Nandi, MCRD: motion coherent region detection in H.264
compressed video, in: Proceedings of ICME, 2013, pp. 16.
[25] B. Furht, O. Marques, Handbook of Video Databases: Design and Applications,
first ed., CRC Press, 2003.
[26] M. Noorkami, Secure and Robust Compressed-Domain Video Watermarking
for H.264, Ph.D. Thesis, Georgia Institute of Technology, 2007.
[27] (2012) HEVC Reference Software <http://hevc.kw.bbc.co.uk/trac/browser/
jctvc-hm/tags>.
[28] (2015) Xiph.org Video Test Media [derfs collection] <https://media.xiph.org/
video/derf/>.
[29] (2015) Database: Images and Video Clip <http://see.xidian.edu.cn/
vipsl/databaseVideo.html>.
[30] D. Salomon, G. Motta, D. Bryant, An Engineers guide to Automated Testing of
High-Speed Interfaces, Artech House, 2010.
[31] G. Feng, G. Wu, Motion vector and mode selection based fragile video
watermarking algorithm, in: Proceedings of ASID, 2011, pp. 7376.