SMART MIRROR REPORT

Figure 1: Determining the size and movement of the FOV according to mirror dynamics

Figure 2: Determining the distance from mirror using the radius of the face detected
Constants and Variables
lcd_length: max num of column of the LCD screen given by the user

lcd_height: max num of row of the LCD screen given by the user

(LCD resolution = lcd_length X lcd_height  the frame to be cropped should be at this

ratio)

rearcamera_length = total number of columns of the rear cameras (combined frames) in

pixels (length of the combined rear camera frames  columns of camera left + columns
of camera right)

rearcamera_height = max row of the rear cameras in pixels (height of the combined
rear camera frames  max row of camera left = max row of camera right)

facecamera_length = max column of the face camera frame in pixels

facecamera_height = max row of the face camera frame in pixels

facecamera_middle_col = center column of the face camera frame

facecamera_middle_row = center row of the face camera frame

 mirror_length = length of mirror given by the user

mirror_height = height of mirror given by the user

(Mirror size = mirror_length X mirror_height  the size of the mirror given

dramatically changes the behavior of the system)

DifX: X distance, center of face detected from the facecamera_middle_col in pixels

DifX = CenterXface_detected - facecamera_middle_col

DifY: Y distance, center of face detected from the facecamera_middle_row in pixels

DifY = CenterYface_detected - facecamera_middle_row

face_radius = the radius of the face detected in pixels (will be used to determine the
distance from mirror/camera, see Figure 2).

ratio_triangle = 15*900 (see Figure 2)

rectangle_length & rectangle_height : the size of the rectangle calculated with respect to the face
detected (distance from camera). This is the area where the image is visualized in the LCD screen
after resizing to LCD ratio. Thus, the rectangle ratio should be equal to LCD ratio to keep original
ratio of the image cropped.

The size of the Rectangle (regarding the size of the face detected)
i. The ratio of the rectangle should be equal to LCD ratio to be able to keep the original ratio.
ii. The size of the rectangle gets bigger and smaller according to distance from mirror and mirror
size (see Figure 1),

ratio_triangle
𝐫𝐞𝐚𝐥_𝐝𝐢𝐬𝐭𝐚𝐧𝐜𝐞 =
face_radius
(distance from mirror) (see Figure 2)

mirror_length
𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗 = (see Figure 1)
real_distance

mirror_height
𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐘 = (see Figure 1)
real_distance

iii. The maximum size of the rectangle should be tuned to the ratio between LCD resolution (LCD
ratio) and the resolution of the combined frames obtained from the rear cameras

lcd_length
𝐥𝐜𝐝_𝐫𝐚𝐭𝐢𝐨 =
lcd_height

rearcamera_length
𝐫𝐞𝐚𝐫𝐜𝐚𝐦𝐞𝐫𝐚_𝐫𝐚𝐭𝐢𝐨 =
rearcamera_height
1. Determining the maximum size of the rectangle while keeping the lcd ratio (i) :
a. if lcd_ratio = rearcamera_ratio then (the ideal case)

𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐦𝐚𝐱_𝐥𝐞𝐧𝐠𝐭𝐡 = rearcamera_length

𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐦𝐚𝐱_𝐡𝐞𝐢𝐠𝐡𝐭 = rearcamera_height

b. if lcd_ratio < rearcamera_ratio then

lcd_length
𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐦𝐚𝐱_𝐥𝐞𝐧𝐠𝐭𝐡 = rear_camera_height ∗
lcd_height

𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐦𝐚𝐱_𝐡𝐞𝐢𝐠𝐡𝐭 = rear_camera_height

c. if lcd_ratio > rearcamera_ratio then

𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐦𝐚𝐱_𝐥𝐞𝐧𝐠𝐭𝐡 = rear_camera_length

lcd_height
𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐦𝐚𝐱_𝐡𝐞𝐢𝐠𝐡𝐭 = rear_camera_length ∗
lcd_length

2. Making sure that the size of the rectangle reaches its maximum size at a given distance

min_distance_Z: The minimum distance from the face camera defined by the user: This
distance will be the distance where the rectangle reaches the maximum size determined by
the rectangle_max_length X rectangle_max_height

max_distance_Z: The maximum distance from the face camera defined by the user: This
distance will be the distance where the rectangle reaches the minumum size

rectangle_standard_length: we define a standard length to make sure that the rectangle

reaches the maximum size at min_distance_Z (in other words at the maximum 𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗)

Therefore, we define rectangle_standard_length as it follows:

rectangle_max _length mirror_length
𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐬𝐭𝐚𝐧𝐝𝐚𝐫𝐝_𝐥𝐞𝐧𝐠𝐭𝐡 =
max _FOV_const_X
, where 𝐦𝐚𝐱_𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗 = min_distance_Z

Since we determine the current rectangle length as it follows:

𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐥𝐞𝐧𝐠𝐭𝐡 = 𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐬𝐭𝐚𝐧𝐝𝐚𝐫𝐝_𝐥𝐞𝐧𝐠𝐭𝐡 ∗ 𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗 (ii)

Similarly,
rectangle_max _height
𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐬𝐭𝐚𝐧𝐝𝐚𝐫𝐝_𝐡𝐞𝐢𝐠𝐡𝐭 =
max _FOV_const_X

(Note that here we used

𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐡𝐞𝐢𝐠𝐡𝐭 = 𝐫𝐞𝐜𝐭𝐚𝐧𝐠𝐥𝐞_𝐬𝐭𝐚𝐧𝐝𝐚𝐫𝐝_𝐡𝐞𝐢𝐠𝐡𝐭 ∗ 𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗
FOV_const_X instead of FOV_const_Y in order to keep rectangle ratio = lcd_ratio)
The movement of the Rectangle (regarding the movement of the face detected)
1. The Principles

The face movement is defined by x,y and z movements where,

DifX = CenterXface_detected - facecamera_middle_col

DifY = CenterYface_detected - facecamera_middle_row

ratio_triangle
𝐫𝐞𝐚𝐥_𝐝𝐢𝐬𝐭𝐚𝐧𝐜𝐞 =
face_radius

And the quantity of the rectangle movement is tuned to the distance from the mirror/camera (see Figure 1)

mirror_length
𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗 =
real_distance

mirror_height
𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐘 =
real_distance

Therefore, regarding the real_distance, x and y movement should be as it follows:

𝑫𝒊𝒇𝑿 = 𝐷𝑖𝑓𝑋 ∗ 𝐹𝑂𝑉_𝑐𝑜𝑛𝑠𝑡_X

𝑫𝒊𝒇𝒀 = 𝐷𝑖𝑓𝑌 ∗ 𝐹𝑂𝑉_𝑐𝑜𝑛𝑠𝑡_Y

2. Making sure that the rectangle can reach the boundry of the rear camera frame

Here we define FOVX_movement_regulator and FOVY_movement_regulator in order to make sure

that even at the minumum rectangle size, the rectangle can travel at the x and y
boundaries of the frame acquired from the rear cameras.

Such that,

𝑫𝒊𝒇𝑿 = 𝐷𝑖𝑓𝑋 ∗ 𝐹𝑂𝑉_𝑐𝑜𝑛𝑠𝑡_X * FOVX_movement_regulator , and

𝑫𝒊𝒇𝒀 = 𝐷𝑖𝑓𝑌 ∗ 𝐹𝑂𝑉_𝑐𝑜𝑛𝑠𝑡_Y * FOVY_movement_regulator

Should satisy the condition that we are able to see the boundaries of the rear frame
considering the size of the minumum rectangle length and the length of the face
detected at this distance. Since the area remained for the face movement should be
able to move the rectangle in the area that remained in the rear camera frame.

For instance, considering x movement at the distance of the minumum rectangle size, if
the area remained for the face movement is 100 pixels and the area remained for the
rectangle movement is 200 pixels, the rectangle should move 2 pixels for every 1
pixels of the face movement and if the FOV_Const_X is 0.5 here, then the
FOVX_movement_regulator has to be 4 in order to satisfy the condition.
Therefore we calculate the rectangle_min_length and rectangle_min_height at the
max_distance_Z.

At max_distance_Z,

mirror_length
we obtain the minumum value of FOV_const_X (𝐦𝐢𝐧_𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗 = max_distance_Z
)

therefore, the minumum rectangle length and the minumum rectangle height are as it follows:

𝐦𝐢𝐧_𝒓𝒆𝒄𝒕𝒂𝒏𝒈𝒍𝒆_𝒍𝒆𝒏𝒈𝒕𝒉 = rectangle_standard_length ∗ 𝐦𝐢𝐧_𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗

𝐦𝐢𝐧_𝒓𝒆𝒄𝒕𝒂𝒏𝒈𝒍𝒆_𝒉𝒆𝒊𝒈𝒉𝒕 = rectangle_standard_height ∗ 𝐦𝐢𝐧_𝐅𝐎𝐕_𝐜𝐨𝐧𝐬𝐭_𝐗

The face width at this distance is:

𝑟𝑎𝑡𝑖𝑜_𝑡𝑟𝑖𝑎𝑛𝑔𝑙𝑒
𝐟𝐚𝐜𝐞_𝐰𝐢𝐝𝐭𝐡_𝐚𝐭_𝐦𝐢𝐧_𝒓𝒆𝒄𝒕𝒂𝒏𝒈𝒍𝒆_𝒍𝒆𝒏𝒈𝒕𝒉 =
𝑚𝑎𝑥_𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒_𝑍

The area remained for the x and y movement of the face is:

face_width_at_min_𝑟𝑒𝑐𝑡𝑎𝑛𝑔𝑙𝑒_𝑙𝑒𝑛𝑔𝑡ℎ
𝒓𝒆𝒎𝒂𝒊𝒏𝒆𝒅_𝒙_𝒎𝒐𝒗𝒆𝒎𝒆𝒏𝒕_𝒇𝒂𝒄𝒆 = facecamera_middle_Col −
2

face_width_at_min_𝑟𝑒𝑐𝑡𝑎𝑛𝑔𝑙𝑒_𝑙𝑒𝑛𝑔𝑡ℎ
𝒓𝒆𝒎𝒂𝒊𝒏𝒆𝒅_𝒚_𝒎𝒐𝒗𝒆𝒎𝒆𝒏𝒕_𝒇𝒂𝒄𝒆 = facecamera_middle_Row −
2

And the area remained for the x and y movement of the rectangle is:

rearcamera_length − min_𝑟𝑒𝑐𝑡𝑎𝑛𝑔𝑙𝑒_𝑙𝑒𝑛𝑔𝑡ℎ
𝒓𝒆𝒎𝒂𝒊𝒏𝒆𝒅_𝒙_𝒎𝒐𝒗𝒆𝒎𝒆𝒏𝒕_𝒓𝒆𝒄𝒕𝒂𝒏𝒈𝒍𝒆 =
2

rearcamera_height − min_𝑟𝑒𝑐𝑡𝑎𝑛𝑔𝑙𝑒_ℎ𝑒𝑖𝑔ℎ𝑡
𝒓𝒆𝒎𝒂𝒊𝒏𝒆𝒅_𝒚_𝒎𝒐𝒗𝒆𝒎𝒆𝒏𝒕_𝒓𝒆𝒄𝒕𝒂𝒏𝒈𝒍𝒆 =
2

Therefore, the ratio has to be:

𝑟𝑒𝑚𝑎𝑖𝑛𝑒𝑑_𝑥_𝑚𝑜𝑣𝑒𝑚𝑒𝑛𝑡_𝑟𝑒𝑐𝑡𝑎𝑛𝑔𝑙𝑒
𝒎𝒖𝒔𝒕_𝒓𝒂𝒕𝒊𝒐_𝒙 =
𝑟𝑒𝑚𝑎𝑖𝑛𝑒𝑑_𝑥_𝑚𝑜𝑣𝑒𝑚𝑒𝑛𝑡_𝑓𝑎𝑐𝑒

𝑟𝑒𝑚𝑎𝑖𝑛𝑒𝑑_𝑦_𝑚𝑜𝑣𝑒𝑚𝑒𝑛𝑡_𝑟𝑒𝑐𝑡𝑎𝑛𝑔𝑙𝑒
𝒎𝒖𝒔𝒕_𝒓𝒂𝒕𝒊𝒐_𝒚 =
𝑟𝑒𝑚𝑎𝑖𝑛𝑒𝑑_𝑦_𝑚𝑜𝑣𝑒𝑚𝑒𝑛𝑡_𝑓𝑎𝑐𝑒

Finally,

𝑚𝑢𝑠𝑡_𝑟𝑎𝑡𝑖𝑜_𝑥
FOVX_movement_regulator =
min_FOV_const_X

𝑚𝑢𝑠𝑡_𝑟𝑎𝑡𝑖𝑜_𝑦
FOVY_movement_regulator =
min_FOV_const_Y
3. Smoothing the rectangle movement
To smoothen the rectangle movement we use gaussian function for the x (DifX),y (DifY) and z movement
(DifZ = face_radius):

We let user define the gaussian size and the sigma (σ), where the gaussian function is:

𝒙𝟐
−𝟐 ∗ 𝝈
𝒇(𝒙) = 𝒆

Due to fact that we do not have the further frames, we use the current frame and the previous frames
depending on the size of the gaussian.

Such that,

If the gaussian size given is 11 and the σ is 2 then the distribution becomes:

[0.0088 0.0271 0.0651 0.1216 0.1770 0.2006 0.1770 0.1216 0.0651 0.0271 0.0088]
[DifXt-5 DifXt-4 DifXt-3 DifXt-2 DifXt-1 DifXt - - - - - ]
[DifYt-5 DifYt-4 DifYt-3 DifY t-2
DifY t-1
DifYt
- - - - - ]
[DifZt-5 DifZt-4 DifZt-3 DifZt-2 DifZt-1 DifZt - - - - - ]

Where DifXt, DifYt and DifZt are the current movements of the face, which are
multiplied by the middle gaussian element (0.2006) and we multiple the previous
movements by the rest of the gaussian elements respectively so that the influence
of the current movements have the higher effect on the movement.

Looking at the half gaussian array, one should see that the we only used the %60 of
the actual movements in the example given above.

Figure 3: Gaussian VS. Half Gauss (DifX)

Thus, we define gauss_regulator in order to keep the previos calculations which we

used for “Making sure that the size of the rectangle reaches its maximum size at a given
distance” and “Making sure that the rectangle can reach the boundry of the rear camera
frame”
gauss_regulator = 1/total_HalfGauss_percentage (1/%60 = 1/0.6 = 1.66 for the
example above)

At figure 3 it is clear that using half gaussian filter reduces the movements size.
However, after using gauss_regulator we obtain a similar function like full gauss
(see figure 4)

Figure 4: Half Gauss Regulated (DifX)

Figure 5: Half Gauss ΔDifX (size of movements between the DifX, DifXt- DifXt-1 )
 Figure 6: Half Gauss Regulated ΔDifX

Figure 7: Full Gaussian Function ΔDifX

Figure 7 shows the output if we had used a full gaussian filter. On the other hand Figure 5 shows the half-
gaussian filter and Figure 6 shows the ΔDifX function after applying gauss_regulator to the
half-gaussian function, which is used in the application.

4. To stop vibrating of the rectangle movements (Ignoring the default face movements)

Although when the face is not moving there will be small movements which turn out to be undesirable
vibratings.

Hence, we define a threshold (movement_threshold) for ΔDifX, ΔdifY and ΔdifZ so that we keep
the rectangle movements stable, where
ΔDifXt = DifXt – DifXt-1
ΔDifYt = DifYt – DifYt-1
ΔDifXt = DifZt – DifZt-1

And, 𝒗𝒆𝒄𝒕𝒐𝒓_𝒍𝒆𝒏𝒈𝒕𝒉_𝟑𝑫 = √ΔDifX 2 + ΔDifY 2 + ΔDifZ 2 ,

IF vector_length_3D < movement_threshold Do Not Move

However, in order to see when the face starts vibrating even though the user is not moving, we use more
than 2 previous movements taking mean of them (avg_delta_DifX, avg_delta_DifY, avg_delta_DifZ)

For instance, if the gaussian size is defined 11, the size of the previous movements we keep will be 6.
Therefore, in this scenerio, we obtain the average of the 5 previous movements and the current
movement to see if there is vibrating.

avg_delta_DifX = MEAN( [ΔDifXt-5 ΔDifXt-4 ΔDifXt-3 ΔDifXt-2 ΔDifXt-1 ΔDifXt])

avg_delta_DifY = MEAN( [ΔDifYt-5 ΔDifYt-4 ΔDifYt-3 ΔDifYt-2 ΔDifYt-1 ΔDifYt])
avg_delta_DifZ = MEAN( [ΔDifZt-5 ΔDifZt-4 ΔDifZt-3 ΔDifZt-2 ΔDifZt-1 ΔDifZt])

𝒂𝒗𝒈_𝒗𝒆𝒄𝒕𝒐𝒓_𝒍𝒆𝒏𝒈𝒕𝒉_𝟑𝑫 = √𝑎𝑣𝑔_𝑑𝑒𝑙𝑡𝑎_𝐷𝑖𝑓𝑋 2 + avg_delta_DifY 2 + avg_delta_DifZ 2 ,

IF avg_vector_length_3D < movement_threshold Do Not Move

*** Note that, using 19 gaussian size with σ = 3 we obtained desirable results.
*** When the movement_threshold = 2.5 we were able to stop vibratings without affecting the rectangle
movements negatively.