EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

EcoStruxure Augmented
Operator Advisor
Builder
User Manual
10/2017
EIO0000003006.00

www.schneider-electric.com
 The information provided in this documentation contains general descriptions and/or technical
characteristics of the performance of the products contained herein. This documentation is not
intended as a substitute for and is not to be used for determining suitability or reliability of these
products for specific user applications. It is the duty of any such user or integrator to perform the
appropriate and complete risk analysis, evaluation and testing of the products with respect to the
relevant specific application or use thereof. Neither Schneider Electric nor any of its affiliates or
subsidiaries shall be responsible or liable for misuse of the information contained herein. If you
have any suggestions for improvements or amendments or have found errors in this publication,
please notify us.
You agree not to reproduce, other than for your own personal, noncommercial use, all or part of
this document on any medium whatsoever without permission of Schneider Electric, given in
writing. You also agree not to establish any hypertext links to this document or its content.
Schneider Electric does not grant any right or license for the personal and noncommercial use of
the document or its content, except for a non-exclusive license to consult it on an "as is" basis, at
your own risk. All other rights are reserved.
All pertinent state, regional, and local safety regulations must be observed when installing and
using this product. For reasons of safety and to help ensure compliance with documented system
data, only the manufacturer should perform repairs to components.
When devices are used for applications with technical safety requirements, the relevant
instructions must be followed.
Failure to use Schneider Electric software or approved software with our hardware products may
result in injury, harm, or improper operating results.
Failure to observe this information can result in injury or equipment damage.
© 2017 Schneider Electric. All Rights Reserved.

2 EIO0000003006 10/2017
 Table of Contents

Safety Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
About the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Getting Started with EcoStruxure Augmented Operator Advisor Builder 13
Overview of the EcoStruxure Augmented Operator Advisor Builder
Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Chapter 2 Projects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Chapter 3 Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Chapter 4 Points of Interest. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Points of Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Variable Points of Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Document Points of Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
External App Points of Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Chapter 5 Scenes and Subscenes . . . . . . . . . . . . . . . . . . . . . . . . . 31
Scenes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Adding Subscenes to a Scene . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Adding Lists to Scenes or Subscenes . . . . . . . . . . . . . . . . . . . . . . . . . 37
Using Variables in Scenes or Subscenes . . . . . . . . . . . . . . . . . . . . . . 39
Using Images in Scenes or Subscenes. . . . . . . . . . . . . . . . . . . . . . . . 40
Using Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Chapter 6 Building a Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Building Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Appendices ......................................... 47
Appendix A Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
How to Choose Between Image and Tag Recognition . . . . . . . . . . . . 50
Guidelines for Taking Photographs . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Tagging Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Glossary ......................................... 59

EIO0000003006 10/2017 3
4 EIO0000003006 10/2017
 Safety Information

Important Information

NOTICE
Read these instructions carefully, and look at the equipment to become familiar with the device
before trying to install, operate, service, or maintain it. The following special messages may appear
throughout this documentation or on the equipment to warn of potential hazards or to call attention
to information that clarifies or simplifies a procedure.

EIO0000003006 10/2017 5
PLEASE NOTE
Electrical equipment should be installed, operated, serviced, and maintained only by qualified
personnel. No responsibility is assumed by Schneider Electric for any consequences arising out of
the use of this material.
A qualified person is one who has skills and knowledge related to the construction and operation
of electrical equipment and its installation, and has received safety training to recognize and avoid
the hazards involved.

6 EIO0000003006 10/2017
 About the Book

At a Glance

Document Scope
This document describes how to use the EcoStruxure Augmented Operator Advisor Builder
software to create an augmented reality project.
This document is intended for project developers.
Read the EcoStruxure Augmented Operator Advisor Quick Start Guide for an overview of
EcoStruxure Augmented Operator Advisor before using this document.
It is recommended that you undertake an EcoStruxure Augmented Operator Advisor training
course before using the product.

Validity Note
This document is valid for Version 2.0 of EcoStruxure Augmented Operator Advisor.

Related Documents

Title of Documentation Reference Number

EcoStruxure Augmented Operator Advisor Quick Start Guide EIO0000003000 (ENG)
EIO0000003001 (FRE)
EIO0000003002 (GER)
EIO0000003003 (SPA)
EIO0000003004 (ITA)
EIO0000003005 (CHS)
EIO0000003032 (JPN)

You can download these technical publications and other technical information from our website
at http://www.schneider-electric.com/en/download

Product Related Information

The application of this product requires expertise in the design and operation of control systems.

EIO0000003006 10/2017 7
 WARNING
LOSS OF CONTROL
 The designer of any control scheme must consider the potential failure modes of control paths
and, for certain critical control functions, provide a means to achieve a safe state during and
after a path failure. Examples of critical control functions are emergency stop and overtravel
stop.
 Separate or redundant control paths must be provided for critical control functions.
 System control paths may include communication links. Consideration must be given to the
implications of unanticipated transmission delays or failures of the link.
 Each EcoStruxure Augmented Operator Advisor implementation must be individually and
thoroughly tested for proper operation before being placed into service.
Failure to follow these instructions can result in death, serious injury, or equipment damage.

CAUTION
FALSE SCENE DETECTION
 Use tags in situations where similar equipment and/or environments could result in false scene
detection.
 Attach tags firmly to the equipment with sufficient protection against damage or deterioration
of the tag.
Failure to follow these instructions can result in injury or equipment damage.

8 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

Chapter 1
Introduction

Introduction

Introduction
This chapter introduces the EcoStruxure Augmented Operator Advisor Builder.

What Is in This Chapter?

This chapter contains the following topics:
Topic Page
Introduction 10
Getting Started with EcoStruxure Augmented Operator Advisor Builder 13
Overview of the EcoStruxure Augmented Operator Advisor Builder Window 15

EIO0000003006 10/2017 9
Introduction

Overview
EcoStruxure Augmented Operator Advisor uses a technology called augmented reality to optimize
the operation and maintenance of industrial sites and equipment:
 The operator points a tablet computer at the site or equipment to be monitored.
 EcoStruxure Augmented Operator Advisor uses special image comparison techniques to match
the scene visible in the field of view of the tablet camera with previously stored photographs of
the same scene.
 When a match is made, markers called points of interest are overlaid on the live scene visible
on the tablet.
 The operator taps the point of interest markers to display information. A wide variety of
information can be displayed, including:
 Process variables
 Values extracted from an SQL database
 Documents or instruction sheets
 Wiring diagrams
 Web pages
 Videos
 Audio

10 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor introduces a number of other key concepts:
 An area is a zone within each site or building—such as production lines, plant rooms, storage
areas, workshops, laboratories, and office areas—that you want to monitor using EcoStruxure
Augmented Operator Advisor.
NOTE: The number of areas you can monitor depends on the type of license you purchase
(see EcoStruxure Augmented Operator Advisor, Quick Start Guide).
 Scenes are things to monitor within an area, for example, machines, electrical cabinets, or
production lines.
 Subscenes are a part of a scene, typically a close up view of a particular area of the scene. A
subscene can contain its own points of interest.
 A project is a container for all the information in areas, scenes, and subscenes.
EcoStruxure Augmented Operator Advisor is made up of the following main components:
 EcoStruxure Augmented Operator Advisor Builder, a Web-based application.
 EcoStruxure Augmented Operator Advisor Runtime, server-based Windows software.
 EcoStruxure Augmented Operator Advisor App, an application running on a tablet computer.

EIO0000003006 10/2017 11
Prerequisites
Before using EcoStruxure Augmented Operator Advisor Builder, perform a top-down analysis:
 The sites where EcoStruxure Augmented Operator Advisor is to be deployed.
 The areas within each site—such as production lines, plant rooms, storage areas, workshops,
laboratories, and office areas—to be monitored.
 The scenes within each area, for example, production lines, machines, and electrical cabinets.
 Optionally, the sub-scenes, that is, parts of a scene that may need to be zoomed in on to
uniquely identify all the controls on the equipment.

Taking Photographs and Tagging Equipment

The first step is to determine whether to use photographs or tags to identify a scene (see page 50).
If you are using photographs, the next step is to take high-resolution photographs of all the areas,
scenes, and sub-scenes (see page 52).
Store the photographs on a hard disk drive, USB key, or network folder accessible from the device
on which EcoStruxure Augmented Operator Advisor Builder is used.
If you are using tags, the next step is to uniquely identify physically similar or identical items of
equipment and attach tags to them (see page 53).

EcoStruxure Augmented Operator Advisor Builder Software

When all prerequisites are met and the images are available, start using EcoStruxure Augmented
Operator Advisor Builder.
EcoStruxure Augmented Operator Advisor Builder is software that you use to:
 Organize and structure the areas, scenes, subscenes, and points of interest in a project
 Create and position the points of interest on photographs
 Define what happens when the operator taps the point of interest

12 EIO0000003006 10/2017
Getting Started with EcoStruxure Augmented Operator Advisor Builder

Overview
EcoStruxure Augmented Operator Advisor Builder is software that runs in a Web browser.
You can use any of the following web browsers:
® ™
 Google Chrome (version 47 or higher)
 Microsoft® Internet Explorer™ (version 11 or higher)
 Mozilla Firefox® (version 43 or higher)
 Microsoft Edge (version 13 or higher)

Logging In to EcoStruxure Augmented Operator Advisor Builder

To use EcoStruxure Augmented Operator Advisor Builder, navigate to https://app.schneider-
electric.com/ecostruxure-augmented-operator-advisor. The login window appears.

If you already have a user account, with login and password, you can log in to EcoStruxure
Augmented Operator Advisor Builder as follows:

Step Action
1 Click in the first box and select your email address from the list of stored email addresses.
Otherwise, type a valid email address.

EIO0000003006 10/2017 13
 Step Action
2 Click in the second box and type your password if the box has not been filled already.
3 Optionally, select Remember me to store the email address and password. The next time you
log in, the corresponding password is entered automatically.
4 Click Login.

Creating a User Account

If you do not have a user account, then you can create one as follows:

Step Action
1 Click Sign Up from the login window.
2 Follow the instructions in the windows to create your account. You will need to provide:
 A valid email address
 Your name
 Your country
 Some company information

3 Create your own password and confirm it, when asked to do so as a part of the sign up process.
4 Complete the sign up procedure by clicking on the link in the email sent by the registration
process.

NOTE: The green dot is for Schneider Electric internal use only. Do not click it.

Logging Out from EcoStruxure Augmented Operator Advisor Builder

You are logged out automatically after 30 minutes of inactivity. To log out from EcoStruxure
Augmented Operator Advisor Builder:

Step Action
1 Click your login name in the top right corner of the main window in EcoStruxure Augmented
Operator Advisor Builder.
2 Choose Log out.
Result: EcoStruxure Augmented Operator Advisor Builder does not ask if you want to save your
changes as all changes are saved automatically.

14 EIO0000003006 10/2017
Overview of the EcoStruxure Augmented Operator Advisor Builder Window

Overview
The main components of the EcoStruxure Augmented Operator Advisor Builder window are as
follows:

1 Toolbar. Displays icons specific to areas or scenes

2 Project toolbar. Displays icons for project-specific actions
3 Tabs to access the areas or scenes windows
4 Allows you to select the project language
5 Shows the currently logged in user name and allows access to license information and the log out
command
6 Displays online help. Gives access to online help, terms & conditions, downloading tags and product
information
7 Left sidebar
8 Main window

EIO0000003006 10/2017 15
 9 Right sidebar

16 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

Chapter 2
Projects

Projects

Projects

Overview
In EcoStruxure Augmented Operator Advisor Builder, a project is a container that allows you to
organize your photographs and define points of interest. When complete, the project must be
copied to the device on which the EcoStruxure Augmented Operator Advisor Runtime is installed.
Refer to Building a Project (see page 45) for more details.
Refer to Limitations (see page 56) for information on the maximum number of elements allowed.
Only one project can be open in the EcoStruxure Augmented Operator Advisor Builder at a time.

Creating a New Project

Step Action
1
Click the icon on the project toolbar.
2 If another project is already open, click Yes to close the project.
3 In the right sidebar, type the Project name.
4 Optionally, type a Project description.
5 The first time you use a project, there is no default language defined. Click + below Supported
Languages to add at least one of the supported languages. Then select the language for the
project in the Reference Language list.
NOTE: You cannot change the reference language of your project afterwards.
6 Click Validate.
7 Optionally, you can include additional supported languages to your project. You can then use
the Translation tab to make your project available in multiple languages.

Opening an Existing Project

Step Action
1
Click the icon on the project toolbar.

EIO0000003006 10/2017 17
 Step Action
2 The Open a Project window appears:

3 Select a project by either:

 Double-clicking a project row
 Clicking anywhere in a project row and clicking Open.

4 If another project is already open, a message is displayed. Click Yes to close it first.

Structure of a Project
When you open a project, a structured view of the project appears in the left sidebar. Each object
you create (area, scene, point of interest, and so on) is automatically added to the structure. By
default, the structure of areas in the project is displayed, for example:

18 EIO0000003006 10/2017
 1 Click to collapse the structure
2 Click to list the scenes in the project
3 Click to expand the structure
4 Click to list the variables, documents, and external applications in this area

Click the SCENES tab to display the corresponding structured view for scenes.

Deleting a Project

Step Action
1 Open the project to delete.
2 Click the Delete icon on the project toolbar.
3 Click Remove element on the confirmation window that appears.
Result: the project and all its components are permanently deleted.

EIO0000003006 10/2017 19
20 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

Chapter 3
Areas

Areas

Areas

Overview
A project is made up of one or more areas. You can create an area, for example, for each
production line or item of equipment to be monitored.

Creating a New Area

Step Action
1 Select the AREAS tab.
2
Click the Area icon on the toolbar.

3 In the right sidebar, type the Area name.

4 Optionally, you can type a short Description of the area.
5 Click Validate.

Deleting an Area

Step Action
1 In the project structure, right-click on the area to delete and choose Remove.
2 Click Remove element on the confirmation window that appears.

EIO0000003006 10/2017 21
22 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

Chapter 4
Points of Interest

Points of Interest

What Is in This Chapter?

This chapter contains the following topics:
Topic Page
Points of Interest 24
Variable Points of Interest 25
Document Points of Interest 27
External App Points of Interest 29

EIO0000003006 10/2017 23
Points of Interest

Overview
A point of interest is a position in a scene for which additional diagnostic or technical information
is available. The point of interest can be placed anywhere in the scene. The number of points of
interest that can be added to each scene is limited: refer to Limitations (see page 56). Various
types of points of interest can be defined, including technical documentation, design drawings,
information contained in a spreadsheet, and the real-time values of process variables generated
by equipment.
When the operator points the tablet camera at the scene, EcoStruxure Augmented Operator
Advisor automatically recognizes it using augmented reality and displays the points of interest on
the scene.
You can define the following types of point of interest:
 Variables (see page 25)
 Documents (see page 27)
 External applications (see page 29)

24 EIO0000003006 10/2017
Variable Points of Interest

Overview
When viewing a scene, a variable point of interest is replaced by the real-time value of a process
variable.
EcoStruxure Augmented Operator Advisor supports the following variable source types:
 Vijeo XD (or Pro-face BLUE) variables. The value is read from the PLC using Vijeo XD (or Pro-
face BLUE) software.
 Internal. The value is read from an SQL database using Node-RED software.

Variable points of interest are specific to an area. For example, you could define a variables with
the name “VAR101” in two different areas with different properties.
NOTE: EcoStruxure Augmented Operator Advisor Builder variable names must be exactly the
same as the variable names defined in Vijeo XD or Node-RED.
For Vijeo XD variables, use the complete name, including the device name part. If the Vijeo XD
variable name includes spaces, replace these with underscores (“_”) when creating the equivalent
EcoStruxure Augmented Operator Advisor Builder variable.

Creating Variable Points of Interest

Step Action
1 Select the AREAS tab.
2 Select Variables in the area structure.
Result: a list of existing variables appears:

3
Click the button
4 In the right sidebar, type the name of the variable, for example “VAR1”.
5 Select the variable type: Vijeo XD or Internal.

EIO0000003006 10/2017 25
 Step Action
6 Click Validate.
Result: a variable point of interest is created and added to the list of variables.

You can also create variables when configuring list points of interest: refer to Configuring List Items
(see page 37).

Modifying Variable Properties

To change the name or properties of an existing variable:

Step Action
1 Select the AREAS tab.
2 Select Variables in the area structure.
3 Select the variable to configure.
4 In the right sidebar, modify the variable name or change the variable type as required.
5 Click Validate.
Result: the updated variable properties are automatically saved.

Using Variables in Scenes

Once you have defined your variables, you can use them when configuring points of interest of type
Variable in scenes or subscenes (see page 39).

Deleting Variable Points of Interest

Step Action
1 Select the AREAS tab.
2 Select Variables in the project structure.
Result: a list of variables appears.
3 Click the checkbox to the right of the variable or variables to delete.
4

Click .
5 Click Remove element on the confirmation window that appears.

26 EIO0000003006 10/2017
Document Points of Interest

Overview
You associate document points of interest with specific positions in a scene. The operator viewing
the scene on the tablet can then tap the point of interest to display the associated document. The
document could be, for example, a wiring diagram for a cable loom, a video, web page, or a user
manual.

Creating Documents

Step Action
1 Select the AREAS tab.
2 In the left sidebar, select Documents in the structure view of the area in which to create the
document.
Result: a list of existing documents appears:

3
Click the button
4 In the right sidebar, type the name of the document, for example “DOC1”.
5 Optionally, select Copy the document locally. The document file is then copied to the tablet
when it is opened for the first time. Thereafter, the local copy is opened.
If the Copy the document locally option is not selected, the document is copied to the tablet each
time the operator taps the associated document point of interest.
To optimize performance, therefore, it is better to copy frequently-consulted documents locally.
NOTE: Storing too many documents locally can fill up the tablet storage.
6 Type the URL of the document, or click the browse button and select the document.
NOTE: The maximum size of document is limited. Refer to Limitations (see page 56).
7 Click Validate.
Result: a document is created and is added to the list of documents.

EIO0000003006 10/2017 27
Modifying Document Properties
To change the name or properties of an existing document point of interest:

Step Action
1 Select the AREAS tab.
2 Select Documents in the area structure.
Result: a list of existing documents appears.
3 Select the document to configure.
4 In the right sidebar, edit the document name or click the browse button to associate the point of
interest with a different document.
5 Click Validate.
Result: the updated document properties are saved.

Using Documents in Scenes

Once you have defined your documents, you can associate them with information in points of
interest type List in scenes or subscenes (see page 37).

Deleting Documents

Step Action
1 Select the AREAS tab.
2 Select Documents in the area structure.
Result: a list of existing documents appears.
3 Click the checkbox to the right of the document or documents to delete.
4

Click the button.

5 Click Remove element in the confirmation window that appears.

28 EIO0000003006 10/2017
External App Points of Interest

Overview
External App points of interest appear as markers when the operator views a scene on the tablet.
The App is launched when the operator taps the marker. The App could be, for example, Vijeo
Design’Air Plus.

Creating External Apps

Step Action
1 Select the AREAS tab.
2 Select External Applications in the area structure.
Result: a list of existing external applications appears.
3
Click the button.
4 In the right sidebar, type the name of the external application, for example “Vijeo Design’Air
Plus”.
5 Type the path of the application or a URL.
6 Click Validate.
Result: an external application point of interest is created and is added to the list of external
applications.

Using External Apps in Scenes

Once you have defined your external applications, you can use them in scenes when configuring
points of interest of type List (see page 37).

Deleting External Apps

Step Action
1 Select the AREAS tab.
2 Select External Applications in the area structure.
Result: a list of existing external applications appears.
3 Click the checkbox to the right of the external application or applications to delete.
4
Click .
5 Click Remove element in the confirmation window that appears.

EIO0000003006 10/2017 29
30 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

Chapter 5
Scenes and Subscenes

Scenes and Subscenes

What Is in This Chapter?

This chapter contains the following topics:
Topic Page
Scenes 32
Adding Subscenes to a Scene 34
Adding Lists to Scenes or Subscenes 37
Using Variables in Scenes or Subscenes 39
Using Images in Scenes or Subscenes 40
Using Triggers 41

EIO0000003006 10/2017 31
Scenes

Overview
A scene is a combination of:
 An image (high-resolution photograph) of, for example, an electrical cabinet, machine, or
production line.
 Points of interest, positioned on the image to indicate where information is being monitored. This
information can be real-time variable data or documents.
 Sub-scenes. A sub-scene is an image overlaid on the main image used to access more detailed
information about a particular area of a scene.
 Other graphics, such as arrows, lines, or text, that are overlaid on the main image to help the
operator better understand or more easily navigate the scene.
Before creating a scene, decide whether you are going to use photographs or tags to identify the
scene (see page 50).

Creating a Scene

Step Action
1 Select the SCENES tab.
2
Click the Scene icon on the toolbar.
3 In Screen Settings on the right sidebar, select Image recognition or Tag recognition.
4 Type a unique name for the scene, for example “Factory A, Second Floor”.
5 If you selected Tag recognition, select the number of the tag that is physically attached to the
equipment in the scene from the drop-down list. See Tagging Equipment (see page 53) for
details.
6 Add images to use for the scene, using any of the following methods:
 Drag and drop images on the Drop images box.
 Click Upload images then browse and select one or more images. Refer to Guideline for
Taking Photographs (see page 52).
 For Image recognition only, click Upload folder of images then browse and select a folder
containing images. All images in the folder are uploaded. You can then select which image
to use for the scene.
7 Click Validate.

Adding Points of Interest to a Scene

When you have added all images to your scene, you can begin to position points of interest on the
scene.

32 EIO0000003006 10/2017
 Step Action
1 Drag and drop one of the icons from the toolbar onto the scene image:

Subscene. Refer to Adding Subscenes to a Scene (see page 34).

List. Refer to Adding Lists to Scenes or Subscenes (see page 37).

Variable. Refer to Using Variables in Scenes or Subscenes (see page 39).

Image. Refer to Using Images in Scenes or Subscenes (see page 40).

2 Configure the point of interest, as described below.
3 Repeat steps 1 and 2 to add other points of interest to the scene

Deleting a Scene

Step Action
1 Select the SCENES tab.
2 In the project structure in the left sidebar, right-click on the scene to delete and choose Remove.
3 Choose Remove element in the confirmation window that appears.

EIO0000003006 10/2017 33
Adding Subscenes to a Scene

Overview
A subscene is a type of point of interest that appears in a scene. When the operator clicks the
subscene point of interest on the tablet, an image is displayed. This image could be, for example,
an electrical cabinet with the doors opened, or a zoomed-in view of part of the scene.
Points of interest can be added to a subscene in the same way as for a scene.

Creating a New Subscene

Step Action
1 Select the SCENES tab.
2 Select a scene in the scene structure.
3
Drag the Subscene icon from the toolbar and drop it on the scene image.
Result: a subscene marker appears in the scene:

Drag the marker to the exact position required in the scene.

4 In the right sidebar, select the subscene type:
 Image recognition. A scene that differs significantly from all the other scenes in a project, for
example, a view of the factory floor.
 Tag recognition. A scene that is identical or very similar to other scenes in the project, for
example, a row of machines. In this case, tags must be printed and attached to the cabinet
doors to assist EcoStruxure Augmented Operator Advisor to uniquely identify the scene and
display the appropriate points of interest. See Tagging Equipment (see page 53) for details.
5 Type a unique name for the subscene, for example “Zoom on pump area”.
6 If you selected Tag recognition, select the number of the tag that is physically attached to the
equipment in the subscene from the drop-down menu. See Tagging Equipment (see page 53)
for information on naming the tag.
7 Add images to use for the subscene, using any of the following methods:
 Drag and drop images on the Drop images box.
 Click Upload images, browse and select one or more images.
 For Image recognition only, click Upload folder of images; browse and select a folder
containing images. All images in the folder are uploaded. You can then select which image
to use for the scene.
8 Click Validate.

34 EIO0000003006 10/2017
Creating a Copy of an Existing Subscene

Step Action
1 Select the SCENES tab.
2 Select a scene in the structure.
3
Drag the Subscene icon from the toolbar and drop it on the scene image.
Result: a subscene marker appears in the scene:

4 In the right sidebar, choose Create from existing scene.

5 In the list of subscenes, choose the subscene to copy from.
6 Click Validate.
Result: a copy is made of the selected subscene, including the image and all points of interest.

Modifying Subscene Properties

Once created, you can adjust a subscene as follows:
 Move the subscene marker by dragging it to a different position on the scene image.
 Click on the subscene marker to hide the subscene image. The subscene marker changes to:

.
 Click the marker again to show the subscene image.
 Click on the subscene image and drag it to a different position relative to the subscene marker.
 Resize a subscene image:

a. Move the cursor over the red triangle in the bottom right corner of the image (notice that the
cursor changes to a double-headed arrow).
b. Drag the mouse diagonally until the graphic is the required size.

EIO0000003006 10/2017 35
Adding Points of Interest to a Subscene

Step Action
1 Select the subscene in the project structure in the left sidebar.
2 If required, modify the properties of the subscene in the right sidebar. For example, you could
change the subscene type from Image recognition to Tag recognition.
3 Add points of interest in the same way as for a scene. Refer to Adding points of interest to a
Scene (see page 32).

Deleting a Subscene from a Scene

Step Action
1 Right-click on the subscene in the project structure in the left sidebar and choose Remove.
2 Click Remove element in the confirmation window that appears.

36 EIO0000003006 10/2017
Adding Lists to Scenes or Subscenes

Overview
When a scene is displayed on the tablet, a list point of interest appears as a list of selectable items.
Each list item can be separately configured to, for example, display text, a document, the value of
a variable, or launch an external App.

Adding a List to a Scene or Subscene

Step Action
1 Select the SCENES tab.
2 Select a scene or subscene in the scene structure.
3
Drag the List icon from the toolbar and drop it on the image. A list marker appears:

Drag the marker to the exact position required in the scene.

4 In the right sidebar, type a name for the list, for example “More information”.
5 Optionally, select Keep open. When this option is selected, all items in the list are automatically
displayed when the scene or subscene is displayed on the tablet. When not selected, only the
list icon is displayed and the operator must tap the list icon to display the list items.
6 Click the arrow to the right of Configure Information and add information in the list, as described
in Configuring List Items (see page 37) below.
7 Click Validate.

Adding and Configuring List Items

Step Action
1 Move the mouse over the list and click the + icon:

2 In the right sidebar, type the list item label.

3 Optionally, deselect Show information to hide the details.

EIO0000003006 10/2017 37
 Step Action
4 Select the type of information to display when the operator taps the list item:
 Text
Enter the text you would like to appear when the operator taps the list item.
 Open document
Either select Create a document and identify the document to open, or select a previously
configured document point of interest (see page 27).
 Open external application.
Either select Create external app and identify the App to open, or select a previously
configured external App point of interest (see page 29).
 Read variable
Either select Create variable and configure a variable to display, or select a previously
configured variable point of interest (see page 25).
5
Optionally, you can remove individual list items by clicking the icon.
6 Click Validate.

Removing a List from a Scene

Step Action
1 Click the list marker to select the list.
2
Click at the top of the list.

38 EIO0000003006 10/2017
Using Variables in Scenes or Subscenes

Overview
A variable is a type of point of interest. When a scene is displayed on the tablet, the real-time value
of the variable appears.

Adding Variables to a Scene or Subscene

Step Action
1 Select the SCENES tab.
2 Select a scene or subscene in the structure view.
3
Drag the Variable point of interest icon from the toolbar and drop it on the image. A
variable marker appears:

If necessary, drag the marker to a more precise position on the image.

4 In the right sidebar, type a name for the variable.
5 Enter the variable label. This is the text that appears on the tablet when the operator taps the
variable marker. You must include the characters {0} in the label. These characters are replaced
by the real-time value of the variable when viewed on the tablet. The following are valid labels:
 Temperature: {0} °C
 {0} rpm

6 Select the variable that contains the value to display. You can either select Create variable and
define the variable to use, or select an existing variable resource (see page 25).
7 Click Validate.

Removing a Variable from a Scene or Subscene

To remove a variable from a scene or subscene:

Step Action
1 In the project structure, right-click on the variable and choose Remove.
2 Click Remove object in the confirmation window that appears.

EIO0000003006 10/2017 39
Using Images in Scenes or Subscenes

Overview
You can import images into a scene or subscene. These images appear in the scene when viewed
on the tablet. For example, you could add an image of an arrow to indicate the position of a
subscene in a scene, or display a hazard icon to warn the operator of danger.

Adding Images to a Scene or Subscene

Step Action
1 Select the SCENES tab.
2 Select a scene or subscene in the scene structure.
3
Drag the Image icon from the toolbar and drop it on the image. An image marker appears:

If necessary, drag the marker to a more precise position on the image.

4 On the right sidebar, type a name for the image, for example “Right arrow”
5 Optionally, select Visible. When selected, the image is initially visible when the scene or
subscene is displayed on the tablet.
6 Add images to use for the scene, using any of the following methods:
 Drag and drop images on the Drop images box.
 Click Upload images, browse and select one or more images.
 For Image recognition only, click Upload folder of images; browse and select a folder
containing images. All images in the folder are uploaded. You can then select the image that
appears in the scene or subscene.
7 Click Validate.

Deleting Images from a Scene or Subscene

Step Action
1 Right-click on the image to remove and click Remove point of interest.
2 Click Remove element in the confirmation window that appears.

40 EIO0000003006 10/2017
Using Triggers

Overview
A trigger contains one or more trigger conditions. A trigger condition compares the current value
of a variable to a specific value according to a test condition and performs a specified action if the
test condition is true. For example, a point of interest could start flashing on the tablet whenever
the value of the Speed variable is less than 100.

Creating a Trigger

Step Action
1 Select the SCENES tab.
2 Select Triggers in the scene structure, in the left sidebar.
Result: the triggers list is displayed:

3
Click .
4 In the right sidebar, enter the trigger name and, optionally, a description.
5 Select the variable to associate with the trigger. Alternatively, click Create variable, then type
the variable name and select the variable type.
6 Click Validate.
Result: the trigger is added to the list.
7
Click in the triggers list to configure the trigger conditions (see page 41) and actions
(see page 42).

Adding Trigger Conditions

A trigger condition is a test performed on the value of a variable. If the test result is “true”, the trigger
actions are performed. If the test result is “false”, nothing happens and the next trigger condition in
the list is evaluated.

EIO0000003006 10/2017 41
 To add a trigger condition:

Step Action
1
Select the trigger in the left sidebar and click , next to Conditions.
2 In the right sidebar, select an operator.
3 Choose what to compare the variable to:
 Variable. Select a second variable to compare two variable values.
 Number. Type a number to compare the variable value to a numeric value.
 Text. Type characters to compare the variable value to a string value.

4 Click Validate.

Result: the condition is added to the list of conditions. Click to create more conditions if
necessary.

Adding Trigger Actions

Step Action
1
Select the trigger in the left sidebar and click , next to Actions.
2 In the right sidebar, select what happens to the point of interest that displays the variable when
the condition evaluates to “true”:
 Change point of interest visibility. The point of interest is hidden (Hide) or displayed (Show).
 Change point of interest background color. The background color of the point of interest
changes.
Select the background color by either:
– Clicking on the color picker box. The corresponding RGBA value appears in the adjacent
text box.
– Typing the RGBA value of the color directly in the text box, for example “255,0,0,1” for red,
fully opaque, or 255,255,0,0 for yellow, fully transparent.
 Starting blinking point of interest. The point of interest starts flashing. Type the interval
between flashes, in milliseconds. For example 1000 means the point of interest flashes once
every second.
 Stop blinking point of interest. An already flashing point of interest stops flashing.

With all options, you must select the scene and the point of interest where the action will be
performed.
3 Click Validate.

Result: the action is added to the list of actions. Click to create more actions if necessary.

Removing a Trigger Condition or Action

Step Action
1 Select the SCENES tab.

42 EIO0000003006 10/2017
 Step Action
2 Select the trigger in the project structure in the left sidebar.
3
Move the mouse over the trigger condition or action to delete and click .
4 Choose Remove in the confirmation window that appears.

Removing a Trigger

Step Action
1 Select the SCENES tab.
2 In the project structure in the left sidebar:
1. Click Triggers to display the list of triggers.
2. Select the checkbox in the last column of the trigger or triggers to remove.

3. Click .

Alternatively, in the project structure, right-click on the trigger to remove and choose Remove.
3 Choose Remove in the confirmation window that appears.

EIO0000003006 10/2017 43
44 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

Chapter 6
Building a Project

Building a Project

Building Projects

Overview
When you have finished defining scenes, you can build the project. After building the project, copy
the built file to the EcoStruxure Augmented Operator Advisor Runtime component database. The
project is then available for use on the EcoStruxure Augmented Operator Advisor App.

Building a Project

Step Action
1
Click the Build icon on the project toolbar.
The Build Project window appears.
2 Click Start Build.
The Build window appears:

Building is complete when the horizontal progress bar reaches 100%.

Click Download Build to download the project file to the local PC. The project file is stored in
your browser’s download folder.
Click Close to close the Build window.
3 Refer to the Getting Started (see EcoStruxure Augmented Operator Advisor, Quick Start Guide)
for information on how to deploy the project.

EIO0000003006 10/2017 45
46 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

Appendices

EIO0000003006 10/2017 47
48 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor

EIO0000003006 10/2017

Appendix A
Additional Information

Additional Information

What Is in This Chapter?

This chapter contains the following topics:
Topic Page
How to Choose Between Image and Tag Recognition 50
Guidelines for Taking Photographs 52
Tagging Equipment 53
Limitations 56

EIO0000003006 10/2017 49
How to Choose Between Image and Tag Recognition

Overview
Depending on the nature of the equipment being imaged and its surrounding environment, the
following detection methods can be utilized for the EcoStruxure Augmented Operator Advisor App:
 Image recognition: compares the complete image used in the EcoStruxure Augmented
Operator Advisor Builder scene with the real scene.
 Tag recognition: used when image recognition is unsuccessful. The EcoStruxure Augmented
Operator Advisor App searches the real scene for a known, uniquely identifiable tag. These tags
are built into the EcoStruxure Augmented Operator Advisor Builder library.
There are a number of criteria that need to be considered when deciding if image or tag recognition
should be used.
First, you should consider the environment. Is the environment stable or a changeable one?
Stability in this case refers to the equipment in the scene as viewed from this particular viewpoint.
As we require a close match between the real scene and photo reference, a big object added to or
missing from the real scene can impact the detection.
Another consideration is the detail in the image. Does the equipment have some identifiable and
unique detail features in the scene that enable it to be both recognizable and correctly identified?
Tag recognition should be considered in cases where there are few details, or if the equipment in
the scene is similar to other scenes or objects.

CAUTION
FALSE SCENE DETECTION
 Use tags in situations where similar equipment and/or environments could result in false scene
detection.
 Attach tags firmly to the equipment with sufficient protection against damage or deterioration
of the tag.
Failure to follow these instructions can result in injury or equipment damage.

Image Recognition
In the image recognition method, the real image of the equipment is compared with the photo
reference in the scene of the EcoStruxure Augmented Operator Advisor project.
Image recognition is suitable in the following instances:
 When the equipment being imaged and its environment are stable, which means:
 The equipment being imaged does not change.
 There is no movement of other objects or people around the equipment being imaged.

 When the equipment being imaged has some unique identifiable details that can enable correct
identification.
 When the equipment or scene is significantly different from other objects or scenes in a project.

50 EIO0000003006 10/2017
 You can also take additional photos of parts of equipment to improve detection in the following
instances:
 When the equipment being imaged is moving or in a changing environment.
 When there are people or other objects moving in the camera’s field of view.

Tag Recognition
In the tag recognition method, detection is made possible by attaching tags to the equipment being
imaged. The tag is a uniquely identifiable object that is placed in the scene to enable detection.
Tag recognition method is suitable in the following instances:
 When image recognition or partial-image recognition methods do not yield satisfactory results.
 When some of the scenes or equipment being imaged may be not be easily identified. For
example, in a row of machines, tags must be printed and attached to the machines to assist
EcoStruxure Augmented Operator Advisor to uniquely identify the scene and display the
appropriate points of interest.
 When the equipment is in a poorly or very brightly lit area that prevents taking high-contrast
photographs.

Procedure to Follow
Follow this procedure to get the best results for image recognition:

Step Action
1 Take a sample photo of the equipment for the image recognition technique.
Consider taking additional sample photos if the equipment is kept outdoors or under a
transparent roof, where shadows can be seen on the equipment.
For information on how to take photographs for the image recognition method, refer to
Guidelines for Taking Photographs (see page 52).
2 Place three points of interest inside the Builder scene placed in a triangular layout.
3 Build the project and copy it to the Runtime device.
4 Go to the same position used to take the sample photo and try using EcoStruxure Augmented
Operator Advisor App to recognize the scene.
Recognition is attained when the points of interest are shown as overlaid on the scene and
remain there in the correct position and in a stable manner.
5 Take new sample photographs of the equipment with a different background, and repeat steps
1 to 4. For example, if it is practical, you could rotate the equipment by 90° about the vertical
axis. Otherwise, you could take photographs of similar equipment in a different environment.
Take a sample photo from a viewpoint with the same zoom, distance and height to confirm
recognition.
6 If the equipment is not recognized after step 5, take additional sample photos of parts of the
equipment (partial-image recognition), and perform steps 1 to 5 again.
7 If the equipment is not recognized after step 6, attach tags and repeat steps 1 to 5. For more
information on tags, refer to Tagging Equipment (see page 53).

EIO0000003006 10/2017 51
Guidelines for Taking Photographs

Overview
The EcoStruxure Augmented Operator Advisor App recognizes items of equipment using
Augmented Reality.

Photograph Guidelines
The following guidelines apply to photographs:
 Take photographs using the integrated camera of the tablet. This means that the optical aspect
of the objects should be similar when detection will take place. Detection generally works better
if the detection point is at the same point or further away than the original image. If the detection
point is closer, then parts of the detection are removed from the field-of-view and we reduce the
match criteria.
 The equipment in the image should contain a number of identifiable and unique objects.
 Refer to Limitations (see page 56) for information on the image resolution. The higher the
resolution, the better the image recognition.
 On identical items of equipment, attach tags (see page 53) before taking the photograph.
 Take multiple photographs of the scene with a with 30° horizontal angle of separation to improve
the match criteria.
 Fill the scene with the primary object. This includes taking close up views of sub-assemblies or
components of the equipment.
 Take photos at 1.6 m height. Take additional images if the user will be viewing the scene from
lower down or higher up.
 The more detail, contrast, and sharp changes in color are available on the photo, the more
successful detection will be.
NOTE: More photos will lead to better detection and tracking, especially with varying light
conditions. A minimum of 300 lux is recommended.

52 EIO0000003006 10/2017
Tagging Equipment

Overview
To avoid potential identification errors, you can attach tags to the items of equipment or place tags
somewhere in the scene. When the EcoStruxure Augmented Operator Advisor App running on the
tablet scans a scene, it recognizes the tags and displays the correct points of interest.
EcoStruxure Augmented Operator Advisor is supplied with a tag library. The tags are numbered,
and the tag number is both visible in clear text on the code and encoded into the graphic

Calculating the Size of Tag to Print

The size of the printed tag you attach to the scene or item of equipment depends on several factors:
 The distance of the operator using the tablet from the tag.
 The viewing angle.

EIO0000003006 10/2017 53
 Use the following graphic to determine the print size to use:

For example, if the operator will be approximately 2.5 meters (approximately 8.2 ft) away and
directly in front of the item of equipment (0° viewing angle), use a 15 cm by 15 cm (5.9 inches by
5.9 inches) tag.

Using a Tag
Follow these rules when using tags:
 Place the tag on a solid flat surface or on a window.
 Do not use a tag in an environment with a grill or a mesh: the detection can be very slow or
impossible, as the software will have difficulty differentiating between the tag and the grill/mesh.
 Reference photos must be taken directly in front of the tag. The photo plane must be parallel to
the tag plane for best results.

54 EIO0000003006 10/2017
  The detection camera plane must be also parallel to the tag plane. If a tag is placed at the top
of a scene, consider tilting the tag downward to respect this parallelism. Any deviation from
parallel of this photo will reduce the probability of tag detection.
 When identifying the tag placement in the images, use the four corners of the tag itself, as
indicated in green here:

Follow this procedure for using tags:

Step Action
1 Identify all the scenes and items of equipment that require tags.
2 Select the tag number to use for each scene.
3 Access the relevant tag file by selecting Download Tags, which can be accessed by clicking on

the icon in the EcoStruxure Augmented Operator Advisor Builder menu.

4 Print the file at the size calculated in Calculating the Size of Tag to Print (see page 53) above.
5 Carefully cut out the tag using scissors. Cut just outside the black border.
6 Attach the tag to the front of the item of equipment.
7 Keep a record of which tag is attached to which item of equipment.
8 Repeat steps 1...6 for all other items of equipment to be tagged.
9 Take photographs of the items of equipment. Refer to Guidelines for Taking Photographs
(see page 52).
10 Import the images into EcoStruxure Augmented Operator Advisor Builder.
11 Identify the tag placement in the images by marking the four corners of each tag.
12 Add points of interest as required.

EIO0000003006 10/2017 55
Limitations

Overview
The following describes the maximum number of elements that can be used in a EcoStruxure
Augmented Operator Advisor project.
The number of elements that can be added may also be limited by the License Requirements
(see EcoStruxure Augmented Operator Advisor, Quick Start Guide).

Projects and Areas

Element Maximum Number

Areas in a project 100(1)
Languages in a project 7
(1) A project must have at least one area and one scene, otherwise an error is generated during the project
build.

Points of Interest

Element Maximum Number

Points of interest in a scene 50
Points of interest in a project 5000
Variables in a project 1000
List items in a List point of interest 5
Characters in labels/names 50
Characters in descriptions 255

Scenes

Element Maximum Number

Scenes in a project 100(1)
Subscene levels 1
Subscenes in a scene 10
2D images in a project 100
Tags in a project 100
Characters in labels/names 50
Characters in descriptions 255
(1) 100 images in 1 scene, or 100 scenes with 1 image

56 EIO0000003006 10/2017
Resources

Element
Image resolution
Image file
Supported document format
Supported video format
Supported audio format
Description
No maximum resolution
PNG or JPG. 2 MB maximum
PDF. 20 MB maximum
MP4. 20 MB maximum
MP3. 20 MB maximum

EcoStruxure Augmented Operator Advisor App

Element Maximum Number

Notes added to a scene 10
Characters in a note 255
Favorites in an area 10

Users

Element Maximum Number

Users 65535

Variables

Element Maximum Number

Characters in variable name 128
Characters in labels/names 50
Characters in descriptions 255

EIO0000003006 10/2017 57
58 EIO0000003006 10/2017
 EcoStruxure Augmented Operator Advisor
Glossary
EIO0000003006 10/2017

Glossary

A
area
Within an industrial plant or factory, you may want to monitor different parts of the site, for example,
a production line, electrical cabinets, or machines. If you have the appropriate license type, you
can create an area for each part of the site to monitor within an EcoStruxure Augmented Operator
Advisor project. You could then choose to install different areas on different tablet computers, for
example.
augmented reality
Technology that superimposes a computer-generated image on a user view of the real world,
providing a composite view of the image with points of interest superimposed.

I
IIoT
Industrial Internet of Things. The application of IoT to the manufacturing industry. Also called the
Industrial Internet or Industry 4.0.
IoT
Internet of Things. A network of intelligent devices that collect and share data.

N
Node-RED
Node-RED is an open source, visual programming tool that requires few or no programming skills.
Instead, it uses pre-programmed, reusable blocks of code called nodes. Nodes can be wired
together to create flows of data.

P
process variable
A process variable is the measured value of a particular part of a process being monitored or
controlled, for example, temperature, pressure level, or flow rate.
project
An EcoStruxure Augmented Operator Advisor project is made up of areas and scenes. When these
have been completed, you build the project and then copy it to the EcoStruxure Augmented
Operator Advisor Runtime device. The operator can then access the project on a tablet computer
running the EcoStruxure Augmented Operator Advisor App.

EIO0000003006 10/2017 59
Glossary

S
scene
A scene corresponds to an image overlaid with points of interest.
SQL
Structured Query Language.
subscene
A part of a scene, typically a close up view of a particular area of the scene. A subscene can
contain its own points of interest.

T
tag
A two-dimensional matrix. Tags can be printed and attached to items of equipment that are
physically identical or very similar. The tags are scanned as part of a scene by the EcoStruxure
Augmented Operator Advisor App and the correct points of interest for the item of equipment
displayed.

V
Vijeo XD
Vijeo XD is human machine interface (HMI) configuration software that enables you to create and
edit application panels to control automation systems for Magelis GTU terminal (Premium and
Open boxes) and Magelis industrial PCs (Panel and Box).
variable
A variable is a type of a point of interest that is replaced by the real-time value of a software object
from a process or database, for example, a system bit, when the scene containing the variable is
displayed on the tablet.

60 EIO0000003006 10/2017