t16p1 01 Common Hmi Uxv

Common HMI for UxVs: Design Philosophy and Design Concept
The work described in this document has been undertaken by the Human Factors Integration Defence Technology Centre, part funded by the Human Capability Domain of the U.K. Ministry of Defence Scientific Research Programme. BAE Systems 2010 The authors of this report have asserted their moral rights under the Copyright, Designs and Patents act, 1988, to be identified as the authors of this work.
Reference ............................................HFIDTCPIII_T16_01 Version .................................................................................3 Date .................................................................. 15 June 2010
BAE Systems 2010. Issued by Aerosystems International Ltd on behalf of the HFI DTC consortium. The HFI DTC consortium consists of Aerosystems International Ltd, Cranfield University, Lockheed Martin, MBDA, SEA, Southampton University and the University of Birmingham
HFIDTCPIII_T16_01 Version 3/ 15 June 2010
Authors
Ian Ashdown Hannah Blackford Nick Colford Fred Elsey
Additional Contributors
Les Bennett Rob Fearnley
ii
Contents
1
1.1 1.2 1.3 1.4 1.5 1.6 1.7
Executive Summary ................................................................................... 1

Background to the report ..................................................................................................... 1 Statement of the problem .................................................................................................... 1 Report assumptions ............................................................................................................ 1 Design Philosophy............................................................................................................... 2 Design approach ................................................................................................................. 2 Conclusions ......................................................................................................................... 3 Recommendations and way forward ................................................................................... 4
2
2.1 2.2 2.3 2.4 2.5
Introduction ................................................................................................ 5
Statement of the problem .................................................................................................... 5 Background to the report ..................................................................................................... 5 Summary of the report ........................................................................................................ 6 Some terms and concepts used in the report ..................................................................... 6 Timescale assumptions ....................................................................................................... 7
3
3.1 3.2 3.3
Overview of the work undertaken ............................................................... 9

Pre-design ........................................................................................................................... 9 Design approach ............................................................................................................... 11 Requirements generation and analysis ............................................................................. 12
4
4.1 4.2 4.3
Design philosophy .................................................................................... 13

Outline. .............................................................................................................................. 13 Formulation of philosophy 1 .............................................................................................. 13 Formulation of philosophy 2 .............................................................................................. 14
5
5.1 5.2 5.3 5.4
Design Concept ........................................................................................ 19

Outline ............................................................................................................................... 19 The mission-centric HMI design process .......................................................................... 19 Abstract design concept .................................................................................................... 23 Concrete design concept .................................................................................................. 23
6
6.1 6.2 6.3 6.4 6.5
Evaluation of the design concept.............................................................. 41

Outline ............................................................................................................................... 41 How the design concept implements the philosophy ........................................................ 41 Producing requirements for a common HMI ..................................................................... 46 Comparison with the requirements set.............................................................................. 48 Comparison with another functional analysis .................................................................... 50
iii
6.6
Summary ........................................................................................................................... 52
7
7.1 7.2 7.3 7.4
Conclusions .............................................................................................. 53
Summary findings.............................................................................................................. 53 Recommendations and way forward ................................................................................. 56 Exploitation ........................................................................................................................ 58 Closing remarks ................................................................................................................ 58
References ............................................................................................... 60
iv
Acronym List
AAR ACO ASTRAEA ATC ATR BDI BLOS CDA DARS DLI Dstl DTC GCI GCS GMTI HOC HFI HMI PT ISTAR JCGUAV JTES MAS MO-ATV Active Array Radar Air Control Order Autonomous Systems Technology Related Airborne Evaluation & Assessment Advanced Technology Centre Automated Target Recognition Battle Damage Indication Beyond Line Of Sight Collateral Damage Assessment Directorate of Aviation Regulation and Safety Data Link Interface Defence Science and Technology Laboratory Defence Technology Centre Ground Control Interface Ground Control Station Ground Moving Target Indicator Head of Capability Human Factors Integration Human Machine Interface Project Team Intelligence, Surveillance, Target Acquisition and Reconnaissance Joint Capability Group Unmanned Air Vehicles Joint Training Evaluation Simulation Military Air Solutions Multi Operated All Terrain Vehicle
MOD NATO SAR SEAD STANAG UGV UAV UCS USV UUV UxV
Ministry of Defence North Atlantic Treaty Organisation Synthetic Aperture Radar Suppression of Enemy Air Defence Standardization Agreement Unmanned Ground Vehicle Unmanned Airborne Vehicle UAV Control System Unmanned Surface Vehicle (marine) Unmanned Underwater Vehicle Unmanned Vehicles of whatever (x) type
vi
1 Executive Summary
1.1 Background to the report
This document reports on part one of HFI DTC Task 16. The aim of Task 16 is to develop Human Machine Interfaces (HMIs) to support dialogue with UxVs (Unmanned Vehicles of whatever (x) type). Within this task there are two streams of work. The first examines the feasibility of a Common HMI for UxVs. The second stream examines the use of multimodal interfaces in the operation of UxVs and is reported elsewhere. The team working on part one of Task 16 engaged extensively with stakeholders. Twelve stakeholders, including military officers, procurement personnel from the MOD and technical development leads from industry, were interviewed and their expertise, top issues and concerns were recorded. Key stakeholders are listed in Table 2 and Table 3.
1.2 Statement of the problem

Unmanned Vehicles (UxVs) are dependent on inputs from human operators for much of their functioning. Currently a single platform-payload system tends to require several operators, but as the on-board and base technology becomes more sophisticated it is expected that a single operator will be able to manage one or even multiple unmanned systems. However, while there is much investment in developing the enabling technologies such as autonomy and interoperability, little is being invested in developing the HMI used by operators. Efforts have been made to standardise the control of UAV Control Systems (UCS) captured in STANAG 4586 (Standard Interface of UAV Control System for NATO UAV Interoperability). However, the HMIs of existing systems have tended to be ad-hoc designs with little concern for usability or standardisation. The potential benefits of a common HMI for UxVs are classically reported as being reduced training costs through integration of operator training, easier transfer of operators from one system to another, better handover of control between operators, reduced logistics burden, and cost savings from avoiding unnecessary duplication. During the course of this work it has been found that there are limitations to achieving these benefits. As a consequence, this report includes guidance on how best to accrue the benefits.
1.3 Report assumptions

In the report a distinction is made between commanding a vehicle and controlling it. For the purposes of this study we have attributed specific meanings to the terms. To command the UxV is to give it a goal which it will achieve without further input from the operator. To control the UxV is to engage in a feedback loop with the UxV whereby the operator sends instructions to the UxV on a more or less continuous basis; the
operator monitors the progress of the UxV towards achieving a goal known to the operator.
1.3.1 Timescale assumptions

It is not the intention that this work be used to influence an HMI design to be retro-fitted to existing UxV systems. It is also not likely to be possible or desirable for this work to influence UxV systems currently under development. It is the intention that this work be exploited and considered in informing the design of future UxV systems.
1.4 Design Philosophy

After talking to stakeholders and reviewing the literature the first part of the design philosophy (Philosophy 1): Mission Centricity emerged. Philosophy 2 was proposed and confirmed in further discussion with the primary stakeholder.
Philosophy 1 Mission Centricity

The HMI should be designed for a specific mission Catchphrase: Payloads with platforms, not platforms with payloads
Philosophy 2 Command versus Control

Command: Common HMI for common tasks being performed Control: Specific HMI for the capabilities being controlled Section 4 expands further on this two part design philosophy. The philosophy looks at the extent to which the design should attempt to be common. The design concept then presents a proposal of how that could look.
1.5 Design approach

The review of literature and earlier research led to a previous piece of BAE Systems work that designed a concept HMI for ISTAR (Imagery Surveillance Target Acquisition and Reconnaissance) UxVs1. This work was identified and selected for further scrutiny because, in agreement with the findings of the early part of this study, it is based upon the
Common Operator Control Interface for Multi-UxV Operations. TES103687 / December 2008
philosophy of mission centricity in the design concept, and provides a very detailed design across several domains air, underwater and ground. The Task 16 team took the approach to revisit and report this mission-centric design process and the resulting concept, which is reported in section 5. An analytical validation was conducted and the concept extended.
1.6 Conclusions
1.6.1 Potential benefits from a common HMI
The potential benefits from a common HMI are listed below, though it is also reported that certain organisation constraints operate and influence which of these benefits can practically be achieved: 1 Performance benefit to the individual operator Individuals who use more than one type of system find it easier to switch between systems if they are similar. The fewer differences there are between the systems the less effort it will require of the operator to remember which system they are using and how it works before going ahead and using each aspect of the HMI. This reduction in mental workload for the operator manifests itself as requiring less time to perform the task, reduced fatigue as a result of performing the task and reduced likelihood of error in the performance of the task. Organisations inherit these benefits from the individuals working within them. Training benefit to the individual operator Individuals will learn to use a new system faster if it has similarities with a system they are already familiar with using. Organisations inherit these benefits from the individuals in whom they invest training. Training development benefit to the organisation Even if no individual uses more than one of the systems that has a common HMI the organisation can obtain a benefit. The organisation can make savings through common training modules being developed for the aspects of the different systems that use the common HMI. Training delivery benefit to the organisation The common HMI training modules can be delivered to groups consisting of future users of any of the different systems that use the common HMI aspects. This is valid even if no individual then goes on to use more than one of the systems that use the common HMI. Logistical complexity benefit to the organisation If the common HMI aspects can be designed as complete logistical units then this can reduce the number of different unit types that need to be supported (hardware, software and even personnel). Logistical volume benefit to the organisation The number of spare units required to maintain availability could be reduced. Cross assurance benefit to the supplier and the procuring organisation Demonstrating assurance of an HMI with human operators is costly. If an HMI both looks and behaves the same way then perhaps that generic HMI could be assured as safe and usable. Then any vehicle type that was fully compliant with the appearance and behaviour of the HMI would be certified as usable and would not have to repeat the full set of certification activities. This might only be partially realisable as there would almost inevitably be differences in behaviour under failure conditions and reversionary modes.
6 7
Cost benefit to the supplier Reuse of an already developed component brings cost savings to suppliers. Not only are development costs reduced, acceptance and certification costs could be reduced as well.
1.7 Recommendations and way forward

The extent to which a common HMI could be designed for a range of platform domains and (to a lesser extent) payload types has been investigated in this study. The design philosophy sets out a feasible architecture for common HMI design that is in keeping with technological trends and the scientific literature. The design concept provides one example of how that could be implemented. Some generalisations have been drawn from the design concept and it has been compared favourably with analytical works in the same field. The study has concluded that it is worth investigating further the idea of a common HMI for the commanding of UxVs through a mission centric design approach. It has also concluded that hands-on control of UxVs (flying, driving and sailing them) is best left for specific HMIs.
2 Introduction
2.1 Statement of the problem
Despite the name, Unmanned Vehicles (UxVs) are dependent on inputs from human operators for much of their functioning. Currently a single platform-payload system tends to require several operators, but as the on-board and base technology becomes more sophisticated it is expected that a single operator will be able to manage one or even multiple unmanned systems. However, while there is much investment in developing the enabling technologies such as autonomy and interoperability, little is being invested in developing the human-machine interface (HMI) used by operators. Efforts have been made to standardise the control of UAV Control Systems (UCS) captured in STANAG 4586 (Standard Interface of UAV Control System for NATO UAV Interoperability). However, these have focussed on architectures, interfaces, communications protocols, data elements and message formats and general requirements for the HMI (Human Machine Interface) rather than the development of the HMI itself. The HMIs of existing systems have tended to be ad-hoc designs with little concern for usability or standardisation. The potential benefits of a common HMI for UxVs are classically reported as being reduced training costs through integration of operator training, easier transfer of operators from one system to another, better handover of control between operators, reduced logistics burden, and cost savings from avoiding unnecessary duplication. During the course of this work it has been found that there are limitations to achieving these benefits. As a consequence, this report includes guidance on how best to accrue the benefits.
2.2 Background to the report

This document reports on part one of HFI DTC Task 16. The aim of Task 16 is to develop HMIs to support dialogue with UxVs (Unmanned Vehicles of whatever (x) type). Within this task there are two streams of work. The first examines the feasibility of a Common HMI for UxVs. The second stream examines the use of multimodal interfaces in the operation of UxVs and is reported elsewhere. Part one of Task 16 had the objective of producing a design philosophy and design concept by consulting with stakeholders, reviewing previous work and analysis of requirements. The team working on part one of Task 16 engaged extensively with stakeholders individually. An unsuccessful attempt was made to hold a workshop to discuss the design concept reported here early in the task in order to have some validation or buy-in before the preparation of the report. A forthcoming dissemination workshop for the task will now provide that opportunity and will be presenting the concepts and results of analyses for discussion and comment.
2.3 Summary of the report

Section 1 explains the background to the report and provides a summary of its content. It also contains a description of how some terms, widely encountered in the UxV literature, are used in the report. Section 2 describes how the project team reviewed literature, including relevant standards and engaged with stakeholders to produce a UxV common HMI design philosophy and a design concept. Sections 3 and 4 describe the design philosophy and the design concept respectively. Section 4 reports the design concept and the process that produced it. It begins with a mission-centric design process that led to the design concept. The design concept is described firstly in the abstract terms of its organisation around operator roles and functional goals. Following that, the concrete appearance of the design concept is presented and explained. Section 5 reports the evaluations that the team has performed on the design concept. The design concept has been evaluated as to the amount of commonality that it exhibits within the HMI and from this generalisations are drawn. The HMI design concept has been compared with several requirement sets that have been developed during Task 16 from reviews of literature and discussions with stakeholders. Section 6 concludes the body of the report with a summary of the findings, a discussion of the benefits, risks and opportunities that a common HMI for UxVs would present, and proposes ways to take the subject further. Section 7 is the reference list and section 8 is composed of appendices. Several large tables are contained in the appendices at the end of the document. They were generated during the analysis of requirements and comparison of the HMI design concept with those requirements. They provide extensive traceability of the work performed.
2.4 Some terms and concepts used in the report

The top level of organisation for vehicles appears to be the medium in which they move. This is widely called the domain in the literature. Payloads are closely associated with the type of mission that they serve. In order to manage these concepts we have used a 2dimensional problem space, described in Table 1. This has proved to be useful in organising discussion and has found favour with the major stakeholder. It is a high level concept and will be presented for comment, discussion, possible extension and modification in the dissemination workshop.
Table 1: The UxV HMI problem space in terms of vehicle domains and mission types Mission Types (Payload Domains) ISTAR Air Domains Vehicle Ground Underwater Surface Logistic Strike Others
In the report we make the distinction between commanding a vehicle (or a payload) and controlling it. The terms command and control are often used to mean different things and are sometimes used in a way that implies that one cannot exist without the other. For the purposes of this study we have attributed specific meanings to the terms. To command the UxV is to give it a goal which it will achieve without further input from the operator. To control the UxV is to engage in a feedback loop with the UxV whereby the operator sends instructions to the UxV on a more or less continuous basis; the operator monitors the progress of the UxV towards achieving a goal known to the operator.
This distinction has been particularly useful in describing the sort of tasks for which commonality in the HMI is useful.
2.5 Timescale assumptions

It is not the intention that this work be used to influence an HMI design to be retro-fitted to existing UxV systems. It is also not likely to be possible or desirable for this work to influence UxV systems currently under development. It is the intention that this work be exploited and considered in informing the design of future UxV systems. How far in the future these future systems may be is difficult to specify. What is important is the context of UxV operations at the time of HMI design and implementation. It is reasonable to assume that the current trend for increased autonomy for vehicle control systems will continue, and the design philosophy should reflect this.
UxVs are no longer necessarily single mission vehicles and it is assumed that this trend towards heterogeneous, multi-mission UxVs will continue. The design philosophy and concept proposed in this report will not be affected by this.
3 Overview of the work undertaken

3.1 Pre-design
3.1.1 Stakeholder engagement
Early stakeholder identification and engagement has been central to this work. Twelve stakeholders, including military officers, procurement personnel from the MOD and technical development leads from industry, were interviewed and their expertise, top issues and concerns were recorded. In addition to these stakeholders a Hermes 450 UAV ground control station crew were interviewed immediately following their return from operations in Afghanistan. The primary stakeholder for this Task is Commander Max Snow, Royal Navy, Head of Regulation Branch, Directorate of Aviation Regulation & Safety (DARS). Cdr Snow has been involved at key stages during the work and has shaped the design philosophy. The DARS representative is active in the Flight in Non Regulated Air Space Group (FINAS). This group reports to the NATO Joint Capability Group Unmanned Air Vehicles (JCGUAV). This Forum manages the STANAG 4586 custodian team. There is interest in starting a separate HMI Working group to investigate issues of commonality within HMIs for future NATO UAVs. Table 2: Stakeholders interviewed
MS MWh RH CC CCB TW IR PW MW JG TH AW
Directorate of Aviation Regulation and Safety (DARS) UAS APT, SO2 WIT/UAV, HQ DRA Fleet Submaritime Technical advisor to DEC AWE for off-board unmanned systems. SME to the SEAS DTC for UAVs, Dstl Capability Advisor to HFI DTC; Advisor in AWE Head of Advanced Information Processing, BAE Systems ATC Senior Human Factors Engineer, BAE Systems Insyte Technologist Advisor HFI, BAE Systems MAS Above Water Research Programmes manager; ISTAR advice team for unmanned systems, Dstl. Watchkeeper Mission System Manager, DE&S UAS PT Project Manager - Autonomy, BAE Systems ATC Principal Scientist, BAE Systems ATC
Table 3: Stakeholders interviewed, mapped against area of expertise Mission Types (Payload Domains) Logistic Strike
MWh, MS, PW, IR TH, AW, TW MWh, MS, PW, IR TW TH, AW RH MW
ISTAR Air Domains Vehicle Ground Underwater Surface

CC, MWh, MS, JG, PW, IR TW RH MW, CC
Others
All interviews were analysed for themes, comments given identification tags and converted into requirements where appropriate. To ensure full traceability of stakeholder comments, these requirements were later mapped to the HMI design concept. Over the course of two meetings with the primary stakeholder, the topic of how to accrue the potential benefits from a common HMI was discussed. This is reported in Section 6.
3.1.2 Literature review

A review of the relevant literature and standards was conducted (issued as an annex to this report). This review indicated that commonality of HMIs could be possible for UxV operation. This was found to be the case because there is similarity of goals, tasks and information objects across UxV operator tasks and operational procedures. This functional similarity along with stakeholder consultation confirmed the intention of the task; that commonality in UxV HMIs could indeed be useful and desirable.
3.1.3 Formulation of design philosophy

The philosophy emerged in two stages. After talking to stakeholders and reviewing the literature the first part of the design philosophy (Philosophy 1): Mission Centricity emerged. Philosophy 2 was proposed and confirmed in further discussion with the primary stakeholder.

The HMI should be designed for a specific mission. Catchphrase: Payloads with platforms, not platforms with payloads
10

Command: Common HMI for common tasks being performed Control: Specific HMI for the capabilities being controlled Section 4 expands further on this two part design philosophy. The philosophy looks at the extent to which the design should attempt to be common. The design concept presents a proposal of how that could look. How it should look is a question of detailed design and good human factors. Following good HFI process, this should be based on a full and comprehensive set of requirements of the actual systems to be covered by this HMI.
3.2 Design approach

The review of literature and earlier research led to a previous piece of BAE Systems work that designed a concept HMI for ISTAR (Imagery Surveillance Target Acquisition and Reconnaissance) UxVs2. This work was identified and selected for further scrutiny because, in agreement with the findings of the early part of this study, it is based upon the philosophy of mission centricity in the design concept, and provides a very detailed design across several domains air, underwater and ground. Examples of common HMIs exist within a single domain but there is very little regarding cross domain HMIs. The design is not related to any commercial product and therefore is able to be discussed openly by UxV industry stakeholders without concern for protection of market position. The design concept for the ISTAR HMI was found to be very thorough and it would have offered no additional value to the customer to recreate or repeat the work. The mission centric design process places great emphasis on the need for validation of the data used and this validation had not yet been carried out on this ISTAR HMI. It represents a much lower risk to validate the concept across one type of mission (ISTAR) before extending it across further missions. It is good practice that a common HMI needs to come from standardisation across proven and successful HMIs rather than purely theoretical studies. The mission centric philosophy can be followed but the detail must be validated and effectiveness demonstrated. The Task 16 team took the approach to revisit and report this mission-centric design process and the resulting concept, which is reported in section 5. An analytical validation was conducted and the concept extended. A further stakeholder validation workshop is planned for later in 2010.
Common Operator Control Interface for Multi-UxV Operations. TES103687 / December 2008
11
3.3 Requirements generation and analysis

A set of requirements, organised around UxV functional roles, was compiled, comprising requirements from STANAG 45863, generic human factors requirements from Def Stan 00-2504 and further requirements generated from human factors experience on other UAV programmes. It is the intention that this requirement set can be taken by the primary stakeholder, DARS, to the NATO JCGUAV for further development and exploitation. The team has cross referenced the design concept with the requirements. Subsequently, the concept has been cross referenced with research by Nehme et al (2007) who report an operator functional taxonomy for unmanned aerial vehicle missions. The results of these exercises provided confidence in the validity of the design philosophy and concept.
3
4
Standard Interfaces of UAV Control System (UCS) for NATO UAV Interoperability Defence Standard 00-250 Human Factors for Designers of Systems
12
4 Design philosophy
4.1 Outline.
The design philosophy provides high-level guidance in two parts for designing a common HMI for UxVs. The first part of the philosophy addresses the question of the overall architecture of a common HMI in order to be in line with operational intent and to maximise leverage of technological capability. The second part of the philosophy addresses the question of how much to make common within the HMI, given the nature of the relationship between the human operator and the technology.
4.2 Formulation of philosophy 1

4.2.1 Statement of philosophy 1

The HMI should be designed for a specific mission type. Catchphrase: Payloads with platforms, not platforms with payloads
4.2.2 Justification for mission-centricity

Mission-centric design has been chosen as our over-arching design philosophy for three reasons: 1. It coincides with the technological trend: platforms, which are related to domains, are becoming increasingly autonomous and they act increasingly in a support role to operations for which their payloads are being controlled or commanded. 2. It coincides with the primary stakeholders approach to the problem and his assessment of the way that technological and operational trends will develop. 3. It coincides with the way in which users of existing systems choose to operate their systems when given the opportunity. These reasons became apparent through the review of the literature and interviews with stakeholders. Operational UAV end users indicated a preference for working mission centrically and indeed they are currently doing so by commanding camera coverage of a location in order to direct the vehicle. Through discussion of this philosophy with the primary stakeholder,
13
it was further refined and the phrase Payloads with platforms, not platforms with payloads was used to encapsulate it. The philosophy of mission-centricity in the design of a common HMI is supported by the increasing autonomy of the platform part of the unmanned vehicle. The default way to navigate a vehicle to a location is to command it to go somewhere rather than for it to be constantly controlled by a human operator. Such vehicles can automatically place themselves where they need to be to deliver the payload. The philosophy to date of individual platforms or vehicles forming the centre of the design, with the payloads they are carrying being secondary, is no longer appropriate for this way of working. The philosophy is to have a HMI designed and arranged around a specific mission type and the payloads that deliver the effects required of the mission. Mission-centric HMI design can also be called effects-based, or goal-oriented, HMI design. It calls for the HMI to be designed with the outcome or goal of the mission in mind rather than the means by which the mission will be achieved. The means by which the same effect is achieved can be made common. This would mean, for example, a common HMI to make different vehicles go to the same place and a common HMI to make the same sort of payload cause the same effect. It would not, however necessarily lead to having a common HMI for making different payloads achieve different effects. In this sense, mission centric design addresses one of the risks of a common HMI: that the operator can forget which system they are using. Thinking they are using system A (for example), they could activate system B. This would not be such a big problem if payload A is an optical camera and payload B is an infra red camera. However if payload B is a weapon the consequence of the error may be more serious. Known as the right action, wrong object error, it is a well-known effect. One the most effective measures for reducing the likelihood of this error (Kirwan, 1994) is to ensure that the HMI for system A and system B, in this example, are clearly different. Mission-centric design encourages similarity in the HMI for payloads with similar effects. Mission-centric design encourages similarity in the HMI for platforms as their effect is similar get the payload to the right place. However mission-centric design encourages a different HMI for payloads with different effects. So in summary, philosophy 1 mission-centric HMI design is in keeping with the way technology is developing, fits with operational thinking and encourages the design of different HMIs for systems with different effects, based on the payloads involved.
4.3 Formulation of philosophy 2

4.3.1 Statement of philosophy 2
The second part of the Design Philosophy is concerned with the extent to which an HMI should aim to be common. This report concludes that there is benefit in commonality in the functions involved in commanding unmanned vehicles in different domains and classes, but that the finer details of controlling the UxVs should remain vehicle-specific.
14
Section 4 develops a candidate design concept, showing which aspects of the UxV functions would add benefit through being common.

Command: Common HMI for common tasks being performed Control: Specific HMI for the capabilities being controlled
4.3.2 Commonality of commands; specificity of controls

Where there is commonality in commands it makes sense that there should be benefits of commonality in the HMI. What makes little sense and does not represent a philosophy of good human factors would be to have a common HMI for incompatible commands. Commonality is easier to achieve when the operator works at an abstract level, insulated from the specific aspects of implementation. These insulating levels of abstraction are easily lost in failure or reversionary modes, making commonality difficult. Figure 1 graphically illustrates the association between command and high level tasks. Command tasks tend to be close to overall goals and abstracted from the mechanisms necessary to implement them. In contrast, low level tasks tend to require a control feedback loop of continuous monitoring. Low level control tasks accommodate the specific details of the mechanisms in use and changes in the environment to achieve the mission goals.
Command.
Similar Goals Abstract
High level tasks.
Control
Means
Concrete
Differing
Low level tasks.
Figure 1: The relationship between abstract command and concrete control tasks The difference between commanding a vehicle and controlling a vehicle is a fundamental difference that contributes to the opportunity to have a common HMI. When a vehicle is being commanded to go to a location it does not matter (to the operator) how the vehicle gets there. The HMI can be the same as long as it allows the operator to tell the vehicle where to go with enough accuracy and precision and as long as it reduces the risk of
15
significant errors. When the operator is controlling the vehicle during its journey then the details of how the vehicle steers are important. The HMI has to be fine-tuned with the capabilities of the operator and the characteristics of the vehicle matched to avoid, for example, overcompensation on the part of the operator. Put simply, in order for a person to control dissimilar vehicles reliably through a similar HMI the vehicles need to respond in the same way to control inputs. In fact people can control the vehicles even if they behave differently, but this will reduce the reliability of the process and means that the operators will need to be trained separately on the different vehicles, thus reducing the benefit from having the similar HMI. So the objective should be to have the different vehicles respond similarly as part of the similarity of the HMI.
4.3.3 Consistency with STANAG 4586

Philosophy 2 supports the approach in NATO Standardisation Agreement (STANAG) 4586. STANAG 4586 is intended to allow NATO nations to achieve UAV interoperability and is the most relevant standard for exploring and proposing HMI commonality across unmanned air vehicles. STANAG 4586 is also of significant interest and importance to the primary stakeholder for this HFI DTC Task. Whilst it is not appropriate to assume the STANAG provides the complete solution or is wholly transferable across to UxVs, consideration of the Standard and comparing it to the design concept proposed in this report increases the potential value in exploitation. The messaging protocols in the STANAG are appropriate for commanding a vehicle whilst stating that vehicle specific messages (VSMs) are required for the more bespoke requirements of controlling unmanned (air) vehicles. The STANAG 4586 concept calls for a certain level of generic control (although this equates to what we are referring to here as command), with the detailed specific vehicle control being provided by the VSM. In practical terms what this means is that the STANAG discusses and provides some outline requirements for commonality in the HMI for the generic elements of command. However, it does not elaborate to this level of detail for the VSM as the STANAG philosophy considers these control elements as vehicle specific and therefore unsuitable for a common HMI.
4.3.4 User feedback

Philosophy 2, to have commonality for command rather than control, reflects the user feedback obtained from a currently operational ground control crew. The crew expressed their frustration with the presence of menu items in the HMI for functions not implemented on their aircraft. In this example the redundant functions are present on the menus without even having been greyed out (though that may still not be an acceptable human factors solution). The users reported that the system gave no feedback to say that a command had not been successful. Not only was this frustrating for the users but it is likely to lead to errors. This user anecdote is an example of commonality for the benefit (convenience) of the manufacturer as opposed to the benefit of the user. Of course, if implemented properly,
16
this type of commonality could bring some benefit to those who use more than one system (see Section 7.1.3). This feedback not only helped to shape philosophy 2 in terms of the appropriate extent of commonality, it also sounds a warning that common does not have to mean identical.
4.3.5 Illustrations of the issue

There are essentially two ways to make the vehicles respond to control inputs in the same way, although most practical cases employ a combination of the two. In the first case the vehicles are designed to be similar to each other in all but the unavoidable differences. The similarities in physical design between Airbus airliners contribute in this way to the ease with which pilots can convert from one type to another. The other way is to introduce a layer of processing between the operator and the vehicle. The operator effectively controls a virtual vehicle with a given set of performance characteristics to travel along a virtual trajectory. The intermediate processing layer then calculates the control actions necessary to make the actual vehicle (with its own actual characteristics) achieve that route. An extreme example of the second approach is the Shuttle Training Aircraft (STA). It is not an example of an unmanned vehicle but it illustrates two important points about creating a common HMI that can control (as opposed to command) the movement of two different types of vehicle. Neither was this common HMI created to achieve any of the benefits that might accrue from a common HMI for unmanned vehicles. The STA is a Gulfstream II business jet that has been modified by NASA so that it can be flown exactly like the Space Shuttle from an altitude of 30,000 feet to landing. The HMI for the STA is the same as that of a Space Shuttle in the left hand pilots seat and that of a normal Gulfstream in the right-hand pilots seat (see Figure 2). When the STA is being flown from the left hand seat it responds to pilot controls just like the Space Shuttle. In order for the 29,000 kg, two-engine Gulfstream to fly like a gliding 100,000 kg Space Shuttle it needs to fly with its landing gear down and thrust reversers deployed, in flight. In addition, the control is achieved through a sophisticated dedicated avionics system. Effectively the left-hand pilot is kept isolated from most of the good handling characteristics of the Gulfstream and a layer of sensors and flight simulation functionality enables the Gulfstream to fly as badly as the Space Shuttle.
17
Common HMI for controlling two different air vehicles
Figure 2: A space shuttle HMI (outlined) integrated into the cockpit of a Gulfstream II business jet This example is cited to illustrate two points: Firstly a common HMI for control tasks requires a sophisticated and dedicated layer of control functionality to translate the operators control of the virtual vehicle into remote control of the actual vehicle. Secondly, control through a completely common HMI will be limited to the lowest common denominator of the capabilities of the range of vehicles covered. The second of these points may well be fine if, as is the current trend, control of vehicles (as opposed to command) will only be needed as a reversionary mode for when something goes wrong. The first point is more serious, and it is more serious in exactly the situations of loss of functionality that would call for manual control of the vehicle. To manually control an unmanned vehicle requires more and better communication links with the vehicle than to command it. For future unmanned vehicles it will probably be easier and more reliable to have automated reversionary modes than manual reversionary modes. The manual control of the vehicle will probably rely on the very functionality that has been lost in the first place. These two points have two practical implications: Firstly one can only fly the Gulfstream like a Space Shuttle if the Gulfstreams on-board Space Shuttle simulator is working. Secondly, if one wants to fly the Gulfstream as a Gulfstream it is better to use the Gulfstream controls.
18
5 Design Concept
5.1 Outline
This section describes the design concept, the process that produced the design concept and examines the validity and applicability of the design concept. It begins with the mission centric design process that led to the design concept. The design concept is described firstly in abstract terms of its organisation around operator roles and functional goals. Following this, the concrete appearance of the design concept is presented and explained. In keeping with the first part of the design philosophy the design concept is missioncentric. Within the scope of a single mission it covers three out of the four domains of unmanned vehicles, as illustrated in Table 4. Table 4: The region of the UxV HMI problem space occupied by the design concept Mission Types (Payload Domains) ISTAR Air Domains Vehicle Ground Underwater Surface
Mission-centric design concept covers these vehicle domains.
Logistic
Strike
Others
5.2 The mission-centric HMI design process

The HMI design concept was developed from information requirements that were derived from an analysis of multi-domain future ISTAR mission scenarios (NATO July 2007). Generic ISTAR functions were identified that can also be related to the four main functions of the intelligence cycle (direction-collection-processing-dissemination):
1. 2. 3. 4. 5. 6. 7. Information requirements are defined The requirements for ISTAR data collection are identified ISTAR assets are assigned to collect required data The ISTAR assets mission is planned The ISTAR assets mission is executed The data collected from the assets is processed The analysts intelligence products are delivered to the commander (direction) (direction) (direction) (direction) (collection) (processing) (dissemination)
19
The functions of relevance to operation of specific unmanned vehicles are those numbered 4, 5 and 6 in the list shown above, because they relate to specific assets. A hierarchical functional analysis broke down mission-specific goals into functions, tasks and actions that were to be achieved by human operators and/or technological systems components. The operator roles are taken from the established ISTAR unmanned vehicle system roles of Mission Manager, Vehicle Operator, Payload Operator and Data Analyst. Detailed UML (Unified Modelling Language) information and interaction diagrams were produced, tracing the information needs and feasible interactions between the assumed technological capability and operator roles. The process was systematic and progressive in its analysis. At each stage of the analysis assumptions were made about the degree of capability in the technology and the resulting implications for allocation of sub-functions to humans or technology. A summary of the major functions analysed is presented in Table 5 although the analysis went several levels lower in order to arrive at the detailed HMI design. Table 5: Assumptions of operational mode for UxV HMI functions.
Ref. no 1 1.1 1.2 1.3 1.4 2 2.1 2.1.1 2.1.2 2.2 2.2.1 2.2.2 2.2.3 3 3.1 3.1.1 3.1.2 3.1.3 3.2 3.2.1 3.2.2 4 4.1 4.2 function Mission (Planning) Asset Tasking Scheduling Sensor Coverage Route Planning Vehicle Operation Launch / Recover Navigation Systems Configuration Management Checks Payload Systems Checks Management Configuration Data Collection Download Exploitation Data Processing Target Detection Mode of Operation Interactive Autonomous Autonomous Autonomous
Interactive Autonomous Autonomous Autonomous Autonomous
Autonomous Autonomous Autonomous Autonomous Interactive Interactive Interactive
The need to account for the capability of the technology in the mission-centric design process is critical to the reliability of the process. It is the practical reality of the statement the devil is in the detail when it comes to designing the HMI for unmanned vehicles. It is the reason why a common HMI would need to be based around proven systems and their validated design assumptions.
20
Validation of the assumptions requires the design, building and testing of real systems. This design concept shows how the mission centric design process can arrive at a design for a common HMI for the operation of different types of vehicles. The design process makes explicit the need for testing of its validity and the need to have specific information about the capabilities of the automatic and, increasingly, autonomous technology that is operated. This aspect of the design process makes this design concept useful as a straw man for discussion and through which the idea of a common HMI can be examined. The detailed analysis of information needs and interaction styles informed the content for the HMI screens that implement the major functions listed above in Table 5. The information requirements are reported in Table 6 below. Table 6: Information and control requirements for UxV HMI functions
Mission function 1.1 - Specific ISTAR assets are tasked with collecting specific types of data [interactive] Information requirements Data collection requirements Available assets and their data collection capabilities Comparison of available assets and their data collection capabilities with data collection requirements Operator-selected asset assignments Auto planning status Auto-generated plan Geospatial/threat intelligence Operator-generated plan Geospatial/threat intelligence Auto planning status Auto-generated data collection plan Geospatial/threat intelligence Operator-generated plan Geospatial/threat intelligence Auto planning status Auto-generated route plan Geospatial/threat intelligence Operator-generated plan Geospatial/threat intelligence Platforms launch readiness status Platforms launch confirm Launch Go / No-Go decision input Manual planning override Manual planning inputs Manual planning override Manual planning inputs Manual planning override Manual planning inputs Control requirements Selection of information Assignment of specific assets to specific requirements
1.3 - Assets sensor coverage is planned [autonomous] 1.2 - Assets data collection scheduling is planned [autonomous] 1.4- Assets routes are planned [autonomous] 2.1.1 - Platforms are launched [interactive]
21
2.1.2 - Platforms are navigated [autonomous]
Auto navigation status (platform locations relative to route/time plans) Geospatial/threat intelligence Events detected requiring replanning Auto route re-planning status Auto revised route plans Operator-generated plan Geospatial/threat intelligence Platform trajectory/dynamics Engine parameters Systems health status Detected system faults Auto fault diagnosis Auto fault repair process Manual override feedback Auto data collection status Geospatial/threat intelligence Manual data collection status Geospatial/threat intelligence Data download status Data download metadata Downloaded data Information requirements Data/image under analysis
Manual navigation override Manual platform navigation
2.2 & 3.1 - Health and safety of systems is maintained as far as possible [autonomous] 3.2.1 - Data is collected by sensors [autonomous] 3.2.2 - Collected data is downloaded from platforms [interactive] 4.1 - Downloaded sensor data is processed [interactive]
Manual diagnosis override Manual repair override Mission abort override Self-destruct override
Manual re-planning of data collection Manual control of data collection Data download management
Data/image processing Data/image management
The design concept that follows is the result of the process described above. The individual screens were designed according to a consistent look and feel in which similar information is consistently located, as shown in the description of the screen that follows in this section. The mission centric design and analysis of information processing across the HMI highlighted the common aspects of the operators tasks. When used together with explicit information about autonomy and interaction across the HMI this led to the design of common HMI screens for many of the functions. The design method emphasises the importance of valid information about the autonomy and capability of the systems being used.
22
5.3 Abstract design concept

The abstract aspect of the design concept is the organisation of the screens making up the HMI. The screens are organised around a functional hierarchy based on analysis of mission functions.
Figure 3: Functional hierarchy of the concept design HMI for operation of UxVs
5.4 Concrete design concept

The HMI Design Concept implements the interaction diagrams reported in the design process. The appearance of the HMI is arranged to enable navigation through the functionality and shares the screen area between contextual information, actionable information and control options through which to action on the information presented. The screen designs make up the concrete aspect of the design concept.
5.4.1 Guide to the screen layout

The design was developed for visualisation and demonstration purposes in PowerPoint. It is a mock-up intended to demonstrate the feasibility of the concept, rather than represent a final design solution. It consists of multiple slides simulating the HMI in use in different tasks. A consistent look-and-feel was implemented throughout. This includes a black background, and green highlights to show current menu selections.
23
Asset selection
Menu navigation
Options Context-relevant Information Main Display
Alternative Displays Troubleshooting info
Timeline
Figure 4: Layout of the design concept HMI screen The basic layout is illustrated in Figure 4. Specific examples from the main functions are presented in the remainder of this section The buttons in the top left corner allow selection of any role and the buttons in the top right select the major functions for that role. Below them are the controls for selecting assets selectable by payload or platform. The centre of the screen is the main display, used to display the principal information. To its left are secondary information and context selection controls such as which task or asset is being operated. On the right of the main display, detailed input boxes, buttons and windows are displayed. On appropriate screens a timeline is displayed at the bottom to provide context within the mission schedule.
5.4.2 Example screens

The following pages show excerpts from the design concept HMI alongside contextual information of where each screen lies in the functional hierarchy. Discussion of the content of the screens is to be found in section 6.2, Table 8.
24
Figure 5: Asset Tasking Screen
25
Figure 6: Scheduling Screen
26
Figure 7: Sensor Coverage
27
Figure 8: Route Planning Screen
28
Figure 9: Launch / Recover - No screen implemented
29
Figure 10: Navigation Screen
30
Figure 11: Configuration Screen
31
Figure 12: Vehicle Management Screen
32
Figure 13: Vehicle Checks Screen
33
Figure 14: Payload Checks Screen
34
Figure 15: Payload Management Screen
35
Figure 16: Payload Configuration Screen
36
Figure 17: Payload Collection Screen
37
Figure 18: Payload Download - No screen implemented
38
Figure 19: Data Processing Screen
39
Figure 20: Target Detection Screen
40
6 Evaluation of the design concept

6.1 Outline
Section 5 reports the evaluations that the team has performed on the design concept. The design concept has been evaluated to assess the amount of commonality that it exhibits within the HMI and from this generalisations are drawn. The HMI design concept has been compared with several requirement sets that have been developed during Task 16 from reviews of literature and discussions with stakeholders.
6.2 How the design concept implements the philosophy

The design concept shows how different types of platforms and payloads could be operated through a single HMI. Within that HMI there are different levels of commonality between the screens and screen areas intended for different functions. The design concept has been reviewed to assess and summarise the extent of commonality within its HMI screens and the layout of information within those screens. The assessment is organised following the designs own hierarchy of functions through which it provides the HMI for the operator to perform the mission functions. The functional hierarchy is presented as a single tree diagram in this review. The review firstly assessed whether the concrete design concept includes screens, or areas on screens, containing HMI material for a given function. If so then the leaf node of the hierarchy is given a colour that represents the degree of commonality present in the design. The meaning of the colours is described in Table 7. Table 7: Colour coding used in evaluation of HMI commonality Colour Green Yellow Red Black Definition The HMI is common across all domains The HMI is common within a domain The HMI is common within a class The HMI is specific to a model
The categorisation, shown by the assignment of a colour, is accompanied by explanatory notes in the analysis (Table 8). The categorisation is summarised in Figure 21, which shows the functional hierarchy and the commonality of the design of the HMI for each function. Two extra branches have been added to the hierarchy in this analysis. They are added below the function 2.1.2 Navigation The detail present in the design allows separate evaluation of the command and control aspects of navigation.
41
Figure 21: Level of commonality apparent in the design concept HMI Table 8 and Table 9, below, provide more detail of the evaluation of the HMI. They also contain an analysis of how the commonality in the concrete design concept may be generalised into future HMI design. The table is split into two parts for pagination purposes. The topic of generalising from the design concept is to be one of the topics for discussion at Task 16s dissemination workshop. An attempt was made to discuss this with stakeholders in a development workshop earlier in the project. The dissemination workshop is scheduled for later in 2010.
42
Table 8: Evaluation of commonality achieved in the design concept (part 1)

Ref. no 1 1.1 1.2 1.3 function Mission (Planning) Asset Tasking Scheduling Sensor Coverage Concept design The information used in allocating platform or sensor assets to mission tasks fits into a common HMI independently of the class or domain of the asset. The design concept uses the same HMI for route planning across all domains. Generalising The allocation of payloads or platforms to tasks and the scheduling of those tasks should be common.
1.4
Route Planning
Some aspects of route planning are domain-specific, such as specifying depth for underwater and altitude for air vehicles. This may require some degree of extra information albeit based on a common plan view.
2 2.1 2.1.1
Vehicle Operation Launch / Recover
The design concept does not implement launch and recovery in the concrete HMI design. Commanding a vehicle to a location is essentially the same as very short term route planning and is implemented through the same interface on the design concept HMI. Controlling the vehicle is referred to as manual operation in the design concept HMI and is vehicle-specific.
It should be possible to have a common protocol and HMI to deliver any vehicle to those physically handling the vehicle. As with route planning some aspects probably will be domain specific and require some differences between domains of vehicle.
2.1.2 2.1.2.1
Navigation Command
2.1.2.2
Control
Given that control is increasingly necessary only in case of failure of onboard systems the introduction of the extra layers of functionality necessary to implement a common HMI across different models does not appear to be wise. It seems reasonable to expect platforms to have automatic configuration. For manual configuration class-specific similarity seems reasonable but commonality across a whole domain seems excessive. When managing and checking individual platforms systems the HMI must faithfully represent the individuality of the system. For these low-level functions it is better for the HMI to be specific and different than generic and common.
2.2 2.2.1
Systems Configuration
The design concept HMI has a common layout for vehicle systems configuration.
2.2.2
Management
2.2.3
Checks
The information required to manage and check systems is specific to the model of platform. It is best laid out in a way that reflects its function and form.
43
Table 9: Evaluation of commonality achieved in the design concept (part 2)

Ref. no 3 3.1 3.1.1 Task Payload Systems Checks Concept design Generalising
3.1.2
Management
Checks and management of payloads, just like platforms, need to inform the user of the specific issues present in the system.
3.1.3
Configuration
The design concept HMI has a common layout at least within a class of payload e.g. imageproducing sensors. The design concept does not implement detailed screens for collection and download in the concrete HMI design.
The task of checking and managing a system depends on accurate information about the specific system being checked. Such HMIs need to be specific to accurately represent the system being examined. High level status information for classes of systems (e.g. hydraulics) may be generalisable. It seems reasonable to expect that classes of sensors within the payload domain of ISTAR missions could be configured by class-specific HMIs. NB: generalised to Payload Operation An HMI to manage or record the consequences (3.2.1) of payloads will be at least domain-specific or specific to mission type; probably specific to classes of payload within those domains. 3.2.2 Control: Definition of classes of payload with common control characteristics seems feasible. 3.2.3 Command: Payloads have very different outcomes depending on mission type (effectively payload domains). Differences between payload types should be designed into the HMI to reduce the risk of mistake in operation. The tools required for data interpretation will be class-specific to ensure optimal performance. Some aspects of target detection, at least the reporting formats, could be made common across all sensors involved in the domain of ISTAR missions.
3.2 3.2.1
Data Collection
3.2.2
Download
4 4.1
Exploitation Data Processing
4.2
Target Detection
The design concept HMI has a common layout at least within a class of payload imageproducing sensors
The generalisations made about the design concept can be summarised in the following hierarchy diagram, Figure 22. Note that item 3.2 in the hierarchy Data for an ISTAR mission is changed to the more general term operation.
44
Figure 22: Generalisations about commonality derived from the design concept HMI The assessment shows how it is possible for HMIs to have significant degrees of commonality, and that it is important to state the extent or parameters of commonality. That is, whether that commonality should be within classes, domains or models of UxV. To be common does not mean to be identical throughout, and as such, the leaf nodes should not be expected all to be green. The design concept makes a convincing case for the suitability of commonality across all domains of UxV for mission management functions. The design concept is also in keeping with design philosophy 2 in that the functions of control (navigational control, payload checks and management etc) are not served by a common interface. This can sensibly be expected to be the case when generalising to other HMIs. Good human factors means that functions such as systems management should not be common across domains and the HMI should be designed in the most intuitive manner for the specific class of vehicle. In formulating design philosophy 1 it was noted that to arrange the HMI mission-centrically from the top down has advantages, but that it is useful to consider the design in a payload-centric manner from the bottom up. The assessment of the design concept (above) shows that the HMI is in keeping with this philosophy that payload systems can be common within a class and should be designed intuitively for that payload.
45
6.3 Producing requirements for a common HMI

6.3.1 Origin of the requirements set
The design concept reported in this document was built upon requirements generated from earlier scenarios and hierarchical goal analysis. In order to be able to further validate the design concept against other literature and standards, and to give an output for further exploitation, an additional requirements set has been produced. The primary stakeholder for this work has expressed an interest in taking the requirements set to the JCGUAV working group for exploitation. The overall requirements set contains entries extracted from STANAG 4586, arranged around mission crew composition. These were then augmented with generic human factors requirements from Def Stan 00-250. These requirements provide a bottom up approach by adding clarity to the set with detailed design requirements for the HMI screens. Further requirements were distilled from human factors expertise and experience on previous unmanned vehicle projects. STANAG 4586 was created in the Air domain which is by far the most advanced and mature domain for unmanned vehicle standards. Evidence of the ability to successfully generalise elements of the standard across domains is in the use of STANAG 4586 in the development of MO-ATV, a BAE Systems UGV. The fact that the STANAG originated in the air domain is addressed in the cross-referencing and validation work. The stages of the requirements process and the role-based HTAs are reported here; the full requirements set can be found in Appendix A.
6.3.2 Requirements process

The requirements set covers a representative UAV mission crew composition of mission manager, payload operator (imagery), payload operator (weapons) and pilot5. A top level task analysis was developed for each of the four UAV mission crew roles. The task analysis was developed from a combination of STANAG 4586 Annex B and extensive existing systems knowledge based on the development of GCS HMI for a number of UAVs (for full references see the HMI requirements set, Appendix A). A Human Factors analysis of the tasks associated with each role was conducted to identify potential HMI user requirements. Each task analysis illustrates the key tasks that are expected of each role. See Appendix A for the break-down of the HMI driven user requirements that are associated with each task. The requirements have been further organised to improve their ease of use. Common requirements were identified and brought together under common headings to give generic UxV HMI requirements.
Pilot is generally a UAV term but in this case should be taken to mean pilot/driver as appropriate for the UxV in question.
5
46
HFIDTCPIII_T H T16_01 Version 3/ 15 June 2010 V
6.3.3 Role e-based task ana alyses
Fig gure 23: M Mission man nagement r role
Figure 24: Paylo Operator imagery role e oad
Figure 25: Payload Operato weapo role e or ons
47 7
HFIDTCPIII_T H T16_01 Version 3/ 15 June 2010 V
Figur 26: Pilot role re t
6.3.4 Developmen of requ nt uirement from stakehold inter ts s der rviews

The m minutes from the stakeh m holder interv views were examined f further r for requirement ts. To ge enerate thes requirements each in se nterview wa broken d as down into p paragraphs or line o items and assign an ID. For each of the item a require ned ms ement was generated where possib (approx. 90 require ble ements) and assigned a thematic category (e.g route plan d g. nning) which subsumed the parag h d graph ID. R Requiremen from al the interv nts ll rviews were then e group together by catego and rep ped r ory, peated or similar requ s uirements ra ationalised into a gener UxV req ric quirement. This proce resulted in 26 req ess d quirements from stakeh holder interv views (see A Appendix B). 6. .3.4.1 Com mparison with requirements f from stakeholder in nterviews The requirements derived fr rom stakeho older intervi iews were m mapped acro to the ab oss bstract n for on s monstrate w where the s stakeholder issues design concept f validatio purposes and to dem are re epresented i the desig See App in gn. pendix C fo the stake or eholder-derived require ements and m mapping to t concept. the .
6.4 Co omparis with the req son h quireme ents set t

To va alidate the abstract de esign, the fu unction goa states ha been co al ave ompared wi the ith requir rements set and any g t gaps identif fied. The full analysis is present as a complex f s ted table (Appendix D) and f x fully maps HMI func ctions acro to grou of individual oss ups requir rements. Ta 10 is in able ntended to su ummarise th with atte his ention given to the key gaps. n y The in nitial colum lists the H functio group fro where th goal stat originates with mn HMI on om he te s, the se econd colum showing the role f mn g from where the corresp ponding sys stem requir rement origin nates. The t third column then lists the overall quality of mappings between th goal n l f he states for the fu s function list and the correspon ted e nding syste ems require ement. The final e colum commen on any g mn nts gaps in the results whe there ma be no co ere ay orresponding goal states s.
48 8
Due to the requirements set being derived from STANAG 4586 the data are skewed toward the air domain. As such, requirements such as Air Tasking Orders and Air Traffic Control are grouped generically under Platform Planning and Platform Navigation goal states. Table 10: Summary of HMI functions compared to system requirements Function from concrete HMI Role from which corresponding system requirements originate Payload Operator: Imagery role Payload Operator: Imagery role Mission Manager role Quality of mappings System requirements with no corresponding goal state
Payload Management Data Exploitation Mission Management
Broadly corresponds Broadly corresponds Broadly corresponds
UAV Emergency Recovery Planning relates to specific tasks not included in the function goal states and thus the HMI design concept. General Planning Tasks relate to specific tasks of entering laser codes and is not included in the function goal states and thus the HMI design concept. Mission Progress relates to specific tasks of reporting to higher levels of command and is not included in the function goal states and thus the HMI design concept. Dissemination of Mission Plan does not have a corresponding function goal state. It is however included as an end state of the other function groups as opposed to a task in isolation.
Vehicle Management
Pilot role
Broadly corresponds
Post-Flight Tasks relate to specific tasks involving the documentation and transfer of flight data and are not included in the function goal states and thus the HMI design concept.
49
It can also be seen that one group of function goals in the HMI - Platform Systems Definition - does not map across to corresponding system requirements. These however are included as part of other groups of systems requirements involving Mission Planning. Overall, it can be seen that there is a good mapping between the requirements set and the function goal states in the HMI design.
6.5 Comparison with another functional analysis

The functional analysis of this design process parallels the work reported by Nehme et al (2007). That work arrived at a set of functional information requirements for a number of mission types that correspond to those looked at in the design process. Nehme covers a wider range of mission types but in less detail than the design concept. The design exercise leading to the design concept went further, developing information usage models and interaction sequences for the functions involved in the mission phase activities. These two approaches have been compared here as a validation exercise in two parts.
6.5.1 Comparison of Nehme et al with abstract design concept

Nehmes work describes a level of grouping of information as phase goals. Many of Nehmes phase goals correspond well to many of the information groupings used in the hierarchical structure of the design concept. These groupings in the design concept appear at different levels of the hierarchy but are always the lowest level leaf nodes. These are referred to as functional goal states. Several functional goal states in the design concept found no correspondence in Nehmes functional analysis. These were found to be functional goal states that address the practical aspects of mission operations. One of Nehmes phase goals did not correspond to the functions in the HMI at all: Scheduling of health and status reports under the heading of Mission Planning. This is a detail of mission planning not implemented in the design concept HMI. The cases where HMI functions were found to correspond to Nehmes analysis are shown in Table 11. The full mapping exercise is documented in Appendix E, part 1.
50
Table 11: Summary of mapping between HMI and Nehmes phase goals Function goal Phase goals state from HMI design concept Platform systems Tracking progress of health and status reports (UAV) management Data processing Image (map) analysis Analyzing BDA results Analyzing EO imagery Listening to transmissions Interpreting transmissions Target Detection Position tracking (only for dynamic) Sensor Coverage Planning path of area to be mapped (sensor) Planning Assessing targets Path planning (areas to search) Image/sensor matching (e.g., Automated Target Recognition) Path planning (location of target to be monitored) Data Collection Resource allocation Scheduling Scheduling of order of assessments if more than one Resource allocation & scheduling Platform Routes Planning path of area to be mapped (route) Planning Assessing routes Path planning (waypoints to the area of interest) Path planning (location of area to be monitored) Platform Tracking progress of UAVs navigation Path Replanning Maintaining flexibility for changing goal states Platform systems Tracking progress of health and status reports (UAV) management It can be seen from the analysis that gaps exist where not all of the functional goal states in the HMI have corresponding phase goals in Nehmes analysis. This is due to the greater level of detail included in the design concept.
6.5.2 Comparison of Nehme with concrete design concept

In the second part of the validation exercise with Nehmes work, elements of the concept HMI were compared with the functional/informational requirements derived from the phase goals by Nehme et al. To do this, the design concept was checked to see whether information satisfying Nehmes requirements could be found in the corresponding HMI screens. The result of this exercise was that information satisfying the requirements was found in all cases except one, the previously mentioned scheduling mechanism. The full details of this gap analysis are reported in Appendix E.
51
6.6 Summary
Overall, the design concept has been found to be consistent with appropriate standards, views of stakeholders, subject matter experts and research performed elsewhere. Where there are gaps between the requirements and the design concept, it is reasonable to say that these are concerned with specific details of the design. Not being developed to an identical set of missions, it is unsurprising that the design concept does not cover every possible aspect of missions that Nehme investigated.
52
7 Conclusions
7.1 Summary findings
7.1.1 Applicability of a common HMI for command versus control tasks
Summarising the findings of the study, a common HMI is applicable to command tasks when operating a UxV. This means that the common HMI is increasingly attractive as UxVs are increasingly operated through commands from an operator rather than by continuous human control. The increasingly infrequent occasions where operator control is required are not as amenable to operation through a common HMI. In order for a human to perform well in a control task the HMI needs to be finely tuned to the specific functionality of the mechanisms being controlled. The levels of abstraction that are intrinsic to a common HMI do not allow this close relationship. Traditionally, human control is seen as the last line of defence against loss of autonomy. Designing a common HMI for control tasks in reversionary modes is further complicated by the complexity of numerous possible failures. This is only made more difficult by the fact that the failures could easily include the very layers of abstraction and intermediate processing that make the common HMI possible.
7.1.2 Benefits and risks from a common HMI

The benefits of a common HMI are described here after a summary of the risks of a common HMI and what the design philosophy does to mitigate them. This order has been chosen to list the benefits as closely as possible to the following section, which addresses the question of how to maximise opportunity to accrue those benefits. 7.1.2.1 Potential risks from a common HMI The risks that arise from one thing looking like another, or behaving like another, are well known. They can be summed up in one word confusion. These are known to the team due to their human factors expertise and examples that arose during discussions with stakeholders. Those risks are listed below in Table 12. They are listed in tabular form together with a statement about how the design philosophy guides designers away from encountering each risk.
53
Table 12: How the design concepts mitigates some risks of a common HMI Potential risk Mitigation in the design philosophy Mission-centricity encourages payloads with similar effects to have a similar HMI, not payloads with different effects. The design is to be payloadcentric from the bottom up.
Increased likelihood of the right action wrong object human error. As it was described in one interview: one doesnt want to drop a bomb on it instead of take a photograph of it. Sub-optimal vehicle performance when under manual control HMIs for controlling vehicles can be optimised so that the control action and resulting effect on the behaviour of the vehicle take full advantage of the vehicles and operators capabilities. If a truly identical HMI were to be used one that behaves the same way as well as looking the same its performance would need to be within the range of all vehicles. It would constrain any vehicle to turn only as quickly as the slowest turner or brake as gradually as the slowestbraking vehicle. Negative transfer for control tasks HMIs may merely look the same and provoke the same general response from vehicles. However the dynamics of the response may differ. In such cases operators can ask too much of the lower performing vehicle: for example cornering a ground vehicle too fast and tipping it over or flying an air vehicle too slow and causing it to stall. Display clutter and complexity If the HMI were permanently to display all data fields and controls potentially relevant to every system that it can control it will be cluttered and complex.
The philosophy encourages the HMIs for command tasks to be similar and HMIs for control tasks to be specific to the individual systems characteristics.
Mission centricity encourages the HMI design to present to the operator the functions relevant to the mission at any given time, not to show all possible functions all the time.
7.1.2.2 Potential benefits from a common HMI The potential benefits from a common HMI are summarised well in the statement of work for this task. In order to examine them in more detail and to compare them with issues encountered during the work they are expanded and listed below. Listing them in more detail made explicit who would obtain the benefit in question. This was important when it came to assessing organisational factors that might affect the opportunity to accrue each potential benefit (see section 7.1.3).
54
6 7
Performance benefit to the individual operator Individuals who use more than one type of system find it easier to switch between systems if they are similar. The fewer differences there are between the systems the less effort it will require of the operator to remember which system they are using and how it works before going ahead and using each aspect of the HMI. This reduction in mental workload for the operator manifests itself as requiring less time to perform the task, reduced fatigue as a result of performing the task and reduced likelihood of error in the performance of the task. Organisations inherit these benefits from the individuals working within them. Training benefit to the individual operator Individuals will learn to use a new system faster if it has similarities with a system they are already familiar with using. Organisations inherit these benefits from the individuals in whom they invest training. Training development benefit to the organisation Even if no individual uses more than one of the systems that has a common HMI the organisation can obtain a benefit. The organisation can make savings through common training modules being developed for the aspects of the different systems that use the common HMI. Training delivery benefit to the organisation The common HMI training modules can be delivered to groups consisting of future users of any of the different systems that use the common HMI aspects. This is valid even if no individual then goes on to use more than one of the systems that use the common HMI. Logistical complexity benefit to the organisation If the common HMI aspects can be designed as complete logistical units then this can reduce the number of different unit types that need to be supported (hardware, software and even personnel). Logistical volume benefit to the organisation The number of spare units required to maintain availability could be reduced. Cross assurance benefit to the supplier and the procuring organisation Demonstrating assurance of an HMI with human operators is costly. If an HMI both looks and behaves the same way then perhaps that generic HMI could be assured as safe and usable. Then any vehicle type that was fully compliant with the appearance and behaviour of the HMI would be certified as usable and would not have to repeat the full set of certification activities. This might only be partially realisable as there would almost inevitably be differences in behaviour under failure conditions and reversionary modes. Cost benefit to the supplier Reuse of an already developed component brings cost savings to suppliers. Not only are development costs reduced, acceptance and certification costs could be reduced as well.
7.1.3 Organisational opportunity to accrue the potential benefits

Given the potential benefits, listed in section 7.1.2, there are constraints on how an organisation can accrue them: An organisation will have the opportunity to accrue benefits 1 and 2 even if training is not merged and optimised to achieve other benefits. An organisation will have the opportunity to accrue benefits 3 and 4 if it merges, or otherwise combines, its training across the systems that utilise the common HMI.
55
An organisation will have the opportunity to accrue benefits 5 and 6 if it merges its management of the vehicles (ownership, operation, support and maintenance) across the systems that utilise the common HMI. To accrue benefit 8 would require a great deal of sharing of investment from one project to another, potentially between suppliers as well as customer organisations.
In summary, an organisation will accrue more or less of the potential benefits depending on the extent to which it: places UxV operators in a position where they will be using different systems with similar HMIs. If operators only stay with one system or do not change roles within a system then they individually do not realise any benefits. can organise training so that individuals who will use the similar systems can be trained on the common aspects in common training sessions. has a sufficiently responsive and integrated logistics (and possibly personnel) structure that will allow supply of replacement components to the whole range of operational locations that use them.
That is, there would be little point at all in having a common HMI across systems if users were not going to operate more than one of them; if it were impossible to train those disparate users on them together or even have them follow the same training courses; or if the owners of those systems did not share logistics and support. Clearly the greater the number of different systems that use the common HMI the greater the potential benefits. This raises the issue of how to organise operations, personnel mobility, interoperability, training, vehicle ownership and vehicle operation across different services that may be operating different UxVs.
7.2 Recommendations and way forward

The extent to which a common HMI could be designed for a range of platform domains and (to a lesser extent) payload types has been investigated in this study. The design philosophy sets out a feasible architecture for common HMI design that is in keeping with technological trends and the scientific literature. The design concept provides one example of how that could be implemented. Some generalisations have been drawn from the design concept and it has been compared favourably with analytical works in the same field. The study has concluded that it is worth investigating further the idea of a common HMI for the commanding of UxVs through a mission centric design approach. It has also concluded that hands-on control of UxVs (flying, driving and sailing them) is best left for specific HMIs. This section proposes a way forward for the MOD to evaluate the potential benefits offered by a common HMI for command tasks.
56
7.2.1 Way forward towards a common HMI for command tasks

As shown in the list of potential benefits from a common HMI most of the benefits would accrue to the user organisation. The benefits that can accrue directly to suppliers relate to commonality across their products. There are certainly benefits to having a common HMI across ones product range but equally undeniable is the benefit of it being difficult for a customer to switch to a competitor. It is therefore unlikely that industry would come together to propose a common HMI to MOD. If MOD wants a common HMI then it would need to go to industry with a proposal. Such a proposal would need to be scoped according to how may vehicles could feasibly accrue the benefits. The basic parameter for accrual of benefits is the number of UxV operators who will ever operate more than one UxV HMI. One example of this could be a soldier who operates an air vehicle to reconnoitre a wide zone in the morning and then operates a ground vehicle to inspect an area of particular interest in the afternoon. Another example could be a seaman who serves for some time on a ship operating surface vehicles and then transfers to another ship operating underwater vehicles. The basic parameter will be modified by the amount of common training that can be delivered to users. If they cannot train together or cannot attend, albeit at different times, the same courses then the benefit accrued will be less. An assessment would need to be made of the extent to which common HMI design could reduce the logistical burden, which is not in the scope of this study. This report contains generalisations about levels of commonality that are feasible in major functional areas of a mission centric UxV HMI design see Table 8 and Table 9. These assessments need to be validated. The dissemination workshop will provide the opportunity for validation. Such general guidelines would need to be revisited in the light of a concrete future intent to proceed with UxVs and some extent of HMI commonality, following a review such as the one described previously. The fully validated design concept, aligned with a firmly established future intent, would inform a robust mission-centric design process, such as the one that produced the common HMI presented here.
7.2.2 Way forward regarding consistency of HMI for control tasks

Although this study concludes that developing common HMIs for particular types of control tasks for UxVs is probably not worthwhile, there is still a way that commonality could benefit control tasks. It is simply the application of the good human factors practice of consistency. For example, many different types of vehicle have fuel tanks so the symbol or terminology for that, for example, should be standardised. However any drive for standardisation needs to take into account the different information needs for the different tasks in which the symbology will be used. Generally,
57
standard symbology used in the design of control tasks should be dictated by the task itself and match the mechanisms that are being controlled. It is recommended to keep a watching brief on UxV control systems to determine if any emerging consensus leads to classes of vehicle in which a common control HMI might be applicable. The UxV market is currently in a phase of expansion and diversification. Control mechanisms, mostly for platforms but also for payloads, are constantly changing. To restrict control mechanisms for the sake of a common HMI might stifle innovation or constrain performance.
7.3 Exploitation
7.3.1 UK involvement in STANAG 4586
The requirements sets developed during this study have been requested by the primary stakeholder for use in an HMI working group to be set up in parallel to the ongoing STANAG 4586 Custodian Support team (CST). A brief progress report on the study has been made to a meeting of the STANAG 4586 committee in January 2010. It has elicited interest, particularly in the requirements set. Work encountered in the study has found the STANAG 4586 applicable outside of the air domain. It is suggested that developers become involved in the STANAG 4586 working group to develop this aspect further.
7.3.2 Links with other DTCs and beyond

The findings of this study are likely to be of interest to the Systems Engineering for Autonomous Systems (SEAS) DTC. Senior members of the SEAS DTC have already been contacted about attending the Task 16 dissemination workshop. The training issues raised in this report should be of interest to the MOD training community, for example, Head of Capability Joint Training Evaluation Simulation (HOC JTES). The current plan is to involve representatives from other DTCs through the dissemination workshop.
7.4 Closing remarks

HFI DTC Task 16 Part 1 has provided an argument for the feasibility of a common HMI for UxVs. It has developed a design philosophy that reflects the concerns and interests of key stakeholders and recent literature. An abstract and a concrete design philosophy have been proposed, being extended where necessary and evaluated for similarity with other research. In addition, this projects has, in some cases, questioned the potential benefits of a common HMI. In response, it has described a view on the extent of commonality and proposed organisational arrangements to ensure accrual of the potential benefits during exploitation. The concrete design concept will be offered up to stakeholders in the UxV
58
community for validation, and any changes to the anticipated exploitation will be submitted to the HFI DTC. This Task looks at feasibility, philosophies and concepts and is therefore, by definition, only part one of any common HMI research or development.
59
8 References
1. Common Operator Control Interface for Multi-UxV Operations. TES103687. BAE Systems. December 2008 2. Standard Interfaces of UAV Control System (UCS) for NATO UAV Interoperability. NATO Standardization Agreement STANAG 4586. Nov 8, 2007. North Atlantic Treaty Organization. 3. Defence Standard 00-250 Human Factors for Designers of Systems. Ministry of Defence. Issue 1 May 2008. 4. Nehme, C., Crandell, J.W., Cummings, M.L. (2007) An Operator Function Taxonomy for Unmanned Aerial Vehicle Missions, submission to 12th ICCRTS. 5. Kirwan, B. (1994). A Guide to Practical Human Reliability Assessment. Taylor & Francis.
60
Appendix A
A.1 Core HMI requirements
ID 1.1
Requirements set
1.2 1.2.1
1.2.1.1 1.2.1.2 1.2.1.3
1.2.1.4 1.2.1.5
1.2.2
1.2.2.1
1.2.2.2
1.2.2.3
HMI Basics: Key User HMI Requirements Issue Statement Rationale Derived Requirement Common and The goal of the Ensure common The UxV HMI consistent HMI following appearance and shall adhere to a design requirements is to behaviour. single / common appearance ensure that all Encourage HMI style guide. and behaviour. UxV HMI design interoperability. follows the same Encourage design principles modular design for look and feel. new modules will For example, the be designed in the same font, same style. appropriate use of Exploitation of colour, same user training and control methods, learning. Increase etc. the rate at which operators learn how the system operates. HMI Basics Visual Key requirements Easy to use HMI, The UxV HMI Appearance for effective visual quick to learn and shall be designed design of the HMI. intuitive. in-line with the following guidance: Simple Design Common Layout / Format Consistent and appropriate use of colour Consistent use of font type and size Common symbology and control icons Core Key requirements Easy to use HMI, The UxV HMI Behaviour for effective quick to learn and shall be designed interaction with intuitive. in-line with the the HMI. following guidance: Consistent and predictable behaviour Navigation easy access to core features Informative and timely feedback Topic
Notes The style guide shall govern both appearance and behaviour of UxV HMIs.
Sources HF Standards DEFSTAN 00-25, 00-250,
HF Standards DEFSTAN 00-25, 00-250,
61
ID 1.2.2.4
Topic
1.2.2.5
1.2.3
Appearance and Location of HCI Elements
1.2.4
Location of Information
1.2.5
HMI Navigation
1.2.5.1
1.2.5.2
1.2.5.3
1.2.5.4
1.2.5.5
HMI Basics: Key User HMI Requirements Issue Statement Rationale Notes Derived Requirement Intuitive and consistent controls Effective alerting mechanism grabs operator attention & facilitates appropriate response Top level This will help The UxV HMI requirements for promote ease of design shall general HMI use and increase ensure consistent appearance. the rate at which location of HCI users can learn / elements. familiarise themselves with the system. Top level This will help The UxV HCI shall requirements for promote ease of present the same positioning of use and increase information in the information within the rate at which same location the HMI. users can learn / within the display, familiarise menus and other themselves with relevant UxV HMI the system. features. Effective and This will help The UxV HMI intuitive promote ease of design shall follow navigation. use and increase the following Information is the rate at which guidelines for HMI quick and easy to users can learn / navigation: find as and when familiarise The UxV HMI required. themselves with shall include an the system. overview window that provides the operator with a top level of view of the location of information within the HMI. Navigation for all UxV HMI will follow a consistent style and logic. To aid navigation within the systems individual menus and pages shall be uniquely identified. Hierarchical structures shall be broad and shallow. Menus structures shall be broad and shallow.
Sources
62
ID 1.2.6
Topic System status/modes
1.2.7
Display Colours
1.2.8
Icons
1.2.9
Feedback
1.2.10
Flexibility
1.2.11
Use of language
HMI Basics: Key User HMI Requirements Issue Statement Rationale Notes Derived Requirement Top level This will help The UxV HMI requirement for ensure that the shall provide the user mode operator is aware operators with awareness. of what mode the clear presentation system is of system operating in and status/mode of thus help prevent operation. mode interaction failures / errors. Key requirements This will help The UxV HMI for effective use of maximise the shall use a colour. consistent colour impact of colour coding. This coding system. should help The UxV HMI reduce the mental shall ensure that workload and colour coding is short-term used as a memory burden secondary source on the operator. of coding. Colour coding will always be used in conjunction with a primary source of coding (e.g. shape). Top level This will help to The UxV HMI shall use icons requirement for reduce the the cognitive and that have been implementation of memory load designed to be icons onto the burden on the recognisable by HMI. operator. the User community building upon everyday objects, and icons already in service. Top level This will help to The UxV HMI requirement for ensure that the shall provide HMI feedback. user is aware of informative and the status of the timely feedback HMI. for all interactions. Top level This will help The UxV HMI requirement for users to set up shall provide the the ability to tailor their HMI to best user with the HMI. match their role ability to tailor the and individual HMI within reason requirements. to fit with role and individual differences. Top level This will help The UxV HMI requirement to ensure that the shall ensure the ensure effective HMI is consistent use of language with operator and working use of language throughout the expectations. reflects UxV HMI. functionality and meets operational and cultural expectations.
Sources HF Standards DEFSTAN 00-25, 00-250,
63
A.2 Alert requirements

ID UxV Alert Based: User-HMI Requirements Draft Derived User Requirements Notes The UxV HMI shall provide the user with an alert display - configured to match user / role Implementation to optimise presentation via requirements both visual and auditory channels. The UxV alert display shall adhere to HF standards. Please see below: General Alert Presentation Guidance Alerts should be divided into three categories: Warnings (the most important alerts), cautions and advisories (of interest only - no immediate action required). Warnings, cautions and advisories indications should be located in the operators primary field of view and should not be obscured by other HCI elements. Where possible the location of alerts and error messages shall not obscure required operational information. Alerts and error messages shall be designed to attract operator attention and will therefore appear above any other items on the screen. Where appropriate alerts and error messages shall appear in a location consistently above or below the items to which it relates. This shall not be the case where such data is in inconspicuous locations or is hidden from view. To differentiate alerts and error messages from pop-up dialogue boxes they shall have a unique application title border. The operator should be alerted by different means to different levels of alerts e.g. Warnings should be presented in an unambiguous manner in the centre of the main display. Advisories should be presented in a less intrusive manner. Consideration should be given to the use of voice warning or audio tone (attenson) to cue the operator to the alert and inform of the criticality. Operators should by a single mouse click be able to access the relevant systems information and specific decision support information. This information should be as specific to the alert as possible providing appropriate context specific information to resolve the issue rather than presenting the operator with general sub-system information. Source
1 1.1
HF Analysis HF Analysis
1.1.1
HF Analysis
1.1.2
HF Analysis
1.1.3
1.1.4
HF Analysis HF Standards DEFSTAN 00-25, 00-250, HF Standards DEFSTAN 00-25, 00-250,
1.1.5
1.1.6
HF Analysis
1.1.7
HF Analysis
1.1.8
HF Analysis
1.1.9
HF Analysis
Key Note: the presentation and management of alerts will be a key factor in effective UxV mission crew performance. A poor alerts system that is not properly integrated and has associated poor HMI implementation will result in high operator workload at critical mission points.
64
A.3 Generic role requirements

ID 1 1.1 Topic Common HMI Common Role Logon Generic HMI Requirements Issue Statement Rationale Draft Derived Requirement The UxV HMI shall provide a role specific log-on capability. Source
The goal of this requirement is to ensure a workstation HMI can be tailored to fit the needs of the role using it.
1.2
Information Presentation
The goal of this requirement is to ensure that the HMI design is flexible to different user screen real-estate requirements. The goal of this requirement is to ensure that the user is provided with ample information to acquire and maintain situation awareness in a data-rich environment.
Save time setting up each display. Provide correct information in correct format (as per task analysis for each role). Redundancy operations - e.g. having to move consoles. Any user can use any terminal (within GCS configuration constraints). Flexibility of GCS ability to introduce more analysts as required. Support hand-over speed - e.g. consoles carry over key information and settings from one user to another during watch hand-over. This will ensure the user can adapt to different screen realestate constraints.
HF Analysis
The UxV HMI shall provide the ability to configure, display and move information across multiple visual workstations (e.g. an operator using three displays).
HF Analysis
2 2.1
Situation Awareness Situation Display
Help the user to develop a mission relative picture
Maintain awareness of UxV position
To help the user appreciate sensor coverage. To help the user
The UxV HMI shall provide HF Analysis a graphical situation display that includes the following features (each function can be filtered on / off): Current UxV position. UxV direction markers (including history trials and velocity leads). UxV mission plan (highlighting legs that have been completed, the current leg and future legs). Waypoints and route points UxV sensor coverage
Threats to UxV (passive
65
ID
Topic
Generic HMI Requirements Issue Statement Rationale
2.2
Status Management System
The aim of this requirement is to ensure the user is aware of key status information, can access key status related information as required and respond to status issues quickly and effectively.
2.3
Alert Management System
Please see appendix section A.2 for further alert requirements.
identify and manage threats to UxV mission. Minimal interruption to The UxV HMI shall provide the user with a the user. status management system that includes the following features: Provide SA overview Status display of general status at a glance UxV health Prevent / Reduce Status display of specific information overload. UxV features (e.g. fuel load, communications, landing gear, etc). Ensure user attention is Current UxV mode. drawn to the most Status display of Ground important issues. Control Station features (e.g. communications setup, systems health, etc) The UxV HMI shall provide a status management system that provides quick access to in-depth status detail upon demand. The UxV HMI shall provide a status management system that is configurable by the user (the user can prioritise what they wish to see and when). This will ensure that the The UxV HMI shall user is made aware of provide the user with an alert management critical warnings and system. will increase the probability of effective early action. This will ensure effective management of communication between GCS mission crew, external sources and the UxV.
Draft Derived Requirement and active)
Source
HF Analysis
HF Analysis
HF Analysis
HF Analysis
3 3.1
Communications Communications
The goal of this requirement is to ensure that mission crew members can communicate internally and externally.
The UxV HMI shall HF Analysis provide a communications management system. The communications HF Analysis management system shall provide the following functionality: Ability to send and receive data between GCS mission crew roles Ability to send and receive information between the GCS and external sources per role. Ability to send and receive data to and from the UxV. Ability to present bandwidth limitations and optimisation options.
66
ID
Topic
Generic HMI Requirements Issue Statement Rationale
Draft Derived Requirement Prioritisation ability - for limited bandwidth operations
Source
A.4 Mission manager role requirements

Mission Manager Role
PreMission Tasks
Mission Build-Up Tasks
Mission Phase Tasks
Taking Duties
Route Planning
Payload Planning
Data-Link Planning
UxV Emergency Planning
Disseminate Mission Plan
Mission Progress
Resource Status
Airspace Control
ATC Duties
Battlefield Picture
Mission Picture
ID 0
1.1
1.1.1
1.1.2
1.1.3
1.1.4
Mission Manager: Tasks and Related User-HMI Requirements Draft Derived User Key Task/s Associated Tasks Notes Requirements Log-on to Mission The UxV HMI shall Manager Role provide a preconfigured mission manager HMI that will be invoked when the user logs-on. Pre-Mission Tasks The UxV HMI shall support mission manager pre-mission tasks. Tasking duties The UxV HMI shall support tasking duties. The UxV HMI shall Air Tasking Order (ATO) and Airspace provide the ability to Control Order (ACO) receive an ATO from tasks defining the external sources. constraints of the UAV flight. The UxV HMI shall provide the ability to visualise an ATO. The UxV HMI shall provide the ability to visualise an ACO. The UxV HMI shall provide the user with ability to edit an ATO.
Source HF Analysis
STANAG 4586 Annex B
STANAG 4586 Annex B HF Analysis
HF Analysis
HF Analysis
HF Analysis
67
ID 1.1.5
1.1.6
1.1.7
1.2
1.2.1
1.2.2
1.2.3
1.3
1.3.1
1.3.2
1.3.3
1.4
Mission Manager: Tasks and Related User-HMI Requirements Draft Derived User Key Task/s Associated Tasks Notes Requirements The UxV HMI shall provide the user with ability to edit an ACO. The UxV HMI shall provide the ability to disseminate an ATO and ACO to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate an ATO and ACO to external sources (outside the GCS). Route planning The UxV HMI shall support route planning activities. The UxV HMI shall provide the ability to display, plan and edit UxV routes. The UxV HMI shall provide the ability to disseminate planned routes to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate planned routes to external sources (outside the GCS). Payload planning The UxV HMI shall support payload planning activities. The UxV HMI shall provide the ability to display, plan and edit UxV payload operations. The UxV HMI shall provide the ability to disseminate payload plans to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate payload plans to external sources. Data-link planning The UxV HMI shall support Data-link planning activities.
Source HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
STANAG 4586 Annex B
68
ID 1.4.1
1.4.2
1.4.3
1.5
1.5.1
1.5.2
1.5.3
1.6
1.6.1
1.6.2
1.6.3
1.6.4
Mission Manager: Tasks and Related User-HMI Requirements Draft Derived User Key Task/s Associated Tasks Notes Requirements The UxV HMI shall provide the ability to display, plan and edit UxV data-link operations. The UxV HMI shall provide the ability to disseminate data-link plans to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate data-link plans to external sources. UAV emergency The UxV HMI shall support UAV recovery planning emergency recovery planning activities. The UxV HMI shall provide the ability to display, plan and edit UxV emergency recovery. The UxV HMI shall provide the ability to disseminate UxV emergency plans to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate UxV emergency plans to external sources. General Planning The UxV HMI shall Tasks support general activities. Detailed knowledge of The UxV HMI shall The geographical area current Phase and provide the ability to within which the UxV Boundary lines, display Autonomy is permitted to Engagement Areas, Boundary lines. operate. The UxV HMI shall provide the ability to display current UxV operating mode/phase. The UxV HMI shall provide the ability to display engagement areas. Hazards, Air Defence The UxV HMI shall Units (ADU) and provide the ability to Control Measures. display hazards to UxV operations.
Source HF Analysis
HF Analysis
HF Analysis
STANAG 4586 Annex B
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
69
ID 1.6.5
Key Task/s
1.6.6
1.6.7
1.6.8
1.6.9
1.6.10
1.6.11
1.6.12
1.6.13
Mission Manager: Tasks and Related User-HMI Requirements Draft Derived User Associated Tasks Notes Requirements The UxV HMI shall provide the ability to alert the user of potential hazards to planned UxV operations. Integrating/interpreting The UxV HMI shall other UAV plans from provide the ability to different systems display plans from other UxV platforms. The UxV HMI shall provide the ability to integrate plans from other UxV platforms with the primary UxV plan. Vehicle performance The UxV HMI shall models (calculate fuel provide the ability to consumption, climb display vehicle rates etc). performance models during the planning phase. The UxV HMI shall provide the ability to alert the user to any significant performance issues that may affect the performance of the UxV plan. Radar shadowing and The UxV HMI shall Line of sight provide the ability to evaluations . display UxV line of sight. The UxV HMI shall provide the ability to alert the users to any periods during the plan where the UxV will pass Beyond Line Of Sight (BLOS). Confliction and inter The UxV HMI shall visibility between provide the ability to points and routes. alert the user as to any These calculations un-feasible routes and require knowledge of waypoints. ADU/Radar characteristics and the plans of other users. The UxV HMI shall provide the user with feedback detailing why routes and/or waypoints are unfeasible.
Source HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
70
ID 1.6.14
2.1
2.1.1
2.1.2
2.1.3
2.1.4
2.1.5
2.1.6
2.1.7
Mission Manager: Tasks and Related User-HMI Requirements Draft Derived User Key Task/s Associated Tasks Notes Requirements Planning for The UxV HMI shall designator operations provide the user with will also require a the ability to enter means of laser codes and coordination/implemen keywords. ting of Laser Codes and Keywords. Mission Build-Up The UxV HMI shall Tasks support mission build-up tasks. Dissemination of The UxV HMI shall Mission Plan support the dissemination of the mission plans. The tasking authority, The UxV HMI shall provide the ability to immediately after generation of the disseminate the mission plan, for mission plan to all airspace de-confliction required recipients. and approval. The UxV HMI shall provide the ability to present sending status feedback to the user. The air vehicle via the The UxV HMI shall Data Link Interface provide the user with (DLI) for those UAVs the ability to send the that can autonomously mission plan to the execute a mission plan UxV. The UxV HMI shall provide the ability to present feedback to the user as to the mission plan sending progress. The UxV HMI shall alert the user if there are any problems with sending the UxV Mission Plan. To another UCS for The UxV HMI shall handover of UAV provide the user with control whether via the the ability to send the DLI and air vehicle or mission plan to other direct via the Ground UxV Ground Control Control Interface (GCI) Stations (GCS) to support hand-over of UxV control. The UxV HMI shall provide the ability to present feedback to the user as to the mission plan sending progress. Mission Phase Tasks The UxV HMI shall support Mission Phase Tasks
Source HF Analysis
STANAG 4586 Annex B STANAG 4586 Annex B
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
STANAG 4586 Annex B
71
ID 3.1
3.1.1
3.1.2
3.1.3
3.2
3.2.1
3.2.2
3.3
3.3.1
3.3.2
3.4
Mission Manager: Tasks and Related User-HMI Requirements Draft Derived User Key Task/s Associated Tasks Notes Requirements Mission progress The UxV HMI shall provide the user with the ability to develop and send mission status updates. Updating higher levels The UxV HMI shall of command as to provide the user with a mission status. Likely standard mission to include: Air vehicle progress report uponposition, Status of on- demand. The report board equipment, will include; air vehicle Fuel levels, On-going position, status of onachievement of board equipment, fuel mission goals. levels and mission completion status. The UxV HMI shall alert the user to any problems with mission progress. The UxV HMI shall Autonomous modes, provide the user with a manual mode, launch, display of the current taxi, etc. UxV mode. Resource availability The UxV HMI shall / status provide the ability to display the status of key GCS components. Maintaining The UxV HMI shall awareness of the provide the ability to following: UAV status display the status of subcomponents; key UxV components. Ground station; Launch system; Recovery system The UxV HMI shall alert the user to any problems with UxV status. Airspace control The UxV HMI shall support Airspace control activities. Structure and The UxV HMI shall scheduling of provide the ability to airspace. display airspace situation displays. The UxV HMI shall alert the user as to any airspace conflicts. Air Traffic Control The UxV HMI shall (ATC) duties support integration with ATC.
Source STANAG 4586 Annex B
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
STANAG 4586 Annex B
72
ID 3.4.1
3.4.2
3.5
3.5.1
3.5.2
3.5.3
3.5.4
3.6
3.6.1
3.6.2
Mission Manager: Tasks and Related User-HMI Requirements Draft Derived User Key Task/s Associated Tasks Notes Requirements Ensuring safety of The UxV HMI shall flight and integration / provide the user with co-ordination with civil the ability to and other military communicate with flights. relevant ATC authorities via the UxV. The UxV HMI shall provide the user with the ability to manually input flight level, flight heading and speed instructions (to adhere with ATC requests). Battlefield picture The UxV HMI shall provide the ability to display an up to date picture of the battlefield. Establishing and The UxV HMI shall maintaining view of provide the user with own and enemy the ability to visually locations and the local highlight tracks on the tactical situation. situation display. Knowledge used to The UxV HMI shall understand the context enable the user to of the current mission input tracks manually and to optimise the onto the situation flight plan. display. The UxV HMI shall enable the user to send manually input tracks to all GCS mission crew. The UxV HMI shall enable the user to send the situation display to higher levels of authority. Mission specific The UxV HMI shall picture provide the user with the ability to develop and display a mission specific picture. Establishing a picture The UxV HMI shall specific to the current provide the ability to mission. display the mission plan on the situation display. The UxV HMI shall display the current position of the UxV on the HMI.
Source HF Analysis
HF Analysis
STANAG 4586 Annex B
HF Analysis
HF Analysis
HF Analysis
HF Analysis
STANAG 4586 Annex B
HF Analysis
HF Analysis
73
ID 3.6.3
Key Task/s
3.6.4
Mission Manager: Tasks and Related User-HMI Requirements Draft Derived User Associated Tasks Notes Requirements The UxV HMI shall provide the ability to display velocity leads from the UxV icon. The UxV HMI shall provide the ability to centre the situation display on the 'own' UxV icon.
Source HF Analysis
HF Analysis
Key Note: This mini TA was developed from STANAG 4586 - annex B.
A.5 Hand-over requirements

ID 1 1.1 Key Task/s Hand-over control UxV Hand-Over Tasks and User HMI-Requirements Associated Tasks Draft Derived User Requirements The UxV HMI shall support UxV hand-over tasks. The UxV HMI shall provide the user with Preparation the ability to collate together hand-over information sent to relevant material. other UAV Control System (UCV) before The UxV HMI shall provide the user with the handover to include: ability to develop a hand-over file to include: Definition of the location and schedule of the handover Data link modes and frequencies Various configuration parameters. Execution coordination and synchronisation between the GCSs voice and data The UxV HMI shall support the execution of messages could be hand-over between Ground control exchanged via the Stations. UAV. The UxV HMI shall provide a communications interface allowing the user to communicate with other GCSs. The UxV HMI shall provide the user with a hand-over status indication. The indicator will display: GCS status, GCS frequencies, success or failure of comms connection (voice and data). Source Source = STANAG 4586 Annex B HF Analysis
1.1.1
HF Analysis
1.2
Source = STANAG 4586 Annex B
1.2.1
HF Analysis
1.2.2
HF Analysis
Key Note: The hand-over between GCSs is likely to be facilitated by the mission manager.
74
A.6 Dynamic re-tasking requirements

ID 1 1.1 1.1.1 Key Task/s Mission plan changes UxV Dynamic Re-Tasking User-HMI Requirements Associated Tasks Draft Derived User Requirements The UxV HMI shall support mission plan changes. Dynamic re-tasking. The UxV HMI shall support dynamic retasking. The UxV HMI shall provide the user with the ability to update and edit the current mission plan. The UxV HMI shall display the impact of changes to the mission plan upon completing mission goals. The UxV HMI shall automatically update mission plan parameters upon confirmation of mission plan changes. The UxV HMI shall provide the ability for the user to undo any changes to the mission plan and revert to the original plan. The UxV HMI shall provide a store of historic mission plans. The UxV HMI shall provide a set of template mission plans to accommodate planning in different environments, mission types (e.g. ISTAR, Strike, etc). The UxV HMI shall provide the user with the ability to send the mission plan update to all GCS mission crew members. The UxV HMI shall provide the user with the ability to receive new mission plans from external sources. The UxV HMI shall provide the user with the ability to send mission plan updates to the UxV. The UxV HMI shall provide feedback to the user that the mission plan has been received and loaded into the UxV. The UxV HMI shall provide the ability to log and time stamp all changes to the mission plan. The UxV HMI will alert the user if the mission plan changes are achievable (e.g. relative to fuel level, terrain, etc). Source Source = STANAG 4586 Annex B Source = STANAG 4586 Annex B
HF Analysis
1.1.2
HF Analysis
1.1.3
HF Analysis
1.1.4
HF Analysis HF Analysis
1.1.5 1.1.6
HF Analysis
1.1.7
HF Analysis
1.1.8
HF Analysis
1.1.9
HF Analysis
1.1.10
HF Analysis
1.1.11
HF Analysis
1.1.12
HF Analysis
Key Note: Dynamic Re-Tasking is likely to be facilitated by the mission manager.
75
A.7 Payload operator imagery requirements
ID
0 1
1.1
1.1.1
1.1.2
1.1.3
1.1.4
1.1.5
1.1.6 2
Payload Operator: Imagery - Tasks and Related User-HMI Requirements Associated Key Task/s Draft Derived User Requirements Notes Tasks Log-on to The UxV HMI shall provide a pre-configured payload payload operator HMI that will be invoked operator role when the user logs-on. Pre-Flight Tasks The UxV HMI shall support Pre-Flight Tasks. Set-up Payload Tools / The UxV HMI shall provide the user with the systems ability configure payload tools. Load mission The UxV HMI shall provide the ability to load plan mission plans onto the payload operator system. View mission The UxV HMI shall provide the ability to display plan on the mission plan on the payload operator situation display system. Set-up imagery Depend on detection mission defaults: type of requirements mission e.g. e.g. IEDs vs. IED, or larger The UxV HMI shall provide the user with the route clearance objects etc ability to configure the system to automatically / large detect objects with specific characteristics. obstacles. Set-up imagery Manage alerts - alert automation user of imagery capabilities (e.g. of interest auto-target detection, feature The UxV HMI shall provide the user with the ability to configure object detection alerts. detection, etc) Identify The UxV HMI shall allow the user to pre-define potential areas specific areas of interest along the planned of interest mission / UxV flight path. The UxV HMI shall allow the user to pre-select Time or which imagery sensors shall be active during geographically each stage of the mission. defined. In-Flight The UxV HMI shall support In-Flight Tasks. Tasks
Source HF Analysis
STANAG 4586 Annex B Source = STANAG 4586 Annex B
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis
HF Analysis STANAG 4586 Annex B
76
ID 2.1 2.1.1 2.1.2
2.1.3 2.1.4 2.2 2.2.1 2.2.2
3 3.1 3.1.1 3.1.2
3.2
3.2.1 3.2.2 3.2.3 3.2.4
3.2.5 3.2.6 3.2.7
3.2.8 3.2.9 3.2.10
Payload Operator: Imagery - Tasks and Related User-HMI Requirements Associated Key Task/s Draft Derived User Requirements Notes Tasks Check sensor The UxV HMI shall support sensor check operability tasks. The UxV HMI shall provide a display of UxV sensor status. The UxV HMI shall alert the user as to any faulty sensor. The UxV HMI shall provide a display of UxV bandwidth availability (for transmission and download tasks) The UxV HMI shall alert the user as to any problems with bandwidth availability. Check imagery The UxV HMI shall support imagery quality quality check tasks. The UxV HMI shall allow the user to request imagery whilst the UxV is in transit. The UxV HMI shall alert the user as to any limits to imagery quality. Active Mission Phase Monitor UxV The UxV HMI shall support UxV monitoring position tasks. The UxV HMI shall provide the ability to display UxV position on the situation display. The UxV HMI shall provide the ability to display the UxV mission plan on the situation display. Manage Imagery The UxV HMI shall support UxV imagery Collection collection tasks. Monitor information The UxV HMI shall provide the ability to display collected still imagery. The UxV HMI shall provide the ability to display full-motion imagery. The UxV HMI shall enable the user to mark imagery of interest. The UxV HMI shall save marked imagery to a specific file location. Download imagery of The UxV HMI shall enable the user to command interest a download of selected imagery. The UxV HMI shall provide a display of download status. The UxV HMI shall allow the user to prioritise imagery download order. Manage sensors (Monitor sensor The UxV HMI shall enable the user to select status) which sensor/s are active. The UxV HMI shall provide a display of which sensors are active. Mark areas of The UxV HMI shall enable the user to mark interest areas of interest on the situation display.
Source STANAG 4586 Annex B HF Analysis HF Analysis
HF Analysis HF Analysis STANAG 4586 Annex B HF Analysis HF Analysis STANAG 4586 Annex B STANAG 4586 Annex B HF Analysis HF Analysis STANAG 4586 Annex B
HF Analysis HF Analysis HF Analysis HF Analysis
HF Analysis HF Analysis HF Analysis
77
ID
3.2.11
3.2.12
3.2.13
3.2.14 4
4.1
4.2
5.1
5.2 5.3 5.4 6 6.1
6.2
6.3 7
Payload Operator: Imagery - Tasks and Related User-HMI Requirements Associated Key Task/s Draft Derived User Requirements Notes Tasks Interact with The UxV HMI shall associate collected imagery maps - zoom in, with its geographical location as displayed on out, pan the situation display. Interact with imagery - zoom in , out, pan, The UxV HMI shall enable the user to interact magnify, rotate, with the situation display: zoom in, zoom out, etc pan around. The UxV HMI shall enable the user to select and view available imagery from the situation display. Manipulate map layers - Insert map layers contours, satellite The UxV HMI shall enable the user to imagery, manipulate the situation display: insert map thermal imagery layers, insert map contours, annotate the map. Analyse The UxV HMI shall support imagery analysis imagery tasks. The UxV HMI shall enable the user to analyse the following imagery types: still imagery, digital motion imagery, mulit/hyperspectral imagery, Synthetic Aperture Radar (SAR); Ground Moving Target Indicator (GMTI). The UxV HMI shall enable the user to interact with available imagery: magnify, zoom in, zoom out, pan, rotate, highlight, annotate. Re-task UxV to collect further The UxV HMI shall support UxV Re-Tasking. imagery The UxV HMI shall provide the user with the ability to re-task the UxV to collect further imagery. The UxV HMI shall enable the user to request imagery from specific areas using specific sensors. The UxV HMI shall enable the user to prioritise re-tasking activities. The UxV HMI shall highlight the impact of retasking activities upon the current mission plan. Send The UxV HMI shall support dissemination of imagery imagery. The UxV HMI shall enable the user to mark imagery to be sent out. The UxV HMI shall enable the user to send imagery to other Ground Control Mission Crew: Mission Manager, Payload Controller weapons; UAV pilot. The UxV HMI shall enable the user to send imagery externally. To sources including; other GCSs, remote image analysts, remote targeteers. Post Flight The UxV HMI shall support Post-Flight Tasks Tasks.
Source
HF Analysis
HF Analysis
HF Analysis
STANAG 4586 Annex B
HF Analysis
STANAG 4586 Annex B
HF Analysis
HF Analysis HF Analysis HF Analysis STANAG 4586 Annex B HF Analysis
HF Analysis
78
ID
7.1
Payload Operator: Imagery - Tasks and Related User-HMI Requirements Associated Key Task/s Draft Derived User Requirements Notes Tasks The UxV HMI shall allow the user to request a Data Transfer post-flight data transfer from the UxV to local from UxV servers.
Source STANAG 4586 Annex B
Key Note: this mini TA was derived from STANAG 4586 - Annex B.
A.8 Payload operator weapons requirements

Payload Operator Weapon Role
Assess Target
Develop Strike Plan
Request Authorisation
Send Strike Command
Abort Strike
Conduct BDI
Command Re-Strike
Receive Target
Visualise Target Position
Conduct CDA
Visualise obstacles to strike plan
Send strike plan
Clear strike plan
Monitor status of UxV
Monitor status of weapons
ID
0 1 1.1 1.1.1 1.1.2 1.2 1.2.1 1.2.2
1.2.3 2 2.1
2.1.1
Payload Operator: Weapons - Tasks and Related User-HMI Requirements Key Task/s Associated Tasks Draft Derived User Requirements Log-on to The UxV HMI shall provide a pre-configured payload payload operator HMI that will be invoked when the user logsoperator role on. The UxV HMI shall provide the user with the ability to Assess Target assess targets. Receive Target The UxV HMI shall enable the user to receive targets. The UxV HMI shall enable the user to receive, display and analyse imagery associated with a target. The UxV HMI shall enable the user to receive, display and analyse intelligence associated with a target. Visualise Target The UxV HMI shall enable the user to visualise the Position target position. The UxV HMI shall enable the user to 'share' imagery with the Payload Operator Imagery role. The UxV HMI shall provide the user with a common situation display. The UxV HMI shall provide the ability to display the UxV position, mission plan and current position on the situation display. Develop Strike The UxV HMI shall provide the user with the ability to Plan develop and visualise strike plan/s against the target. Conduct CDA The UxV HMI shall support CDA activities. The UxV shall enable the user to visualise Collateral Damage Assessment for a variety of weapons types against the target.
Source
HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis
HF Analysis
79
ID 2.2
2.2.1 2.2.2 2.2.3 2.2.4 3 3.1 3.1.1
3.2 3.2.1 3.2.2 4 4.1. 4.1.1 4.1.2 4.1.3 4.2 4.2.1 4.2.2
4.2.3 4.2.4 5 5.1
Payload Operator: Weapons - Tasks and Related User-HMI Requirements Key Task/s Associated Tasks Draft Derived User Requirements Visualise obstacles The UxV HMI shall enable the user to visualise the to strike plan strike plan. The UxV HMI shall enable the user to display 'No strike Zones' and 'Restricted Strike Zones' on the situation display. The UxV HMI shall alert the user if any strike plans impact on non-engagement zones. The UxV HMI shall enable the user to display active threats to the UxV on the situation display. The UxV HMI shall alert the user if the strike plan endangers the safety of the UxV. Request The UxV HMI shall enable the user to request strike Authorisation authorisation. The UxV HMI shall enable the user to send the strike Send strike plan plan. The UxV HMI shall provide the user with the ability to disseminate the strike plan to required sources. Clear strike plan with authorities get engagement The UxV HMI shall enable the user to accept strike green light authorisation. The UxV HMI shall support two-way dialogue between the Payload weapons role and the authorising body. The UxV HMI shall provide visual indication of strike clearance to all GCS roles. Send Strike The UxV HMI shall enable the user to send the strike Command details to the UxV. Monitor status of The UxV HMI shall enable the user to monitor the UxV status of the UxV. The UxV HMI shall display confirmation that the UxV has received the strike plan. The UxV HMI shall visualise the position of the UxV along the strike path. The UxV HMI shall alert the user if the UxV diverges from the planned strike. Monitor status of The UxV HMI shall enable the user to monitor the status weapons of the UxV weapons. The UxV HMI shall provide a status display, detailing the status of the UxV and the status of the UxV payload. The UxV HMI shall display weapons status to the operator. the UxV HMI shall provide a strike status display: including, time to target, UxV status, Payload status, time to weapons release, weapons release. The UxV HMI shall provide the user with a display of time to target. The UxV HMI shall provide the user with the ability to Abort Strike abort the strike plan. Ability to abort The UxV HMI shall provide the user with feedback strike regards strike-abort status. Conduct Battle Damage Indication The UxV HMI shall support BDI activities.
Source HF Analysis
HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis
HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis
HF Analysis HF Analysis HF Analysis HF Analysis
HF Analysis
80
ID
6.1 6.2 6.3 7 7.1 7.2 7.3 7.4
Payload Operator: Weapons - Tasks and Related User-HMI Requirements Key Task/s Associated Tasks Draft Derived User Requirements The UxV HMI shall provide the user with the ability to command the UxV to conduct Battle Damage Indication activities. The UxV HMI shall provide the ability to display BDI imagery. The UxV HMI shall provide the user with the ability to analyse BDI imagery. Command ReThe UxV shall support re-strike activities. Strike The UxV HMI shall enable the user to command the UxV to re-strike the target. The UxV HMI shall provide the user with a status report on the feasibility of re-striking the target. The UxV shall display imagery associated with the restrike. The UxV HMI shall provide the user with the ability to analyse the imagery associated with a re-strike.
Source
HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis
Key Note: this mini TA was derived from HF ANALYSIS of Technology Demonstrator Programmes.
81
A.9 Pilot role requirements

UxV Pilot Role
Pre-Flight Tasks
Launch UxV
In-flight Tasks
Liaison With ATC
Emergency Take-Over
Post-Flight Tasks
UxV Status Check
Manual Control Status
Monitor Automatic Launch
Manual Launch
UxV Status Monitoring
ID
0 1 1.1 1.1. 1.1.2 1.1.3 1.2 1.2.1 1.2.2 1.2.3 2 2.1 2.1.1
Key Task/s Log-on to payload operator role Pre-Flight Tasks
Launch
2.1.2
2.1.3
UxV Pilot Role: Tasks and Related User-HMI Requirements Associated Tasks Draft Derived User Requirements Source The UxV HMI shall provide a pre-configured payload operator HMI that will be invoked when the user logson. HF Analysis STANAG 4586 The UxV HMI shall support Pre-Flight Tasks - Annex B UxV Status Checks The UxV HMI shall support UxV status checks. HF Analysis The UxV HMI shall provide a status display of all key UxV components. HF Analysis The UxV HMI shall alert the user if there is a problem with any UxV component. HF Analysis The UxV status display shall be available to all GCS mission crew members. HF Analysis Manual Control The UxV HMI shall support manual control check status tasks. HF Analysis The UxV HMI shall provide a status display of the GCS manual control components. HF Analysis The UxV HMI shall alert the user if there is a problem with any component of the manual control equipment. HF Analysis The UxV HMI shall provide the user with the ability to command the UxV to launch autonomously. HF Analysis STANAG 4586 The UxV HMI shall support Launch activities. - Annex B Monitor Launch The UxV HMI shall support autonomous launch autonomous activities. HF Analysis The UxV HMI shall provide a situation display of the UxV launch - including feeds from on-board cameras. HF Analysis The UxV HMI shall enable the user to configure which camera feeds are displayed on the main pilot situation display. HF Analysis The UxV HMI shall provide a shared situation display visualising the UxV current position and UxV route/mission plan against a geographical background (e.g. maps). HF Analysis
82
ID 2.2
Key Task/s
UxV Pilot Role: Tasks and Related User-HMI Requirements Associated Tasks Draft Derived User Requirements Manual Launch The UxV HMI shall support manual launch activities. The UxV HMI shall provide representative manual UxV controls that provide the user with appropriate tactile feedback when used. The UxV HMI shall provide the user with a set of manual control displays that provide sufficient information for the user to safely launch the UxV. The UxV HMI shall provide a set of manual control user display that enable the user to access key system and environmental information as required. The UxV HMI shall alert the user as to any issues with manual launch status. The UxV HMI shall support In-flight tasks. Vehicle status monitoring The UxV HMI shall support UxV status monitoring. The UxV HMI shall alert the user as to any changes in UxV mode status. The UxV HMI shall display current communications coverage status. The UxV HMI shall alert the user as to any predicted times when the UxV will pass beyond line of sight. The UxV HMI shall display current UxV position against the mission plan / route. The UxV HMI shall alert the user as to any significant deviations by the UxV from the mission route. The UxV HMI shall support ATC liaison activities. The UxV HMI shall enable the user to liaise with civilian ATC Bodies. The UxV HMI shall allow the user to command manual control of the UxV in order to respond to ATC requests. The UxV HMI shall allow the user to command automated manoeuvres in order to respond to ATC requests. The UxV HMI shall visualise the impact of any unplanned changes upon the mission plan. The UxV HMI shall support emergency take-over activities. The UxV HMI shall alert the user if manual control is required. The UxV HMI shall enable the user to take manual control of the UxV. The UxV HMI shall provide an emergency situation display that provides information as to the; problem, current UxV position and recommended actions. The UxV HMI will alert the user if they are attempting manual manoeuvres that put the UxV and other parties at risk. The UxV HMI shall provide the ability for the UAV pilot to co-ordinate emergency take-over with the Mission Manager. The UxV HMI shall support post-flight tasks.
Source STANAG 4586 - Annex B
2.2.1
HF Analysis
2.2.2
HF Analysis
2.2.3 2.2.4 3 3.1 3.1.1 3.1.2 3.1.3 3.1.4 3.1.5 4 4.1 4.2 Liaison with ATC - civilian In-flight tasks
HF Analysis HF Analysis STANAG 4586 - Annex B HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis STANAG 4586 - Annex B HF Analysis HF Analysis
4.3 4.4 5 5.1 5.2 Emergency Take-over
HF Analysis HF Analysis HF Analysis HF Analysis HF Analysis
5.3
HF Analysis
5.4
HF Analysis
5.5 6 Post-Flight tasks
HF Analysis STANAG 4586 - Annex B
83
ID 6.1
Key Task/s
6.2
UxV Pilot Role: Tasks and Related User-HMI Requirements Associated Tasks Draft Derived User Requirements Source The UxV HMI shall support transfer of post-mission data from UxV to GCS HF Analysis The UxV shall provide a record of flight history and detail any maintenance that is required (any problems that have been experienced during transit) HF Analysis
Key Note 1: This is a highly demanding role. The role requirements will vary with the introduction of higher levels of autonomy. Key challenges at higher levels of autonomy include; acquiring situation awareness when assuming manual control in emergency situations; task boredom when monitoring UxV during autonomous transit and control of multiple UxVs. Key Note 2: this mini TA was derived from a combination of STANAG 4586 - Annex B and HF analysis of Commercial UAV and UAV Technology Demonstrator Programmes.
A.10 References
Reference Categories HF sources Title / Notes DEFSTAN 00-25 / 00-250 Reference Ministry of Defence Defence Standard 00-250: Human Factors for Designers of Systems Part 0: Human Factors Integration; Part 4: HFI Methods, Tools and Techniques STANDARDISATION AGREEMENT (STANAG) 4586: Standard Interfaces of UAV Control System (UCS) for NATO UAV Interoperability," vol. March, N. S. A. (NSA), Ed., 2 NATO, 2005. Date / Version Issue 1 Publication Date 23 May 2008
UxV sources
STANAG 4586 - Annex B
NATO, 2005.
HF Analysis
HF analysis refers to analysis derived from previous experience with commercial UAV programmes and UAV technology demonstrator programmes.
2007 to present
Key Note: The scope of the analysis contained within this database was limited to only 2 key sources of UxV and HF guidance. Whilst the analysis was supported by extensive UxV programme experience the HMI requirements would benefit from the analysis of further HF and UxV material.
84
Appendix B
Stakeholder requirements
B.1 Final UxV requirements

UxV Reqt ID UxV1 New Requirement Text The HMI should validate any operator entered data or selected commands that could result in an increase in risk to the UxV e.g. route plans and manoeuvres The HMI should allow routes/flight paths to be planned for the UxV The HMI should permit the UxV to be manoeuvred using operator entered waypoints / transition-to points The HMI should allow routes/flight paths to be modified whilst the UxV is deployed The HMI should allow exclusion zones / no-go areas to be defined for the UxV The HMI should allow targets / or scan areas to be defined The HMI design should be scalable to portable/handheld hardware interfaces The HMI should contain functionality to support the following mission phases for UxVs: > Initialisation / start-up > Deployment > Modes of operation > Shut-down / recovery Multi-modal controls, such as voice control, should be considered for non-safety critical tasks e.g. cycling displays or communications frequencies Multi-modal feedback should be considered for communicating UxV health status to the operator e.g. engine sound, stick shaking The HMI should display map and tactical environment data or images The HMI should display route / flight path overlay for the UxV(s) under control The HMI should display the mission or task timeline for the UxV(s) under control The HMI should display UxV information, including: > Vehicle position (co-ordinates, range) > Vehicle orientation > Target information (time-to/on-top, ID) > Sensor feeds > Environment data (wind speed, weather) > Fuel remaining The HMI should display equipment health status of the UxV(s) in a clear and understandable fashion to provide operator awareness during operations and/or handover The HMI should provide audio feedback to the operator to indicate changes in equipment health status of the UxV(s) The HMI should enable interrogation of equipment health status of the UxV(s) The HMI should present information at an appropriate level of fidelity to ensure operator awareness of UxV(s) status and configuration during operations and handover The HMI should include pre-defined commands for operator intervention during crisis as a minimum, e.g. abort mission, go-safe, transition to end-point Interview Trace Group Validation
UxV2 UxV3 UxV4 UxV5 UxV6 UxV7 UxV8
Route planning Route planning Route planning Route planning Route planning Portability Phases
UxV9
Multi-modal Control Multi-modal Control Info Display Info Display Info Display Info Display
UxV10 UxV11 UxV12 UxV13 UxV14
UxV15
Info Display
UxV16 UxV17 UxV18
Info Display Info Display Info Display
UxV19
Control
85
UxV Reqt ID UxV20 UxV21 UxV22 UxV23
New Requirement Text The HMI should consider games controllers as an input device The HMI should permit control of multiple UxVs The HMI should include reversionary controls for the UxV for operator intervention following system failure The HMI should not allow automatic weapons release for the UxV(s), but include hardware release switches commensurate with manned systems The HMI should be intuitively designed, cognisant of human factors standards and good design principles The HMI should support configuration of UxV sub-systems The HMI should allow communication between the operator, the UxV and other stakeholders e.g. Mission Commander, Air Traffic Control, other UxV detachments
Interview Trace
Group Control Control Control Control
UxV24 UxV25 UxV26
Consistency Configuration Communication
B.2 Interview trace

UxV Reqt ID UxV1 New Interview Group Requirement Trace Text Validation The HMI should validate any operator enter data or selected commands that could result in an increase in risk to the UxV e.g. route plans and manoeuvres Interview Interview Text Trace Derived Requirement
The Human Machine The HMI should include Interface (HMI) provides the validation / data checks sole interaction with operators. It displays environmental data. Supported environmental data include bathymetry and bottom type. It displays mission objective geometry and accepts operator changes to it. It displays vehicle status information. It displays task schedule information and accepts operator changes to the tasks. Supported changes include vehicle assignment. It displays vehicle route information and accepts operator changes to it. Supported changes include waypoint addition, deletion, or movement. It also displays alerts when a plan is infeasible, when the mission objectives cannot be achieved with the assets available, when the task schedule is infeasible, when the routes are infeasible.
86
UxV Reqt ID
New Requirement Text
Interview Group Trace
Interview Interview Text Trace The sequence of events that take place when a mission change is received. The change may come from the network (representing higher command authority). Alternately, it may come from the operator station. Mission objectives arrive from the network actor and are translated into an internal representation including geometries and task types to be performed within those geometries as well as mission constraints such as restricted areas. These can be reviewed by the Operator in a separate use case. These objectives are validated (checked against any constraints). Validated objectives are then checked against the current plan to see if they can be accommodated without change. If not, planning proceeds. The operator cannot command the AV into stall conditions. Hermes 450 1 AV Operator (+autopilot), 1 Payload Operator. Desert Hawk 1 AV Controller, 1 Payload Operator. AV controller is monitoring flight parameters. This is a UK philosophy (not shared with US who use a single operator for AV and Payload operation. In the UK, AV and P/L operators are interchangeable i.e. trained to do both roles. Predator Dedicated roles (AV Operator and Payload operator, also mission commander).
Derived Requirement
The HMI should include validation / data checks
The HMI should allow validation / data checks to mitigate risk to the platform (e.g. manoeuvres beyond design tolerance)
87
UxV Reqt ID
Interview Interview Text Trace MW recalled a story of the operation of UAVs in the Congo. The UAV [specific vehicle unknown] had preprogrammed fly-to points, under the control of the operator. An emergency transition point was located in Ireland, to which the vehicle was commanded and subsequently crashed; CC commented that validation checks or warnings should have formed part of the system. CC went on to talk about the Predator Ground Control Station (GCS) and gave an example of where issues arose with common controls. The GCS is dual console one being responsible for piloting, the other for payload, but can be interchanged during reversionary states. An incident occurred where the US Customs lost a UAV, where a fault with one workstation resulted in the 2nd workstation being reconfigured to handle both payload and piloting. It was noted that an operator error during reconfiguration resulted in the thrust control being configured as a zoom control for the payload, leading to the UAV crashing. CC commented that there should have been a process/checklist to follow to mitigate this kind of error. CCB interjected, stating that a system should always configure to a stable state, and process/procedures should not be relied on to achieve this.
Derived Requirement
The HMI should include data checks to ensure entered data is valid
The HMI should only be able to be configured to a stable and safe state
The HMI should only be able to be configured to a stable and safe state
88
UxV Reqt ID UxV2
New Interview Group Interview Interview Text Requirement Trace Trace Text Route planning The HMI should Routes are planned for allow routes/flight assigned vehicle to traverse paths to be between tasks in schedule planned for the order conforming to UxV constraints in the task specification. If routes are infeasible, they are sent back for the Operator to relax constraints. Otherwise the mission plan is compiled and stored. RH stated that there are no real-time controls for controlling the path of the vehicle, and all manoeuvres are predetermined by parameters entered on the planning GUI (the only exception to this is an abort message, which instructs the vehicle to stop and surface, or go to the start or end position). The planning GUI has a MS Windows look and feel, and is primarily a point and click HMI to set geographic areas, supported by data entry dialogues. Flight plan uploaded to air vehicle. Flight plan verification required. Launch Launch to waypoint 1. AV flight is waypoint driven including required recovery pattern. May decide to launch to initial waypoint and then immediately re-plan route (dynamic mission planning). This enables detachment to get AV airborne asap (with a preprepared flight plan) and then subsequently update the flight plan as required. (Require) refuel maintenance, loading mission data, mission planning (We choose) Dropping weapons, safe ditching (certainly choose the location, probably fly the aircraft into the ground as well)
Derived Requirement
The HMI should allow routes to be defined for the vehicle
The HMI should enable route planning for UUVs
The HMI should be able to generate a route plan or have a route plan uploaded to it for the UxV.
The HMI should facilitate mission planning for the UxV
89
UxV Reqt ID UxV3
UxV4
UxV5
UxV6
UxV7
New Interview Group Interview Interview Text Requirement Trace Trace Text Route planning The HMI should Loading (human). Route permit the UxV to planning (automatic onbe manoeuvred board converts GPS using operator waypoints into a driveable entered path). Navigate to waypoint waypoints / (automatic on-board or transition-to teleoperated or manually points driven). Platform reports location to operator in task me or follow me modes. Route planning The HMI should Flight plan uploaded to air allow routes/flight vehicle. Flight plan paths to be verification required. Launch modified whilst Launch to waypoint 1. AV the UxV is flight is waypoint driven deployed including required recovery pattern. May decide to launch to initial waypoint and then immediately re-plan route (dynamic mission planning). This enables detachment to get AV airborne asap (with a preprepared flight plan) and then subsequently update the flight plan as required. Route planning The HMI should RH commented that a allow exclusion shortcoming for the vehicle is zones / no-go that it does not have a areas to be sense and avoid capability. defined for the Instead they can be UxV programmed to exclude and not enter areas input by the operator on the planning GUI. In addition, a minimum operational depth setting can be entered to prompt the vehicle to turn towards the next leg if the depth of water below the vehicle is less than the preset minimum (see figure 1). This is used to prevent beaching of the asset. Route planning The HMI should The parameters entered allow targets / or during planning activities scan areas to be include: defined Portability The HMI design Hand-held PDA with GUI. should be GPS and Wi-Fi enabling. scalable to portable/ handheld hardware interfaces
Derived Requirement
The HMI should enable navigation by operator entered waypoints
The HMI should allow changes to be made to the route plan when the UxV is deployed
The HMI shall enable exclusion/no-go zones to be defined
The HMI should enable target / scan areas to be constructed The HMI should be scalable for portability e.g. handheld PDA
90
UxV Reqt ID UxV8
New Requirement Text The HMI should contain functionality to support the following mission phases for UxVs: > Initialisation / start-up > Deployment > Modes of operation > Shut-down / recovery
Interview Group Trace Phases
Interview Interview Text Trace
Derived Requirement
They all have an initialisation The HMI should support and configuration aspect. UxV initialisation Then there's a launch phase but mission phases are specific. Launch can vary but there are special things that you do.
Initialisation and start-up deployment mission phases in different modes shutdown
Significant USV characteristics will be: Payload capability to carry the UUVs Payload handling and interface to deploy and retrieve the UUVs Communications to the UUV, as well as to the host platform Modes: manually driving teleoperating task on me follow me
The HMI should include functionality to manage UxV: > Initialisation / start-up > Deployment > Mission phases > Mission modes > Shutdown / recovery The HMI should support launch and recovery
Mission modes could include: > Manually driving > Remote driving > Task on me > Follow me Initialisation and start-up: Initialisation and start-up Health check could include: localisation of platform > Health check start-up PDAs for users > Localisation of platform > Start-up of HMIs for users Mission phases Mission phases could Changing modes of include: operation > Changing modes of Monitor platform and PDA operation positions > Monitoring platform Navigate platform. positions Monitor comms link between > Navigating platform platform and PDA > Monitoring comms link Check on health of platform between platform and Sending mode information other stakeholders > Health checking > Sending mode/operational information
91
UxV Reqt ID
Interview Interview Text Trace Monitoring modes of operation Inputting teleoperation commands. Select modes of operation using PDA. Manual mode: Engage manual mode on platform. Drive vehicle as normal Teleoperation mode: Set speed (distance up) and turn rate (distance left/right ) on GUI. Task on me mode: Push "Come here" button on PDA monitor approach of platform. Follow me mode: Initiate "follow" mode Monitor platform following in user's footsteps. MS gave examples of current typhoon voice activated commands: Change communication Frequencies Change radar displays Change IFF Squawk modes Change displays Examples of multimodal feedback were discussed. It was felt that there are occasions where the sound of the engine rather than the instrument panel provides important feedback to the pilot. It was suggested that detailed health and monitoring software could be used to generate a fake hum to be played to the UAV operator. Feedback to the operator of the health of the engine through the use of a stick shaker was discussed. MS suggested that the same cue does not need to be used.
Derived Requirement
The HMI should include pre-programmed navigation commands: > Abort > Transition to end-point > Transition to start-point > "Follow me"
UxV9
UxV10
Multi-modal controls, such as voice control, should be considered for non-safety critical tasks e.g. cycling displays or communications frequencies Multi-modal feedback should be considered for communicating UxV health status to the operator e.g. engine sound, stick shaking
Multi-modal Control
Multi-modal (voice) controls should be used for non-safety critical tasks e.g. cycling displays or comms frequencies
Multi-modal Control
Multi-modal HMI should consider representing 'engine' sound to provide system health status feedback to the operator e.g. 'comfort' background hum.
Multi-modal HMI should consider haptic feedback (e.g. stick shaker) to provide system health status feedback to the operator.
92
UxV Reqt ID UxV11
New Interview Group Requirement Trace Text Info Display The HMI should display map and tactical environment data or images
Interview Interview Text Trace The HMI serves as the exclusive interaction with the operator. It provides a tactical situation display on which geographic information appears. Overlaid on the tactical situation display appear objective geometries and routes. Also appearing in the HMI in information on the available vehicles and scheduling information on tasks. From here the operator has the ability to display information on these items and the ability to accept or edit data such as waypoints. The Human Machine Interface (HMI) provides the sole interaction with operators. It displays environmental data. Supported environmental data include bathymetry and bottom type. It displays mission objective geometry and accepts operator changes to it. It displays vehicle status information. It displays task schedule information and accepts operator changes to the tasks. Supported changes include vehicle assignment. It displays vehicle route information and accepts operator changes to it. Supported changes include waypoint addition, deletion, or movement. It also displays alerts when a plan is infeasible, when the mission objectives cannot be achieved with the assets available, when the task schedule is infeasible, when the routes are infeasible.
Derived Requirement
The HMI should provide a tactical situation display
The HMI should display environmental data appropriate to the UxV e.g. video imagery, acoustics feeds
93
UxV Reqt ID
Interview Interview Text Trace RH explained that during the mission the operators workload (due to the system), is typically low, and adopt a system monitoring role, using the operations HMI (a schematic is shown in figure 2). The operations HMI predominantly displays map and position data, vehicle movement data and equipment status data during the mission, with simple vehicle controls. These include controls for initiating vehicle tests and vehicle configuration when the vehicle is onboard, and controls to remotely signal mission abort during operations. For UGVs negotiating difficult terrain is a major challenge. It would be better to have more camera views. But this would require more bandwidth and slow down operations. That's going to vary by armed forces, tasks and the way that the control station provides the control for what you're going to do. Take off landing has most accidents. US armed force with worst record is the air force and army has best. Reason is HMI and training and experience. Air Force gives controller a pilot's eye view. Army gives them an exocentric view with a height against range view. It was noted that an improvement could be made to the current system should it be possible to represent the hydrographical map information in 3D, specifically to help the operator better understand the depth profile of the area of operations. However, this is more of an issue with the information supporting the system, rather than the HMI itself.
Derived Requirement
The HMI should display map and position data
The HMI should provide multiple camera views for UGVs to negotiate terrain.
The HMI should provide multiple views (exocentric, pilot's eye) to enhance operator spatial awareness
The HMI should consider 3D representations to enhance situational understanding of the operator
94
UxV Reqt ID UxV12
New Interview Group Requirement Trace Text Info Display The HMI should display route / flight path overlay for the UxV(s) under control
Interview Interview Text Trace Consequently, he stated the preference is to plan to deploy units on shorter missions to bring forward analysis. For example, for a 6 hour mission, 3 units could be deployed, each for 2 hours, at 2 hour intervals. This means for a 6 hour total mission, data analysis is completed 8 hours after the first vehicle enters the water. The process for defining the mission schedule. Given the objectives, information is gathered about the current situation (e.g. constraints on vehicle availability and on their capabilities, environment constraints on the objectives and the vehicle capabilities [e.g. bottom type may render vehicle hunting capabilities ineffective for certain regions], known mine like echo contacts, mine like objects and mines [these may be the targets of tasks or objects to avoid for other tasks]). Given these conditions, the system develops the vehicletask pairings. If tasks require uniform coverage planning, that is invoked for each of the pairings. Each of the resulting tracks may be considered a task to be scheduled along with identification and neutralization tasks. Additional scheduling constraints are added (e.g. launch windows and arrival windows to avoid collisions) and the tasks are scheduled if feasible. Of course, it is entirely possible that the problem is over constrained such that no solution is feasible. In this case, the results are sent back for mission review. Otherwise, they are sent for schedule review and route planning.
Derived Requirement
The HMI should enable tracking of multiple UUVs when deployed
UxV13
The HMI should display the mission or task timeline for the UxV(s) under control
Info Display
The HMI should display the mission or task schedule / timeline
95
UxV Reqt ID
Interview Interview Text Trace Ground preparation (would include getting it out of the box) getting it ready to taxi Safe taxi from dispersal point (civilian shared airport). Take off En route phase (exact boundary between take off and en-route not defined). The project is considering that different control stations would control the vehicle for different phases. In-flight refuelling (one of the ASTRAEA partners is FRL. for high endurance Active Array Radar (AAR) is a key capability) On task phase (military) ingress to target, search, reconnaissance, suppression of enemy air defences (SEAD) fly down an air corridor trying to get enemy air defences to open up and when they do you take them out (Wild Weasel missions). return to base, cruise coming back. Approach (in missed approach) Landing Taxi Shutdown Mission Planning Detachment plans mission weather, range, time required on/at target. May need to plan more than 1 mission, involving more than one air vehicle. Scope of mission may involve more than one detachment. May be a requirement for airspace deconfliction (between tasked air vehicles and with other airborne units).
Derived Requirement
The HMI should present the mission phases for the UxV to ensure awareness of mission timeline
UxV14
The HMI should display UxV information, including: > Vehicle position (co-ordinates, range) > Vehicle orientation > Target information (timeto/on-top, ID) > Sensor feeds > Environment data (wind speed, weather) > Fuel remaining
Info Display
The HMI may be required to provide additional information feeds: > Weather information
RN deploy COTS Automated The HMI should support Underwater Units for mine mine detection and identification and detection. identification for UUVs. RH was responsible for the acquisition of these units, and generation of operating procedures for their deployment.
96
UxV Reqt ID
Interview Interview Text Trace GPS. Fuel supply diesel.
Derived Requirement
UxV15
The HMI should display equipment health status of the UxV(s) in a clear and understandable fashion to provide operator awareness during operations and/or handover
Info Display
The HMI should include position and fuel information No video feedback because The HMI should include of using small PDA - not vehicle information: enough room for control GUI > Vehicle orientation and video feed on screen at > Range same time. > Video Difficult to tell orientation of platform at, for example, 200m distance. It is defined for the air The HMI should include domain. Messages on wind vehicle information: speed are not relevant for a > Wind speed ground vehicle. Some values that the UGV needed were communicated by "shoehorning" them into packages not originally intended for them. RH explained that during the The HMI should display mission the operators map and position data workload (due to the system), is typically low, and adopt a system monitoring role, using the operations HMI (a schematic is shown in figure 2). The operations HMI predominantly displays map and position data, vehicle movement data and equipment status data during the mission, with simple vehicle controls. These include controls for initiating vehicle tests and vehicle configuration when the vehicle is onboard, and controls to remotely signal mission abort during operations IR suggested that general The HMI should provide representation of the health graphical of the vehicle could be representations of displayed graphically. system health status It is important to switch to The HMI should present show the relevant an appropriate level of information at the relevant alert information with an part of the mission. Need to ability to 'drill-down' to be able to query alerts for interrogate the alert. more information when necessary.
97
UxV Reqt ID
Interview Interview Text Trace Handover and takeover activity between controllers. What are the HF and GCS issues associated with this phase? There are likely to be several. Take-off and landing are better understood. A number of systems have been lost because of lack of understanding between operators and understanding by operator of the configuration of the system they are taking over. Could be a shift change at the same workstation, handover between colocated workstations or handover between separated workstations. Is it analogous to ATC handover of control where remote workstations pass control from one to another? If platform is armed how do you ensure that weapon drop is safe, legal and effective? In ASTRAEA the organisational model is similar to an existing freight operator. (A) As it is a ground station operators could be rotated in and out of the GCS every half hour or so to avoid boredom. (A) An option considered is to have specialist operators who would step in for emergencies and critical phases, e.g. takeoff/landing. Watchkeeper procured to be used by Royal Artillery. New systems don't necessarily need to be operated by them. Other organisations will need to be changed to support UAV operations. It was agreed that tonal warnings are useful, if raised appropriately.
Derived Requirement
The HMI should clearly indicate system status (e.g. health, operational phase) to ensure effective handover between operators.
The HMI should clearly indicate system status (e.g. health, operational phase) to ensure effective handover between operators.
UxV16
The HMI should provide audio feedback to the operator to indicate changes in equipment health status of the UxV(s)
Info Display
The HMI should provide appropriate audio warnings e.g. change in health status
98
UxV Reqt ID
Interview Interview Text Trace The HMI also includes some multi-modal aspects. This is limited to audio attention getting warnings in support of equipment state changes during the mission, and is used to support the status indicators on the HMI. It was felt that no further multimodal elements would lend much value to the current system given that the operator workload during operations is low. It is important to switch to show the relevant information at the relevant part of the mission. Need to be able to query alerts for more information when necessary. System told user when in/out of range. They would have liked more graceful degradation.
Derived Requirement
Multi-modal designs should be considered where workload is high
UxV17
The HMI should enable interrogation of equipment health status of the UxV(s) The HMI should present information at an appropriate level of fidelity to ensure operator awareness of UxV(s) status and configuration during operations and handover
Info Display
The HMI should present an appropriate level of alert information with an ability to 'drill-down' to interrogate the alert
UxV18
Info Display
The HMI should provide information at an appropriate level of fidelity to ensure awareness of status e.g. communication range of vehicle to groundstation vs. IN/OUT of range
In ASTRAEA the organisational model is similar to an existing freight operator. (A) As it is a ground station operators could be rotated in and out of the GCS every half hour or so to avoid boredom. (A) An option considered is to have specialist operators who would step in for emergencies and critical phases, e.g. takeoff/landing. Watchkeeper procured to be used by Royal Artillery. New systems don't necessarily need to be operated by them. Other organisations will need to be changed to support UAV operations.
The HMI should clearly indicate the current configuration of the UxV to ensure effective handover between operators.
99
UxV Reqt ID
Interview Interview Text Trace CCB said that it was important to balance automation and allocation of functions to the user. He stated that it is not necessary to engage the user for all functions, but be wary that the system can not expect the user to rapidly move from out of the loop to in the loop. CC commented that presenting the right fidelity of information is important to manage this type of overlap. CCB then stated that whilst controlling multiple UxVs of different types an operator is more likely to be out of the loop so information presentation/management becomes much more of an important consideration. CC noted that this would be a particular challenge for programmes in the US that intend to control multiple UxVs from a single control module. Can afford to have more detail in the HMI as time pressure is not generally so great
Derived Requirement
The HMI should provide an appropriate level of system awareness to ensure operator is 'inthe-loop' when monitoring the UxV
The HMI should provide high level status/summary information to ensure operator awareness when controlling multiple UxVs
The HMI should present an appropriate level of information matched to workload / temporal pressure
100
UxV Reqt ID UxV19
New Interview Group Requirement Trace Text Control The HMI should include predefined commands for operator intervention during crisis as a minimum, e.g. abort mission, gosafe, transition to end-point
Interview Interview Text Trace This use case permits the Operator to review and alter objectives. This may be arrived at through a change in objectives, the discovery of invalid objectives, or the infeasibility of the objectives given constraints later in the planning process (e.g. availability of assets). If it was arrived at from a change in objectives, the Operator may have configured the system to proceed without Operator intervention. This behaviour is useful for testing later behaviours. Otherwise the human machine interfaces state is set and the Operator is presented with the objectives. If he or she makes changes, the revised objectives are sent back and handled in the Mission Change use case. In any case, state is reset before proceeding. Modes: follow me; task on me (come to me); teleoperate; manual driving.
Derived Requirement
The HMI should allow operator intervention in order to change preplanned actions / objectives for the UxV
Monitoring modes of operation Inputting teleoperation commands. Select modes of operation using PDA. Manual mode: Engage manual mode on platform. Drive vehicle as normal Teleoperation mode: Set speed (distance up) and turn rate (distance left/right )on GUI. Task on me mode: Push "Come here" button on PDA monitor approach of platform. Follow me mode: Initiate "follow" mode Monitor platform following in user's footsteps.
The HMI should have pre-defined commands for controlling movement of the vehicle (e.g. follow me) The HMI should include functionality to manage navigation: > Speed setting > Turn rate
101
UxV Reqt ID
Interview Interview Text Trace MS noted the Desert hawk Button A approach i.e. if the operator wants to abort a current reconnaissance mission then he presses button A and the UAV flies back to a preordained recovery location. RH explained that during the mission the operators workload (due to the system), is typically low, and they adopt a system monitoring role, using the operations HMI (a schematic is shown in figure 2). The operations HMI predominantly displays map and position data, vehicle movement data and equipment status data during the mission, with simple vehicle controls. These include controls for initiating vehicle tests and vehicle configuration when the vehicle is onboard, and controls to remotely signal mission abort during operations.
Derived Requirement
The HMI should have mission abort controls which directs the UxV to a pre-designated location
The HMI should display map and position data
102
UxV Reqt ID UxV20
New Requirement Text The HMI should consider games controllers as an input device
Interview Group Trace Control
Interview Interview Text Trace Desert Hawk (DH) P/L operator Joystick control of sensor. Desert Hawk III (Lockheed Martin) now has Xbox controller interface. Crews trained to understand the need for AV controller P/L operator interaction. E.g. AV controller told by P/L operator I want to follow that [ground-based] vehicle. AV operator should understand the impact that the AV flight path has on the P/L operators ability to maintain the target in the sensor FoV (which is not 360). Exploitation Imagery Analysis is considered a specialist skill and there is no intention to train DH operators to have the same level of skills in this area. Instead they are trained in Imagery Interpretation, such that they are able to assess imagery in terms of (e.g.) presence of objects of interest, changes in imagery etc. but not necessarily able to analyse the imagery to the same degree as a fully trained IA. For Desert Hawk there is a Remote Viewing Terminal (RVT) located at BG HQ enabling commanders to view the same imagery as the detachment operators. DH Payload Operator will normally communicate directly with TAC Group Commander (part of ISTAR cell within BG HQ). TAC Group Commander will have a good understanding of DH ops. May also (but less likely) communicate with Company Commander. Training for DH all encompassing takes 2 months. Training for (more specialist) tactical UAV will be approx 1 year to achieve full capability. Operation of DH is considered to be very intuitive no major training challenges.
Derived Requirement
The HMI should consider games controllers as an input device
103
UxV Reqt ID
Interview Interview Text Trace In a way there are more people involved in operating a UxV than a manned equivalent. So there's no real cost benefit. If it is possible to control more UxVs with fewer people then there's a cost saving to be had.Airbex. The soldier trying it said what it wanted was a Playstation or X-Box interface. A consortium formed of QinetiQ, EDO and ATLAS have developed FAST (Future Automated Sweep Technology), an autonomously piloted minesweeper, and this could be utilised to deliver a system such as SEAFOX (mine disposal system). This is typical of systems at the moment. Issue is that it takes too many people to control UAVs. You can often have a safety pilot with an overview of the operations and a payload operator. Some of our platforms can require 9 people in the ground control station to effect the mission. Target of much research is to get to opposite situation in which a single person operates multiple UAVs. Loading/unloading: lifting and carrying Tasking/route planning: Follow me mode- select follow me mode. Likewise for task on me. Teleoperation and manual N/A. Platform moving to location: Follow me and task on meno tasks. Teleoperating, user commands speed and heading. Manual, throttle control of all wheels together, steering - direction of wheels. independent braking each side. Platform moving in proximity of person (user): User can stop the platform (available all the time).
Derived Requirement
The HMI should consider games controllers for controlling vehicle movement
UxV21
The HMI should permit control of multiple UxVs
Control
The HMI should enable control of USVs and UUVs for joint operations e.g. remote delivery of an UUV to a target area by an USV
The HMI should permit control of multiple UxVs
UxV22
The HMI should include reversionary controls for the UxV for operator intervention following system failure
Control
The HMI should include manual controls for steering, throttle and braking for UGVs
104
UxV Reqt ID
Interview Interview Text Trace Basic flight control functions (in manned aircraft as well as unmanned); Takeoff and landing has been shown to be safer automated; navigation. Taxiing on military bases, takeoff & landing. Health and status monitoring. However when the systems fail the operator has to step in. If we are talking about highly autonomous systems then it would be a choice of which functions not to automate. With autonomy launch and recovery can be autonomous. Critical things are in the mission phases. Weapons release requires interaction with human. If something goes wrong you'll want to query the vehicle. Depends on the level of autonomy. Target identification should have been done before. Should the GCS replicate the protections against accidental weapons release used in manned aircraft, such as master armament switch and late arm switch? Hopefully they are being addressed by operational users of, for example, predator. Air Warfare Centre are doing work on this. MS also noted that there is a STANAG 7192 that defines the medical requirements of UAV operators, this will help shape the common HMI for example it measures such attributes such as colour blindness and necessary hand to eye co-ordination skill level. Therefore the common HMI can be designed with the knowledge that the operator has a set of necessary human skills.
Derived Requirement
The HMI should support reversionary control of the UxV should operator intervention be required on system failure
UxV23
The HMI should not allow automatic weapons release for the UxV(s), but include hardware release switches commensurate with manned systems
Control
Weapons release should not be automated but controlled by the operator
The HMI should include hardware preventions for weapons release commensurate with manned aircraft (e.g. master armament and late arm switch)
UxV24
The HMI should be intuitively designed, cognisant of human factors standards and good design principles
Consistency
The HMI should be cognisant of potential human limitations or skills e.g. colour blindness, manual dexterity, co-ordination
105
UxV Reqt ID
Interview Interview Text Trace The RN deploy Hydroids REMUS (Remote Environment Monitoring Units) 100s and 600s (see attached pdf for details). Initially 12 REMUS 100s were procured, and deployed and followed by 2 REMUS 600s. These have been deployed for Operation Telic, and 2 REMUS 100s are currently deployed. The main driver for the procurement of the additional REMUS 600s was for enhanced capability (increased operating depth and endurance) as well as an identical operating HMI to the REMUS 100s, reducing the need for retraining. It was noted that the vehicle HMI is very intuitive, and training in the use of the whole system, including familiarisation with the HMI, takes no longer than one week. It was stated that an intuitive HMI was borne out of engagement with end users when the REMUS units were initially delivered; operators provided feedback which led to a software upgrade to the system postdelivery. RH concluded that the most important requirement for an HMI is for it to be intuitive and follow consistent design conventions, rather than being a common HMI for a number of platforms. Currently they are looking at extracting UAV relevant information out of STANAG 00-250 which is a huge task. Non-specialist operators not focussed on a specific platform, therefore interface needs to be more intuitive (this is where a common HMI will offer benefits).
Derived Requirement
The HMI design should have consistent controls for UUVs to reduce training and re-training
The HMI should be designed cognisant of conventions/affordances for the design of HMI
The HMI should be designed to be intuitive
The HMI should be designed cognisant of DEF STAN 00-250 The HMI should be designed to be intuitive
106
UxV Reqt ID UxV25
New Requirement Text The HMI should support configuration of UxV sub-systems
Interview Group Trace RH7, TW3 Configuration
Interview Interview Text Trace 2 people operate the vehicle, usually a Petty Officer and a Leading Hand. One operator is responsible for programming the vehicle pre-deployment, and the other acts as quality assurance to ensure correct programming. RH commented that it is not necessarily the higher ranked individual acting as QA should their level of experience be less than the lower rank. They all have an initialisation and configuration aspect. Then there's a launch phase but mission phases are specific. Launch can vary but there are special things that you do. Significant USV characteristics will be: Payload capability to carry the UUVs Payload handling and interface to deploy and retrieve the UUVs Communications to the UUV, as well as to the host platform
Derived Requirement
The HMI should facilitate configuration of UUVs subsystems predeployment
The HMI should support UxV configuration
UxV26
The HMI should allow communication between the operator, the UxV and other stakeholders e.g. Mission Commander, Air Traffic Control, other UxV detachments
Communication
The HMI should support communication to the UUV + other stakeholders
Concept of operations MS said that Predator is mission driven through voice co-ordination of front and back seat operators. Industry are looking at confirmation of weapon release, and evaluating if it is feasible to replicate authorisation of weapon release in the manned platform weapon chain in an unmanned aircraft weapon chain.
The HMI architecture should allow voice communication between the operator and mission commander
107
UxV Reqt ID
Interview Interview Text Trace Mission Planning Detachment plans mission weather, range, time required on/at target. May need to plan more than 1 mission, involving more than one air vehicle. Scope of mission may involve more than one detachment. May be a requirement for airspace deconfliction (between tasked air vehicles and with other airborne units). The biggest training challenge is developing the culture. Fit to fly safe in the air. For a tactical UAV (e.g. Watchkeeper) there will be a third person a Mission Commander seated between the AV operator and the PL operator. Mission Commander will be the primary interface between the UCS (principally the PL operator) and the SO2 WIT/UAV suggested that a combined UAV & UGC capability might be used in Counter IED operations (detection, identification and subsequent neutralisation). Both UAV and UGV system might be operated from one operating system. Tactical UxVs Owned and operated at Brigade/Div level Few Requiring specialist operator skills Bespoke interfaces?Sub-tactical UxVs Owned and operated at BG level and below Plentiful Requiring basic operating skills Scope for common interface
Derived Requirement
The HMI should provide communication to other stakeholders e.g. ATC, other UxV detachments
The HMI should provide communication to other stakeholders e.g. ATC, mission commander, other UxV detachments
108
UxV Reqt ID
Interview Interview Text Trace Operators use chat windows to communicate between platform operators and mission planners, pilots of other UAVs. Handovers between UAVs are difficult. For example ensuring that the target being tracked by the outgoing UAV is the one being picked up by the incoming UAV. There's also handover of a target from other observers, for example special forces, and the UAV Automatic selection of which data to send over limited bandwidth. ATC? -Would be "other personnel involved". ATC would call UAV on the radio and expect a response from it just as if it were a manned aircraft. Civil pilots in same area? They would hear conversations between ATC and UAV just as they would conversations between ATC and other manned aircraft. Manned aircraft are asked to report passing designated points and surrounding pilots listen to those calls to build up picture of where other aircraft are. Informing ATC of UAV position by datalink would cut surrounding pilots out of that information flow. The US military in some circumstances provide a military ATC feed to the UAV operator in complex (but still military) airspace. This is still in restricted military airspace. On military side industry is interested because "it is very scalable". In a benign environment the monitoring of several planned data collection flights could be done by one person. Not suitable for an armed UAV with dynamic tasking, hostile environment. GCS has to support scalability. Does the image analysis role need to be considered as part of the UAV system?
Derived Requirement
The HMI should consider a chat facility to communication with other operators to manage handover or coordination
The HMI should support communications between the UxV operator and other stakeholder organisations (e.g. Air Traffic Control)
109
INTENTIONALLY BLANK
110
Appendix C
Stakeholder requirements mapped to design concept

Mission Asset tasking Sensor coverage Scheduling Route planning Vehicle Systems Operation Payload Systems Data Exploitation Data Processing Target Detection Target Recognition
New Requirement Text UxV Reqt ID
Interview Trace
Group
Configuration
Checks
Management
Maintenance
Launch/ Recover
Navigation
Configuration
Checks
Maintenance
Collection
Download
Transfer
Storage
Management
UxV1
UxV2 UxV3
UxV4
UxV5
The HMI should validate any operator entered data or selected commands that could result in an increase in risk to the UxV e.g. route plans and manoeuvres The HMI should allow routes/flight paths to be planned for the UxV The HMI should permit the UxV to be manoeuvred using operator entered waypoints / transition-to points The HMI should allow routes/flight paths to be modified whilst the UxV is deployed The HMI should allow exclusion zones / no-go areas to be defined for the UxV
Validation
Route planning Route planning Route planning Route planning
x x
x x
111
Mission Asset tasking Sensor coverage Scheduling Route planning
Vehicle Systems Operation
Payload Systems Data
Exploitation Data Processing Target Detection Target Recognition
Interview Trace
Group
Configuration
Checks
Management
Maintenance
Launch/ Recover
Navigation
Configuration
Checks
Maintenance
Collection
Download
Transfer
Storage
Management
UxV6 UxV7
UxV8
UxV9
UxV10
The HMI should allow targets / or scan areas to be defined The HMI design should be scalable to portable/handheld hardware interfaces The HMI should contain functionality to support the following mission phases for UxVs: > Initialisation / start-up > Deployment > Modes of operation > Shut-down / recovery Multi-modal controls, such as voice control, should be considered for non-safety critical tasks e.g. cycling displays or communications frequencies Multi-modal feedback should be considered for communicating UxV health status to the operator e.g. engine sound, stick shaking
Route planning Portability
x x x x x x x x x x x
x x x
x x x x x x x x
x x x
Phases
Multi-modal Control
Multi-modal Control
112
Interview Trace
Group
Configuration
Checks
Management
Maintenance
Launch/ Recover
Navigation
Configuration
Checks
Maintenance
Collection
Download
Transfer
Storage
Management
UxV11 UxV12
UxV13
UxV14
The HMI should display map and tactical environment data or images The HMI should display route / flight path overlay for the UxV(s) under control The HMI should display the mission or task timeline for the UxV(s) under control The HMI should display UxV information, including: > Vehicle position (co-ordinates, range) > Vehicle orientation > Target information (time-to/on-top, ID) > Sensor feeds > Environment data (wind speed, weather) > Fuel remaining
Info Display Info Display
x x
x x
x x
Info Display
Info Display
113
Interview Trace
Group
Configuration
Checks
Management
Maintenance
Launch/ Recover
Navigation
Configuration
Checks
Maintenance
Collection
Download
Transfer
Storage
Management
UxV15
UxV16
UxV17
UxV18
The HMI should display equipment health status of the UxV(s) in a clear and understandable fashion to provide operator awareness during operations and/or handover The HMI should provide audio feedback to the operator to indicate changes in equipment health status of the UxV(s) The HMI should enable interrogation of equipment health status of the UxV(s) The HMI should present information at an appropriate level of fidelity to ensure operator awareness of UxV(s) status and configuration during operations and handover
Info Display x x x x
Info Display x Info Display x x x x x x x x x
Info Display
114
Interview Trace
Group
Configuration
Checks
Management
Maintenance
Launch/ Recover x
Navigation x
Configuration
Checks
Maintenance
Collection
Download
Transfer
Storage
Management
UxV19
UxV20 UxV21 UxV22
UxV23
UxV24
UxV25
The HMI should include pre-defined commands for operator intervention during crisis as a minimum, e.g. abort mission, go-safe, transition to endpoint The HMI should consider games controllers as an input device The HMI should permit control of multiple UxVs The HMI should include reversionary controls for the UxV for operator intervention following system failure The HMI should not allow automatic weapons release for the UxV(s), but include hardware release switches commensurate with manned systems The HMI should be intuitively designed, cognisant of human factors standards and good design principles The HMI should support configuration of UxV sub-systems
Control
Control Control Control
n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a x x x
Control
n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a
Consistency
n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a
Configuration
115
Interview Trace
Group
Configuration
Checks
Management
Maintenance
Launch/ Recover
Navigation
Configuration
Checks
Maintenance
Collection
Download
Transfer
Storage
Management
UxV26
The HMI should allow communication between the operator, the UxV and other stakeholders e.g. Mission Commander, Air Traffic Control, other UxV detachments
Communication
116
Appendix D
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Payload systems definition Onboard resource requirements for each sensor to complete its task are identified Datalink systems required for downlink are identified Data download/storage procedures are defined Mission-specific system requirements are recorded/disseminated as appropriate
Comparison of HMI functions with requirements set

Specific Role Based System Requirements derived from STANAG 4586 and SME input
1 1.1 1.1.1 1.1.2 1.1.3
Pre-Flight Tasks
1.1.4 1.1.5
1.1.6
1.4 1.4.1 1.4.2 1.4.3
Pre-Flight Tasks
Payload Operator: Imagery Role The UxV HMI shall support Pre-Flight Tasks. Set-up Payload Tools / systems The UxV HMI shall provide the user with the ability to configure payload tools. Load mission plan The UxV HMI shall provide the ability to load mission plans onto the payload operator system. View mission plan on situation display The UxV HMI shall provide the ability to display the mission plan on the payload operator system. Set-up imagery detection defaults: type of The UxV HMI shall provide the user with the ability to mission e.g. IED, or larger objects etc configure the system to automatically detect objects with specific characteristics. Set up imagery alerts - alert user of imagery The UxV HMI shall provide the user with the ability to of interest configure object detection alerts. Identify potential areas of interest The UxV HMI shall allow the user to pre-define specific areas of interest along the planned mission / UxV flight path. The UxV HMI shall allow the user to pre-select which imagery sensors shall be active during each stage of the mission. Data-link planning The UxV HMI shall support Data-link planning activities. The UxV HMI shall provide the ability to display, plan and edit UxV data-link operations. The UxV HMI shall provide the ability to disseminate datalink plans to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate datalink plans to external sources.
117
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Ready payload systems for mission All payload systems are physically prepared All payload systems are functionally checked Payload readiness decision is made Payload readiness decision is recorded/disseminated as appropriate
2 2.1 2.1.1 2.1.2 2.1.3 2.1.4 2.2 2.2.1 2.2.2
In-Flight Tasks
Sensor operation / Data collection Sensors collect data in accordance with planned scheduling and sensor coverage Data collection status is monitored Sensors are operated to collect unplanned data if necessary Completion of data collection is recorded/disseminated as appropriate
3 3.1 3.1.1 3.1.2 3.2 3.2.1 3.2.2 3.2.3 3.2.4
Active Mission Phase
Payload Operator: Imagery Role The UxV HMI shall support In-Flight Tasks. Check sensor operability The UxV HMI shall support sensor check tasks. The UxV HMI shall provide a display of UxV sensor status. The UxV HMI shall alert the user as to any faulty sensor. The UxV HMI shall provide a display of UxV bandwidth availability (for transmission and download tasks) The UxV HMI shall alert the user as to any problems with bandwidth availability. Check imagery quality The UxV HMI shall support imagery quality check tasks. The UxV HMI shall allow the user to request imagery whilst the UxV is in transit. The UxV HMI shall alert the user as to any limits to imagery quality. The UxV HMI shall support Mission Phase Tasks Monitor UxV position The UxV HMI shall support UxV monitoring tasks. The UxV HMI shall provide the ability to display UxV position on the situation display. The UxV HMI shall provide the ability to display the UxV mission plan on the situation display. The UxV HMI shall support UxV imagery collection tasks. The UxV HMI shall provide the ability to display still imagery. The UxV HMI shall provide the ability to display full-motion imagery. The UxV HMI shall enable the user to mark imagery of interest. The UxV HMI shall save marked imagery to a specific file location.
Manage Imagery Collection Monitor information collected
118
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals 3.2.8 3.2.9 3.2.10 5 5.1 5.2 5.3 5.4 Data download Download of sensor data/imagery is initiated Download status is monitored Download process is interrupted if required Download status information is recorded/disseminated as appropriate 3 3.2 3.2.5 3.2.6 3.2.7 6 6.1 6.2
Re-task UxV to collect further imagery
Payload Operator: Imagery Role Manage sensors (Monitor sensor status) The UxV HMI shall enable the user to select which sensor/s are active. The UxV HMI shall provide a display of which sensors are active. Mark areas of interest The UxV HMI shall enable the user to mark areas of interest on the situation display. The UxV HMI shall support UxV Re-Tasking. The UxV HMI shall provide the user with the ability to retask the UxV to collect further imagery. The UxV HMI shall enable the user to request imagery from specific areas using specific sensors. The UxV HMI shall enable the user to prioritise re-tasking activities. The UxV HMI shall highlight the impact of re-tasking activities upon the current mission plan. The UxV HMI shall support Mission Phase Tasks The UxV HMI shall support UxV imagery collection tasks. The UxV HMI shall enable the user to command a download of selected imagery. The UxV HMI shall provide a display of download status. The UxV HMI shall allow the user to prioritise imagery download order. The UxV HMI shall support dissemination of imagery. The UxV HMI shall enable the user to mark imagery to be sent out. The UxV HMI shall enable the user to send imagery to other Ground Control Mission Crew: Mission Manager, Payload Controller weapons; UAV pilot.
Active Mission Phase Manage Imagery Collection Download imagery of interest
Send imagery
119
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals 6.3
7 7.1 Payload systems management Health of payload systems is monitored throughout mission Faults are detected Faults are diagnosed if possible Faults are repaired if possible Sensor task is aborted if necessary 2 2.1 2.1.1 2.1.2 2.1.3 2.1.4 2.2 2.2.1 2.2.2
Post Flight Tasks
In-Flight Tasks
Payload Operator: Imagery Role The UxV HMI shall enable the user to send imagery externally. To sources including; other GCSs, remote image analysts, remote targeteers. The UxV HMI shall support Post-Flight Tasks. Data Transfer from UxV The UxV HMI shall allow the user to request a post-flight data transfer from the UxV to local servers. The UxV HMI shall support In-Flight Tasks. Check sensor operability The UxV HMI shall support sensor check tasks. The UxV HMI shall provide a display of UxV sensor status. The UxV HMI shall alert the user as to any faulty sensor. The UxV HMI shall provide a display of UxV bandwidth availability (for transmission and download tasks) The UxV HMI shall alert the user as to any problems with bandwidth availability. The UxV HMI shall support imagery quality check tasks. The UxV HMI shall allow the user to request imagery whilst the UxV is in transit. The UxV HMI shall alert the user as to any limits to imagery quality. Payload Operator: Imagery Role The UxV HMI shall support imagery analysis tasks. The UxV HMI shall enable the user to analyse the following imagery types: still imagery, digital motion imagery, mulit/hyperspectral imagery, Synthetic Aperture Radar (SAR); Ground Moving Target Indicator (GMTI). The UxV HMI shall enable the user to interact with available imagery: magnify, zoom in, zoom out, pan, rotate, highlight, annotate.
Check imagery quality
Data exploitation goals Data processing Sensor data/imagery is sorted, evaluated and/or annotated as necessary
4 4.1
Analyse imagery
Sensor data/imagery is analysed for information content
4.2
120
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Extracted information is recorded/disseminated as appropriate Target detection
3 3.2.11
Payload Operator: Imagery Role The UxV HMI shall support Mission Phase Tasks Interact with maps - zoom in, out, pan The UxV HMI shall associate collected imagery with its geographical location as displayed on the situation display. The UxV HMI shall enable the user to interact with the situation display: zoom in, zoom out, pan around. The UxV HMI shall enable the user to select and view available imagery from the situation display. The UxV HMI shall enable the user to manipulate the situation display: insert map layers, insert map contours, annotate the map.
Sensor data/imagery is searched for targets Targets referred to in High Payoff Target List are detected Potential targets of opportunity are detected
3.2.12 3.2.13 3.2.14
Interact with imagery - zoom in, out, pan, magnify, rotate, etc
Manipulate map layers - Insert map layers contours, satellite imagery, thermal imagery
Sensors and/or sensor platforms are redirected to track mobile targets if necessary Essential target information is recorded/disseminated as appropriate Mission management goals Asset assignment Data collection requirements are identified Available assets and their data collection capabilities are identified Mission Manager Role The UxV HMI shall support Mission Phase Tasks Resource availability / status The UxV HMI shall provide the ability to display the status of key GCS components Maintaining awareness of the following: UAV The UxV HMI shall provide the ability to display the status status subcomponents; Ground station; of key UxV components Launch system; Recovery system The UxV HMI shall alert the user to any problems with UxV status.
3 3.2 3.2.1
3.2.2 Available asset capabilities are crossreferenced with data collection requirements Specific assets are assigned to meet specific requirements Asset assignments are recorded/disseminated as appropriate
121
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Sensor coverage planning Sensor coverage requirements are identified Available sensor coverage capabilities are identified Available sensor capabilities are crossreferenced with sensor coverage requirements Specific sensors are assigned to perform specific coverage tasks Asset sensor coverage plan is recorded/disseminated as appropriate Data collection scheduling Temporal demands for data collection are identified Environmental constraints on and opportunities for data collection are identified Temporal demands are cross-referenced with constraints/opportunities Data collection scheduling is defined Data collection scheduling is recorded/disseminated as appropriate Platform routes planning Launch areas are identified on map Data collection areas are identified on map
1 1.3 1.3.1 1.3.2
Pre-Mission Tasks
Payload Operator: Imagery Role The UxV HMI shall support mission manager premission tasks. Payload planning The UxV HMI shall support payload planning activities. The UxV HMI shall provide the ability to display, plan and edit UxV payload operations. The UxV HMI shall provide the ability to disseminate payload plans to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate payload plans to external sources.
1.3.3
1 1.1 1.1.1
Pre-Mission Tasks Tasking duties Air Tasking Order (ATO) and Airspace Control Order (ACO) tasks defining the constraints of the UAV flight.
The UxV HMI shall support mission manager premission tasks. The UxV HMI shall support tasking duties. The UxV HMI shall provide the ability to receive an ATO from external sources.
122
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Potential routes from launch area to data collection area are developed for each platform Potential routes are evaluated and preferred routes selected Planned (and contingency) routes are deconflicted with other traffic Planned (and contingency) routes are recorded/disseminated as appropriate
1.1.2
Payload Operator: Imagery Role The UxV HMI shall provide the ability to visualise an ATO.
1.1.3 1.1.4 1.1.5 1.1.6 1.1.7 1.2 1.2.1 1.2.2 1.2.3 1.6 1.6.1 1.6.2 1.6.3 1.6.4 1.6.5 Hazards, Air Defence Units (ADU) and Control Measures. General Planning Tasks Detailed knowledge of current Phase and Boundary lines, Engagement Areas. Route planning
The UxV HMI shall provide the ability to visualise an ACO. The UxV HMI shall provide the user with ability to edit an ATO. The UxV HMI shall provide the user with ability to edit an ACO. The UxV HMI shall provide the ability to disseminate an ATO and ACO to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate an ATO and ACO to external sources (outside the GCS). The UxV HMI shall support route planning activities. The UxV HMI shall provide the ability to display, plan and edit UxV routes. The UxV HMI shall provide the ability to disseminate planned routes to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate planned routes to external sources (outside the GCS). The UxV HMI shall provide the ability to display Autonomy Boundary lines. The UxV HMI shall provide the ability to display current UxV operating mode/phase. The UxV HMI shall provide the ability to display engagement areas. The UxV HMI shall provide the ability to display hazards to UxV operations. The UxV HMI shall provide the ability to alert the user of potential hazards to planned UxV operations.
123
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals 1.6.6 1.6.7 1.6.8 1.6.9
1.6.10 1.6.11
1.6.12
1.6.13 3 3.5 3.5.1 Mission Phase Tasks
3.5.2
3.5.3
Payload Operator: Imagery Role Integrating/interpreting other UAV plans from The UxV HMI shall provide the ability to display plans from different systems other UxV platforms. The UxV HMI shall provide the ability to integrate plans from other UxV platforms with the primary UxV plan. Vehicle performance models (calculate fuel The UxV HMI shall provide the ability to display vehicle consumption, climb rates etc). performance models during the planning phase. The UxV HMI shall provide the ability to alert the user to any significant performance issues that may effect the performance of the UxV plan. Radar shadowing and Line of sight The UxV HMI shall provide the ability to display UxV line evaluations . of sight. The UxV HMI shall provide the ability to alert the uses to any periods during the plan where the UxV will pass Beyond Line Of Sight (BLOS). Confliction and inter visibility between points The UxV HMI shall provide the ability to alert the user as and routes. These calculations require to any un-feasible routes and waypoints. knowledge of ADU/Radar characteristics and the plans of other users. The UxV HMI shall provide the user with feedback detailing why routes and/or waypoints are unfeasible. The UxV HMI shall support Mission Phase Tasks Battlefield picture The UxV HMI shall provide the ability to display an up to date picture of the battlefield. Establishing and maintaining view of own The UxV HMI shall provide the user with the ability to and enemy locations and the local tactical visually highlight tracks on the situation display. situation. Knowledge used to understand the context The UxV HMI shall enable the user to input tracks of the current mission and to optimise the manually onto the situation display. flight plan. The UxV HMI shall enable the user to send manually input tracks to all GCS mission crew.
124
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals 3.5.4 3.6 3.6.1 3.6.2 3.6.3 3.6.4
Payload Operator: Imagery Role The UxV HMI shall enable the user to send the situation display to higher levels of authority. Mission specific picture The UxV HMI shall provide the user with the ability to develop and display a mission specific picture. Establishing a picture specific to the current The UxV HMI shall provide the ability to display the mission. mission plan on the situation display. The UxV HMI shall display the current position of the UxV on the HMI. The UxV HMI shall provide the ability to display velocity leads from the UxV icon. The UxV HMI shall provide the ability to centre the situation display on the 'own' UxV icon. Pre-Mission Tasks UAV emergency recovery planning The UxV HMI shall support mission manager premission tasks. The UxV HMI shall support UAV emergency recovery planning activities. The UxV HMI shall provide the ability to display, plan and edit UxV emergency recovery. The UxV HMI shall provide the ability to disseminate UxV emergency plans to all UxV mission crew members. The UxV HMI shall provide the ability to disseminate UxV emergency plans to external sources. The UxV HMI shall provide the user with the ability to enter laser codes and keywords.
1 1.5 1.5.1 1.5.2 1.5.3 1.6 1.6.14
General Planning Tasks Planning for designator operations will also require a means of coordination/implementing of Laser Codes and Keywords. Mission Build-Up Tasks
The UxV HMI shall support mission build-up tasks.
125
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals 2.1 2.1.1
2.1.2 2.1.3
2.1.4 2.1.5 2.1.6
2.1.7 3 3.1 3.1.1 Mission Phase Tasks
3.1.2 3.1.3
Payload Operator: Imagery Role Dissemination of Mission Plan The UxV HMI shall support the dissemination of the mission plans. The tasking authority, immediately after The UxV HMI shall provide the ability to disseminate the generation of the mission plan, for airspace mission plan to all required recipients. de-confliction and approval The UxV HMI shall provide the ability to present sending status feedback to the user The air vehicle via the Data Link Interface The UxV HMI shall provide the user with the ability to send (DLI) for those UAVs that can autonomously the mission plan to the UxV execute a mission plan The UxV HMI shall provide the ability to present feedback to the user as to the mission plan sending progress. The UxV HMI shall alert the user if there are any problems with sending the UxV Mission Plan. To another UCS for handover of UAV control The UxV HMI shall provide the user with the ability to send whether via the DLI and air vehicle or direct the mission plan to other UxV Ground Control Stations via the GCI (GCS) to support hand-over of UxV control. The UxV HMI shall provide the ability to present feedback to the user as to the mission plan sending progress. The UxV HMI shall support Mission Phase Tasks Mission progress The UxV HMI shall provide the user with the ability to develop and send mission status updates. Updating higher levels of command as to The UxV HMI shall provide the user with a standard mission status. Likely to include: Air vehicle mission progress report upon-demand. The report will position, Status of on-board equipment, include; air vehicle position, status of on-board equipment, Fuel levels, On-going achievement of fuel levels and mission completion status. mission goals The UxV HMI shall alert the user to any problems with mission progress. The UxV HMI shall provide the user with a display of the current UxV mode.
126
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Vehicle management goals Platform systems definition Resource requirements for each platform to complete its mission are identified Networked C4I systems required for the mission are identified Inter-platform communication and coordination procedures are defined Command/control arrangements (including changes/handovers) are defined Mission-specific system requirements are recorded/disseminated as appropriate Platform systems mission readiness 1 All platform systems are physically prepared 1.1 All platform systems are functionally checked 1.1. Mission launch readiness state is assessed Mission launch go-ahead decision is made Mission launch go-ahead decision is recorded/disseminated as appropriate 1.1.2 1.1.3 1.2 1.2.1 1.2.2 1.2.3
Payload Operator: Imagery Role Pilot Role
Pre-Flight Tasks UxV Status Checks
Manual Control status
The UxV HMI shall support Pre-Flight Tasks The UxV HMI shall support UxV status checks. The UxV HMI shall provide a status display of all key UxV components. The UxV HMI shall alert the user if there is a problem with any UxV component. The UxV status display shall be available to all GCS mission crew members. The UxV HMI shall support manual control check tasks The UxV HMI shall provide a status display of the GCS manual control components. The UxV HMI shall alert the user if there is a problem with any component of the manual control equipment The UxV HMI shall provide the user with the ability to command the UxV to launch autonomously.
127
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Platform launch/recovery Individual platforms are launched/recovered Individual platform launches/recoveries are confirmed Mission launch/recovery status is recorded/disseminated as appropriate
2 2.1 2.1.1 2.1.2
Launch
2.1.3
2.2 2.2.1
2.2.2
2.2.3
2.2.4 Platform navigation Platforms are navigated according to planned routes Environment is monitored for unexpected events/changes Planned routes are re-assessed in light of new environment information Planned routes are altered if necessary (contingency selected) 3 3.1 3.1.3 3.1.4 3.1.5 In-flight tasks
Payload Operator: Imagery Role The UxV HMI shall support Launch activities. Monitor Launch - autonomous The UxV HMI shall support autonomous launch activities. The UxV HMI shall provide a situation display of the UxV launch - including feeds from on-board cameras. The UxV HMI shall enable the user to configure which camera feeds are displayed on the main pilot situation display. The UxV HMI shall provide a shared situation display visualising the UxV current position and UxV route/mission plan against a geographical background (e.g. maps). Manual Launch The UxV HMI shall support manual launch activities. The UxV HMI shall provide representative manual UxV controls that provide the user with appropriate tactile feedback when used. The UxV HMI shall provide the user with a set of manual control displays that provide sufficient information for the user to safely launch the UxV. The UxV HMI shall provide a set of manual control user display that enable the user to access key system and environmental information as required. The UxV HMI shall alert the user as to any issues with manual launch status. The UxV HMI shall support In-flight tasks. Vehicle status monitoring The UxV HMI shall support UxV status monitoring. The UxV HMI shall alert the user as to any predicted times when the UxV will pass beyond line of sight. The UxV HMI shall display current UxV position against the mission plan / route. The UxV HMI shall alert the user as to any significant deviations by the UxV from the mission route.
128
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals 4 Changes to routes are recorded/disseminated as appropriate Platforms are navigated off-route if necessary 4.1 4.2 4.3 4.4 3 3.3 3.3.1 3.3.2 3.4 3.4.1 3.4.2
Liaison with ATC civilian
Payload Operator: Imagery Role The UxV HMI shall support ATC liaison activities. The UxV HMI shall enable the user to liaise with civilian ATC Bodies. The UxV HMI shall allow the user to command manual control of the UxV in order to respond to ATC requests. The UxV HMI shall allow the user to command automated manoeuvred in order to respond to ATC requests. The UxV HMI shall visualise the impact of any un-planned changes upon the mission plan. The UxV HMI shall support Mission Phase Tasks Airspace control The UxV HMI shall support Airspace control activities. Structure and scheduling of airspace The UxV HMI shall provide the ability to display airspace situation displays. Structure and scheduling of airspace The UxV HMI shall alert the user as to any airspace conflicts. Air Traffic Control (ATC) duties The UxV HMI shall support integration with ATC Ensuring safety of flight and integration / co- The UxV HMI shall provide the user with the ability to ordination with civil and other military flights. communicate with relevant ATC authorities via the UxV. The UxV HMI shall provide the user with the ability to manually input flight level, flight heading and speed instructions (to adhere with ATC requests). The UxV HMI shall support emergency take-over activities. The UxV HMI shall alert the user if manual control is required. The UxV HMI shall enable the user to take manual control of the UxV. The UxV HMI shall provide an emergency situation display that provides information as to the; problem, current UxV position and recommended actions.
Mission Phase Tasks
5 5.1 5.2 5.3
Emergency Take-over
129
Function and sub function goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals 5.4
5.5
Platform systems management Health of platform systems is monitored throughout mission Faults are detected Faults are diagnosed if possible Faults are repaired if possible Platform plan is aborted if necessary Platform is destroyed if necessary
3 3.1 3.1.1 3.1.2
In-flight tasks
Payload Operator: Imagery Role The UxV HMI will alert the user if they are attempting manual manoeuvres that put the UxV and other parties at risk. The UxV HMI shall provide the ability for the UAV pilot to co-ordinate emergency take-over with the Mission Manager. The UxV HMI shall support In-flight tasks. Vehicle status monitoring The UxV HMI shall support UxV status monitoring. The UxV HMI shall alert the user as to any changes in UxV mode status. The UxV HMI shall display current communications coverage status.
6 6.1 6.2
Post-Flight tasks
The UxV HMI shall support post-flight tasks. The UxV HMI shall support transfer of post-mission data from UxV to GCS The UxV shall provide a record of flight history and detail any maintenance that is required (any problems that have been experienced during transit)
* Sections shown in blue denote no corresponding function goal state * Sections shown in beige denote no corresponding systems requirement
130
Appendix E
Function and subfunction goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Payload systems definition Onboard resource requirements for each sensor to complete its task are identified Datalink systems required for downlink are identified Data download/storage procedures are defined Mission-specific system requirements are recorded/disseminated as appropriate Ready payload systems for mission All payload systems are physically prepared All payload systems are functionally checked Payload readiness decision is made Payload readiness decision is recorded/disseminated as appropriate Sensor operation / Data collection Sensors collect data in accordance with planned scheduling and sensor coverage Data collection status is monitored
Comparison of design concept with Nehme et al. (2007)

Mission Phase (Nehme) Phase Goals (Nehme) Part 2 Analysis Functional / Information Requirements Concept HMI elements developed from (Nehme) function and subfunction goal states (for ISTAR activities) Threat area information and no fly zone information Threat area information and no fly zone information Mission Management - Maps on overview and maps and data input on task screens Mission Management - Maps on overview and maps and data input on task screens
Mission Planning
Mapping Planning path of area to be mapped (sensor) Planning path of area to be mapped (route) Scheduling of health and status reports
Mission Management
Tracking progress of UAVs Tracking progress of health and status reports (UAV) Tracking progress of health and status reports (sensor) Image (map) analysis Resource allocation
Mission Replanning
Scheduling mechanism Not explicitly shown in concept HMI Decision support for path planning (including Mission Management - Mission and Task loitering) Requirements on overview screens Vehicle Management - Maps, status, video display screens on overview and tasks Health and Status indicators screens Vehicle Management - Not explicitly Health and Status indicators shown in concept HMI Payload Management - Not explicitly Health and Status indicators shown in concept HMI Image analysis tools (zoom pan filtering) Exploit - Visual data display screens Mission Management - Mission and Task Asset coverage re-plan decision support Requirements on overview screens
Battle Damage Assessment Mission Planning Assessing targets Assessing routes Threat area information and no fly zone information Threat area information and no fly zone information Mission Management - Maps on overview and maps and data input on task screens Mission Management - Maps on overview and maps and data input on task screens
131
Function and subfunction goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Sensors are operated to collect unplanned data if necessary Completion of data collection is recorded/disseminated as appropriate Data download Download of sensor data/imagery is initiated
Mission Phase (Nehme)
Phase Goals (Nehme)
Part 2 Analysis Functional / Information Requirements Concept HMI elements developed from (Nehme) function and subfunction goal states (for ISTAR activities) Mission Management - Timeline on overview screens Payload Management - Timeline screens
Mapping Scheduling of order of assessments if more than one Scheduling mechanism
Scheduling of health and status reports
Download status is monitored Download process is interrupted if required Download status information is recorded/disseminated as appropriate Payload systems management Health of payload systems is monitored throughout mission Faults are detected Faults are diagnosed if possible Faults are repaired if possible Sensor task is aborted if necessary
Mission Management
Tracking progress of UAVs Tracking progress of health and status reports (UAV) Tracking progress of health and status reports (sensor) Analyzing BDA results
Mission Replanning
Resource allocation Target Acquisition (Static and Dynamic) Path planning (areas to search) Path planning (waypoints to the area of interest) Scheduling of health and status reports
Scheduling mechanism Not explicitly shown in concept HMI Decision support for path planning (including Mission Management - Mission and Task loitering) Requirements on overview screens Vehicle Management - Maps, status, video display screens on overview and tasks Health and Status indicators screens Vehicle Management - Not explicitly Health and Status indicators shown in concept HMI Payload Management - Not explicitly Health and Status indicators shown in concept HMI Exploit - Real time and recorded visual Image analysis tools (zoom pan filtering) data display screens Mission Management - Mission and Task Asset coverage re-plan decision support Requirements on overview screens
Mission Planning
Threat area information and no fly zone information Threat area information and no fly zone information
Mission Management - Maps on overview and maps and data input on task screens Mission Management - Maps on overview and maps and data input on task screens
Data exploitation goals
Scheduling mechanism Not explicitly shown in concept HMI Decision support for path planning (including Mission Management - Mission and Task loitering) Requirements on overview screens
132
Function and subfunction goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals
Phase Goals (Nehme)
Part 2 Analysis Functional / Information Requirements Concept HMI elements developed from (Nehme) function and subfunction goal states (for ISTAR activities) Vehicle Management - Maps, status, video display screens on overview and tasks screens Vehicle Management - Not explicitly shown in concept HMI Payload Management - Not explicitly shown in concept HMI Exploit - Real time and recorded video display screens Mission - Not explicitly shown in concept HMI Exploit - Not explicitly shown in concept HMI Exploit - Not explicitly shown in concept HMI Not explicitly shown in concept HMI Mission Management - Mission and Task Requirements on overview screens Mission Management - Mission and Task Requirements on overview screens
Mapping Mission Management
Data processing Sensor data/imagery is sorted, evaluated and/or annotated as necessary Sensor data/imagery is analysed for information content Extracted information is recorded/disseminated as appropriate Target detection
Tracking progress of UAVs Tracking progress of health and status reports (UAV) Tracking progress of health and status reports (sensor) Analyzing EO imagery Image/sensor matching (e.g., ATR) Position tracking (only for dynamic)
Health and Status indicators Health and Status indicators Health and Status indicators Support for viewing results and storing results Support for sensor matching Support for tracking position of target Signal detection Predictive path planning
Sensor data/imagery is searched for targets Targets referred to in High Payoff Target List are detected Potential targets of opportunity are detected Sensors and/or sensor platforms are Mission redirected to track mobile targets if Replanning necessary Essential target information is recorded/disseminated as appropriate Mission management goals Asset assignment Data collection requirements are identified Available assets and their data collection capabilities are identified Mission Planning
Path Replanning Resource allocation Listening Path planning (location of target to be monitored) Path planning (location of area to be monitored) Scheduling of health and status reports
Re-planning decision support Rescheduling decision support
Threat area information and no fly zone information Threat area information and no fly zone information Scheduling mechanism
Mission Management - Maps on overview and maps and data input on task screens Mission Management - Maps on overview and maps and data input on task screens Not explicitly shown in concept HMI
133
Function and subfunction goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Available asset capabilities are crossreferenced with data collection requirements
Phase Goals (Nehme)
Part 2 Analysis Functional / Information Requirements Concept HMI elements developed from (Nehme) function and subfunction goal states (for ISTAR activities) Decision support for path planning (including Mission Management - Mission and Task loitering) Requirements on overview screens Vehicle Management - Maps, status, video display screens on overview and tasks Health and Status indicators screens Vehicle Management - Not explicitly Health and Status indicators shown in concept HMI Payload Management - Not explicitly Health and Status indicators shown in concept HMI Exploit - Not explicitly shown in concept Audio signal detection HMI Exploit - Not explicitly shown in concept Alert management HMI Exploit - Not explicitly shown in concept HMI Mission Management - Mission and Task Requirements on overview screens Mission Management - Mission and Task Requirements on overview screens
Mapping
Specific assets are assigned to meet specific Mission Management requirements Asset assignments are recorded/disseminated as appropriate Sensor coverage planning Sensor coverage requirements are identified Available sensor coverage capabilities are identified Available sensor capabilities are crossreferenced with sensor coverage requirements Mission Specific sensors are assigned to perform Replanning specific coverage tasks Asset sensor coverage plan is recorded/disseminated as appropriate Data collection scheduling Temporal demands for data collection are identified Environmental constraints on and opportunities for data collection are identified Temporal demands are cross-referenced with constraints/opportunities Data collection scheduling is defined Data collection scheduling is recorded/disseminated as appropriate
Tracking progress of UAVs Tracking progress of health and status reports (UAV) Tracking progress of health and status reports (sensor) Listening to transmissions
Interpreting transmissions Maintaining flexibility for changing goal states Resource allocation & scheduling
Signal analysis decision support Re-planning decision support
134
Function and subfunction goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Platform routes planning Launch areas and are identified on map Data collection areas are identified on map Potential routes from launch area to data collection area are developed for each platform Potential routes are evaluated and preferred routes selected Planned (and contingency) routes are deconflicted with other traffic Planned (and contingency) routes are recorded/disseminated as appropriate Vehicle management goals Platform systems definition Resource requirements for each platform to complete its mission are identified Networked C4I systems required for the mission are identified Inter-platform communication and coordination procedures are defined Command/control arrangements (including changes/handovers) are defined Mission-specific system requirements are recorded/disseminated as appropriate Platform systems mission readiness All platform systems are physically prepared All platform systems are functionally checked Mission launch readiness state is assessed
Phase Goals (Nehme)
Part 2 Analysis Functional / Information Requirements Concept HMI elements developed from (Nehme) function and subfunction goal states (for ISTAR activities)
Mapping
135
Function and subfunction goal states Mission Phase derived from Hierarchical Goal Analysis (Nehme) (for ISTAR activities) Payload management goals Mission launch go-ahead decision is made Mission launch go-ahead decision is recorded/disseminated as appropriate Platform launch/recovery Individual platforms are launched/recovered Individual platform launches/recoveries are confirmed Mission launch/recovery status is recorded/disseminated as appropriate Platform navigation Platforms are navigated according to planned routes Environment is monitored for unexpected events/changes Planned routes are re-assessed in light of new environment information Planned routes are altered if necessary (contingency selected) Changes to routes are recorded/disseminated as appropriate Platforms are navigated off-route if necessary Platform systems management Health of platform systems is monitored throughout mission Faults are detected Faults are diagnosed if possible Faults are repaired if possible Platform plan is aborted if necessary
Phase Goals (Nehme)
Mapping
136
Function and subfunction goal states derived from Hierarchical Goal Analysis (for ISTAR activities) Payload management goals Platform is destroyed if necessary
Phase Goals (Nehme)
Mapping
* Background colour denotes where function and subfunction goal states correspond to Nehme's phase goals
137
- End of Document -
138
139

t16p1 01 Common Hmi Uxv

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

t16p1 01 Common Hmi Uxv

Uploaded by

Copyright:

Available Formats

Common HMI for UxVs: Design Philosophy and Design Concept

Reference ............................................HFIDTCPIII_T16_01 Version .................................................................................3 Date .................................................................. 15 June 2010

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

Executive Summary ................................................................................... 1

Overview of the work undertaken ............................................................... 9

Design philosophy .................................................................................... 13

Design Concept ........................................................................................ 19

Evaluation of the design concept.............................................................. 41

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

1.2 Statement of the problem

1.3 Report assumptions

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

1.3.1 Timescale assumptions

1.4 Design Philosophy

Philosophy 1 Mission Centricity

Philosophy 2 Command versus Control

1.5 Design approach

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

1.7 Recommendations and way forward

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

2.2 Background to the report

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

2.3 Summary of the report

2.4 Some terms and concepts used in the report

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

2.5 Timescale assumptions

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

3 Overview of the work undertaken

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

ISTAR Air Domains Vehicle Ground Underwater Surface

3.1.2 Literature review

3.1.3 Formulation of design philosophy

Philosophy 1 Mission Centricity

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

Philosophy 2 Command versus Control

3.2 Design approach

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

3.3 Requirements generation and analysis

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

4.2 Formulation of philosophy 1

Philosophy 1 Mission Centricity

4.2.2 Justification for mission-centricity

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

4.3 Formulation of philosophy 2

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

Philosophy 2 Command versus Control

4.3.2 Commonality of commands; specificity of controls

Similar Goals Abstract

High level tasks.

Low level tasks.

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

4.3.3 Consistency with STANAG 4586

4.3.4 User feedback

HFIDTCPIII_T16_01 Version 3/ 15 June 2010

4.3.5 Illustrations of the issue

HFIDTCPIII_T16_01 Version 3/ 15 June 2010