i. Abstract

The Incident Management Information Sharing (IMIS) Internet of Things (IoT) Pilot established the following objectives.

•   Apply Open Geospatial Consortium (OGC) principles and practices for collaborative development to existing standards and technology in order to prototype an IoT approach to sensor use for incident management.
•   Employ an agile methodology for collaborative development of system designs, specifications, software and hardware components of an IoT-inspired IMIS sensor capability.
•   Development of profiles and extensions of existing Sensor Web Enablement (SWE) and other distributed computing standards to provide a basis for future IMIS sensor and observation interoperability.
•   Prototype capabilities documented in engineering reports and demonstrated in a realistic incident management scenario.

These principles continued through the IoT Pilot Extension, with additional objectives of:

•   Integration into the existing Next Generation First Responder (NGFR) Apex development program process as part of Spiral 1;
•   Defining steps to begin the integration of existing incident management infrastructure, e.g., pulling in National Institute of Emergency Management (NIEM) message feeds; and
•   Demonstration and experimentation in a ‘realistic’ incident environment using two physically separate sites–an incident site within an active first responder training facility (Fairfax County Lorton site), and a command center (DHS S&T Vermont Avenue facility).

The initial Pilot activity has been documented in three OGC public engineering reports. The present report describes and documents the additional activities and innovations undertaken in the Extension.

ii. Business Value

The IMIS IoT Pilot Extension continued the work of the IMIS IoT Pilot to develop, test and demonstrate the use of networked sensor technologies in a realistic test scenario developed in collaboration with DHS and first responder stakeholders. The Pilot Extension demonstrated an IoT approach to sensor use for incident management. Prototype capabilities included ad hoc, nearly automatic deployment, discovery and remote access to sensor information feeds, as well as derivation of actionable information in common formats for use in Computer Aided Dispatch, Emergency Operations Centers, Geographic Information systems and mobile devices.

iii. Keywords

IMIS, IoT, OGC, Pilot, DHS, S-Hub, Sensor Things API, SOS, Datacasting

1       Introduction

1.1           Scope

This report describes the technical development, testing and demonstrations undertaken within the Department of Homeland Security (DHS) Internet of Things (IoT) Pilot Extension, integrated into the DHS Science and Technology Directorate (S&T) Next Generation First Responder (NGFR) APEX Program Spiral 1 development activity. As such, this document describes follow-on activities to the IMIS IoT Pilot which is itself covered by three earlier Engineering Reports (See  Section 2 References).

The extension activities were carried out in close collaboration with the NGFR program and affiliated performers, and this experiment was part of Spiral 1 of that program. The extension work consisted of further development of IoT sensing and observation exploitation capabilities, integration with capabilities developed by the other NGFR performers, and demonstration of a new incident scenario involving an explosion and building collapse.

1.2           Document contributor contact points

All questions regarding this document should be directed to the editor or the contributors:

1.3           Revision history

1.4           Future work

A description of identified areas requiring future work are provided in Section 8.

1.5           Forward

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium (OGC) shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

2       References

The following documents are referenced in this document.

1. OGC 15-118 IMIS Profile Recommendations for OGC Web Services
2. OGC 16-093 IMIS IoT Protocol Mapping Engineering Report (ER)
3. OGC 16-014r1 IMIS IoT Pilot Architecture ER
4. OGC 16-121 SWE-IoT Reference Model
5. OGC 06-121r3 Web Services Common Specification
6. 16-121 IMIS IOT Reference Architecture Draft (In preparation).

3       Terms and Definitions

The following primary terms and acronyms are used in this document:

Internet of Things (IoT)     

Methodology for exploiting sensor information via the Internet and Web.

Catalogue Web Map Service (CatWMS)

An OGC Compliant Web Map Service rendering maps of catalogued sensors within the IMIS IoT Pilot.  (Source: IMIS IoT Pilot)

Catalogue Service for the Web (CSW)      

OGC Catalogue Service Specification.   (Source: http://www.opengeospatial.org/standards/csw)

Datacasting     

Transmission of data via broadcast channels (e.g., television).    

Department of Homeland Security (DHS)

U.S. Government institution responsible for the first responder.

First Responder Extensible Sensor Hub (FRESH)         

A capability integrating standard first responder information encoded as NIEM into a web service environment. (Source: DHS)

HAZMAT   

Hazardous Material.  

Hub Catalogue (HubCat)

Sensor Hub Catalogue used within IMIS IoT to catalogue sensor hubs available in a given operation. (Source: IMIS IoT Pilot)

Incident Management Information Sharing (IMIS)     

Technology pilot undertaken by the OGC on behalf of the DHS. (Source: http://www.opengeospatial.org/pressroom/pressreleases/2211)

Inertial Measurement Unit      (IMU)

 Sensor used to measure inertial changes and determine direction of movement.

Message Queue Telemetry Transport (MQTT) 

Machine-to-machine (M2M)/IoT connectivity protocol. (Source: MQTT.orghttp://mqtt.org/)

National Information Exchange Model (NIEM)

Framework, data model and set of XML Schema documents for information message exchanges.

Next Generation First Responder (NGFR)

DHS research program looking at first responder capability.

Public Safety Cloud (PSCloud)

A private computing and storage cloud supporting public safety. (Source: DHS)

Pan-Tilt-Zoom (PTZ) Camera

A camera, often in a fixed location which supports remote control of Pan Tilt Zoom.

(Source: Well known concept)

Sensor Hub (S-Hub)

A Sensor Observation Services (SOS or STA compliant Web Server that publishes data from locally networked / connected sensors. (Source: IMIS IoT Pilot)

Scanning Long-Term Evolution (LTE) Emission Detector (SLED)   

UAV-mounted cellphone activity sensor.

Sensor Observation Service (SOS)

An OGC standard for delivering sensor information. (Source: OGC http://www.opengeospatial.org/standards/sos)

Sensor Planning Service (SPS)

An OGC standard used to task sensors. (Source: OGC http://www.opengeospatial.org/standards/sps)

SensorThings API (STA)

OGC standard for delivering sensor information. (Source: OGC)

Unmanned Aerial System (UAS)

Unmanned drone used to support camera or other sensor. (Source: UAVS.org https://www.uavs.org/index.php?page=what_is)

Web Map Service (WMS)

Web service delivering rendered map portrayals. (Source: http://www.opengeospatial.org/standards/wms)

Web Registry Processing Service (WRPS)

Web service harvesting content from other service capabilities documents and populating a Catalogue Service for the Web (CSW) Electronic Business Registry Information Model (CSW-ebRIM) compliant registry service. (Source: IMIS IoT Pilot)

Web Feature Service (WFS)

Web service delivering geographic features. (Source: OGC)

4       Conventions

4.1           UML notation

Most diagrams that appear in this standard are presented using the Unified Modeling Language (UML) static structure diagram, as described in Subclause 5.2 of OGC 06-121r3.

5       Experimentation and Demonstration

5.1           Context

The initial IMIS IoT Experiment/Demonstration focused on an outdoor incident around a road traffic accident with consequential damage to infrastructure, chemical release, etc. This involved the significant use of street-scale geospatial data and mapping. Responders to the incident dealt with the consequences of traffic chaos, powerline damage and chemical dispersal over a wide scale.

The IoT Pilot Extension scenario was based on the response to a collapsed building event. This had a much narrower geographic spread, with the scenario nominally limited to an area within the Fairfax County emergency training site at Lorton, Virginia, and focused on responders evaluating the exterior and interior of the collapsed building.

5.1.1      Goals of the Trials/Demonstration

The demonstration had three primary goals.

1. To show that information from sensors connected via a range of standards and interfaces can be integrated and shared through common interfaces. The experiment demonstrated how the information produced from existing and new sensor sources can be federated and processed via a common information infrastructure and made available to all users across a variety of applications and platforms.
2. To show that this could be achieved in a ‘realistic’ environment, i.e., an event in the field, by deploying a scene-specific Field Sensor Hub (S-Hub) to serve as the main communication, tasking and data management hub for all on-scene sensors.
3. To provide connectivity and selective real-time streaming of sensor data to a central command center via an S-Hub deployed in the Public Safety Cloud (PSCloud), and to show how a range of information could be accessed broadly and efficiently in this manner.

5.2           Environment

The scenario was implemented across two sites, with participants deployed at the Lorton site simulating the incident occurrence and response, while those stationed at the DHS Vermont Avenue site (the location of the actual demonstration) simulated the incident command center. The latter site doubled as a secondary incident location for sensors in some of the experiments. The two sites were connected using a number of voice, video and data communications paths, none of which depended on existing DHS communications infrastructure.

Communication at the Lorton Facility consisted of a local Wi-Fi network and cellular LTE gateway established with a number of ‘Field S-Hubs’ through which local sensors received commands and transmitted observations. These hubs were implementations of the S-Hub concept developed for the IMIS IoT Pilot / NGFR Spiral 0 activity. In addition, ‘Cloud S-Hubs’ were established and deployed in the DHS PSCloud environment to implement the S-Hub concept. Cloud S-Hubs served to cache or mediate information from Field Hubs to lighten the load on responder worn or deployed devices. In addition to Wi-Fi and cellular LTE, a Datacasting capability was used to demonstrate additional communications approaches. Over the various communication channels, information such as location/orientation, responder heartrate, measured gas concentrations and high-definition (HD) video was transmitted from the Lorton site to the simulated command center at Vermont Avenue.  

5.3           Demonstration Scenario

5.3.1      Overall Scenario

An incident is reported at the Lorton site and logged by dispatchers at the control center. An explosion has resulted in the collapse of a building. There may be trapped / or injured individuals, as well as exposure to hazardous materials and vapors.

5.3.2      Pre-arrival Preparation

As part of logging the event, the incident dispatcher entry automatically establishes an event ‘website,’ which is the focus of information related to the incident. This allows all information and sensors available to be identified in a catalogue for use by responders and incident coordinators. Certain standard information views can be created automatically. 

5.3.3      Dispatching Resources

Responders are dispatched, an S-Hub is established at the incident scene, and unmanned aerial systems (UAS) or drones are deployed by responders to reconnoiter the affected building and surroundings. This gives command center operators an overall view of the situation. The site is deemed stable enough to work in, although there are concerns around possible gas leaks.

5.3.4      Preparation for Entry

First responders, as part of standard kit, are equipped with video cameras, external environment sensors (temperature, gas, chemical, etc.), and wearable physiological sensors measuring heart rate, breathing rate, etc. The responder can also receive or initiate alerts based on their own kit readings or other information. To interfere as little as possible with a responder’s tasks, each wears a smartwatch on which they can receive alerts. Each responder kit connects automatically to the incident sensor network upon activation to send and receive pertinent information.

Indoor drones also plug-and-play with respect to the incident sensor network, and provide a view ahead of the responders into the collapsed building space. These provide live video feeds include and may also include onboard environmental sensors. In addition to their smartwatches, responders carry smartphones and/or tablets on which they can view the drone video.

5.3.5      Drones and Responders Progress into the Building

Drones enter the building followed by responders. The drone video can be monitored at the command center and allow the responders to be directed to specific areas. While the drones are equipped for autonomous operation, manual control is used due to the difficult interior conditions of the site (discussed later). 

5.3.6      Responders and Sensor Systems Collaborate in the Building

A key goal of this scenario and the initial Pilot activity has been to demonstrate that every sensor, whether on drones, on responders, deployed in situ, model-derived or “humans-as-sensors,” can contribute to a globally accessible pool of incident information. Sensors’ observations are transmitted via field S-Hubs available at the incident scene to the field incident command post, to the Command Center and on to other participating organizations. Reliable field communications is always an issue, however, and so there is always a need to work around bandwidth or connectivity issues to ensure that sensor observations are preserved both locally and globally. During an incident response, therefore, field S-Hubs cache and serve sensor data on-scene and upload the data whenever possible to cloud-based S-Hubs. Responders then access the cloud whenever possible to minimize bandwidth demands on the field sensor network. Other innovative communications techniques (e.g., Datacasting) are also employed to ensure the best possible information availability during the response.

5.3.7      On-going Background Monitoring

Sensor information is automatically processed into a number of formats to ensure it can be exploited easily and to make critical information easier to recognize. Standardized observation data formats, in particular, allow application of geospatial analysis techniques, such as geofencing, that can alert busy responders and commanders to dangerous situations.

5.3.8      Hazardous Gas Fumes Alarm Propagation

First responders entering the collapsed building carry wireless air quality sensors that provide the responder and the sensor network with readings on hazards such as natural gas fumes. During investigation of the collapsed building, one of these sensors registers an increase in the level of gas. As the concentration level hits a configured threshold value, an alert is generated by the corresponding S-Hub and a notification is sent to all responders’ smartwatches and the command center. The responders are directed to withdraw immediately until the gas level issue is rectified, but leave the gas sensor within the building to provide confirmation of safe conditions once the gas supply to the building can be cut off. While this particular notification is generated automatically based on sensor readings, an emergency notification can also be broadcast by any responder who separately discerns a hazard or safety issue.

5.3.9      Command Center Support to Responders

The command center monitors a range of information relating to the event as it unfolds. The command center has two primary means of visual engagement.

•   The overall situational display (display wall). This visually conveys the overall situation, as well as specific critical or actionable information.

 

•   Individual workstation displays. These are allocated to specific functions and manned by specialists dealing with an element of the response, for example overall coordination, utilities analysis, victim triage, etc.

Displays provide views of a number of types of information.

●      General location/numbers of responders (implemented for outside locations) and their health state. Triggers indicate surpassing safe threshold levels of heart-rate, breathing rate or other health indicator in any responder.

•   Environmental status of the site (structural stability, air quality, electrical hazard, etc.).
•   Situation overview as video or still imagery (UAS, responder bodycam, PTZ camera, etc.).
•   Existing framework data, maps and plans of site, critical site features such as power locations, entry and exit routes, etc.

6       Architecture

6.1           Overall Architecture

The overall component interaction architecture of the IoT Extension is shown below.

The Pilot Extension sensor network consisted of components of a number of different types, in most cases provided by more than one vendor. The key component types are:

•   Catalogues (provided by Compusult and Envitia);
•   Datacasting (provided by SpectraRep);
•   Field S-Hubs (provided by Botts Innovative, Compusult and Sensor Up);
•   Cloud S-Hubs (provided by Botts Innovative, Compusult and Sensor Up);
•   Common Operating Picture Clients (provided by Compusult, Envitia);
•   Mobile/Wrist Based Clients (provided by Sensor Up and Noblis);
•   Sensor devices (provided by Botts Innovative, Compusult and Sensor Up);
•   WiFi LAN + LTE WAN @ VTA (provided by Compusult & Tumbling Walls);
• Wireless LAN – Lorton (Provided by Botts Innovative (T-Mobile), Noblis (Verizon, T-Mobile)); and
•  Routers for emergency services tracking (via NIEM) (provided by Ardent).

7       Technology Advances

7.1           Sensor Hubs in the PSCloud

One goal of this Pilot has been to establish, test, and demonstrate the ability and benefits of establishing S-Hubs within a cloud environment. For NGFR, the PSCloud was set up using Amazon Web Services. S-Hubs deployed within the PSCloud environment served as a globally accessible source of both real-time and archived observations during response and for post-emergency evaluation.

For the Pilot Extension, several S-Hubs were deployed on the PSCloud, each implementing one or more SWE and IoT standards. Since most interactions with cloud S-Hubs were through transactional services, basic OGC services’ capabilities without specific local sensor drivers were sufficient.

For those cloud S-Hubs supporting Transactional Sensor Observation Service services (SOS-T), sensors can be added using a registerSensor() request. Real-time observations can then be streamed from the newly registered sensor and stored on a cloud S-Hub. Observation data persistence is an optional capability in Open Sensor Hub (OSH) implementations of S-Hub. These observations are then available in real-time or as archived data to any SOS client. Cloud S-Hubs were provided by three different participants: Botts Innovative, Compusult and Sensor Up.

7.1.1      Cloud Hub 1 – Field Sensor Caching

The sensor observations that were uploaded to, and then accessed from, the Botts Innovative OSH-based cloud S-Hub included:

• Video and location data from two Garmin VIRB XEs;[1]
• Chest-strap heartbeat data also from the Garmin VIRB XEs;
• Video and location/orientation data from four Raspberry Pi [2]GeoCams;
• Video and location/orientation data from an Android phone and tablet;
• Remote locations from a TruPulse Laser Rangefinder;
• Video and location/orientation data from an Android-attached FLIR infra-red camera;
• Health data from an Angel Sensor wristband;
• Video and PTZ data from a Dahua HD video camera; and
• Video and location/orientation data from a 3DR Solo quadcopter. 

Several of these video sources were successfully accessed in real-time by the SpectraRep datacasting team and incorporated into the PBS broadcast pipeline. This capability is described under Section 7.10.1 (communications infrastructure).

7.2           Cloud Hub 2 – Interfacing to DHS Message Feeds

One of the goals of the Pilot Extension was to integrate with the First Responder Extensible Sensor Hub (FRESH) Router implemented by Ardent, a message router service deployed in PSCloud to share first Responder and emergency management data as NIEM messages. The FRESH router provides a simple RESTful interface that uses the OASIS EDXL Distribution Element (DE) to handle messages. The following operations are available in the interface:

The FRESH Router provides two ways to receive DE Messages:

• RESTful Interface
  • Using the interface provided above, users can query for the DE Messages.
• Message Federation
  • Endpoints can be manually configured to have DE Messages federated to them.

The FRESH router is not directly compliant with OGC SWE standards (SOS/STA), so it was connected via a custom adapter to a cloud S-Hub provided by Compusult. This driver implements the FRESH RESTful interface because there is no current means to configure the router dynamically for message federation. S-Hubs are activated and de-activated dynamically, so it would not have been practical to manually configure the router endpoint each time an S-Hub was activated. The RESTful interface also does not currently provide any way to perform custom queries, so the S-Hub would query the FRESH router on a schedule, and then compare the current result to the previous to identify any changes. This process unavoidably introduces a delay equal to the polling schedule and will definitely not scale well.

The S-Hub driver connected to the FRESH router also converts the data it receives so that it can be published as a SensorThings API (STA) service. The FRESH router provides both Cursor on Target (CoT) and NIEM messages, so the STA service provides a separate data-stream for each message type. The driver is able to pull out the date and location from each message type and map them directly into the STA response. There is no clear mapping currently established for the rest of the COT or NIEM message content, so the entire XML-formatted message is actually returned in the observation result. This approach is not compatible with Sensor Web clients per se, but would allow any clients that supported both STA and these message formats to digest them.

7.3           Sensor Hub as a Local Field Node

7.3.1      Lorton Deployed Sensor Hubs

Another goal of the Pilot Extension was to establish and test one or more field S-Hubs that anchor to local sensor networks at the incident scene. Field S-Hubs serve two main functions: persist and provide observations from sensors deployed in the field for the on-scene responders, and upload those same observations to cloud S-Hubs for access by the Command Center whenever there is adequate network connectivity from the field to do so.

For the Pilot Extension, an OSH instance was deployed on a laptop to serve as a field S-Hub and associated local Wi-Fi access point supporting the ingestion of and access to sensor observations. The field S-Hub also supported sensor tasking. While the deployed S-Hub used a rather ad hoc arrangement of cellular modems for WAN connectivity, future field S-Hubs will likely include robust multiplex remote connectivity capabilities, as well as a larger range of local area protocols (Wi-Fi, BlueTooth LE, Z-Wave, LTE, ZigBee, LinkITOne, etc.).

The local OSH-based field S-Hub successfully supported a range of sensors, including Garmin VIRB cameras, GeoCams, health monitors, Android phones, laser rangefinders, FLIR cameras and camera-equipped UAVs. Much of the data from these sensors were also uploaded to cloud S-Hubs from the incident scene via cellular LTE connections (phones and modems), including several video streams.

The biggest challenges with field S-Hub deployment involved network communications, both locally in the field and from the field to the PSCloud. Challenges included:

•   Providing adequate range of Wi-Fi to sensors in the field;
•   Powering local networks in the field away from power connections;
•   Maintaining adequate LTE data bandwidth and throughput, particularly in competition with nearby highway usage that varied enormously with time of day; and
•   Cellular data plans suited to response operations.

Other challenges involved maintaining enough battery capacity to support the different sensors in the field, as in adequately sized batteries, sufficient spare batteries and an effective means of recharging batteries as needed. The deployment of the OSH-based field S-Hub itself was successful. Almost all system challenges centered around the communications and power issues discussed above.

7.3.2      Vermont Avenue Deployed Sensor Hubs

Although the deployment of sensors and S-Hubs at the Command Center lacked some degrees of realism, it did provide a way to show how sensors at multiple locations could deliver information into the entire network and deliver information to all responders. This also allowed a more vivid illustration of gas detection and alert generation (by way of a butane lighter) for the demonstration attendees, really showing the sensor functionality. Compusult and Sensor Up each deployed sensors at the Vermont facility, as well as the ability to deliver an alert simultaneously to Command Center personnel and responders at the Lorton site.

7.4           New Sensors and Platforms

7.4.1      IMIS IoT Initial Pilot

During the initial IMIS IoT Pilot activity, a wide range of sensors were integrated. These included:

•   Fixed infrastructure sensors, weather stations, building alarm sensors, etc.;
•   Unmanned Aerial Systems (UAS) with cameras;
•   Real-time streaming video cameras (drop cams, body cams, etc.);
•   Laser rangefinder (tagging remote locations);
•   Plume models; and
•   Responder deployed sensors (biometrics).

7.4.2      Enhanced Sensors for IOT Extension

For the Pilot Extension, the same set of sensor information types was gathered (with notable additions) with more integrated sets of sensors being tested, for example, a set of sensors integrated into a bodycam for a responder to wear. Additional video sensors were introduced, creating in some cases bandwidth issues from high-definition video stream. Table 1 shows the sensors deployed and/or tested in the Pilot Extension. In many cases, similar products and capabilities were provided by different participant companies; the overlap of capabilities allowed both testing of sensor interchangeability and opportunities to pool and compare experiences.

Environment

Physiology

Visual

Location and Other

7.4.2.1     Garmin VIRB (Botts Innovative, Compusult, Sensor Up)

The Garmin VIRB camera device comprises an off-the-shelf video camera that includes GPS location and acceleration, but not geo-orientation. The VIRB also supports some ANT+ sensors, such as heart rate monitors and external temperature sensors. This device has primarily been designed to record HD video onto a local SD card, but does support a real-time stream of lower-resolution video and still imagery.

The VIRB supports a wide variety of accessories for attaching the camera to people and things. Real-time streaming data is delivered through a Wi-Fi network connection using a JSON-based protocol. This protocol is proprietary and requires its own S-Hub software driver to make the data accessible through SWE standard interfaces. 

VIRB units were supported in this Pilot by Botts Innovative,  Compusult and SensorUp S-Hubs. SensorUp’s implementation (tutorial and source code) can be viewed at https://bitbucket.org/geosensorweblab/sta-virb-xe. Garmin also provides a VIRB network services API and the Compusult S-Hub provided support for this.

A number of issues were identified with VIRB.

•   No geomagnetic sensors to provide view compass direction.
•   Temporal resolution of navigation data is not designed for real-time application, so it is not suitable to support geolocation of imagery from a moving platform.
•   Currently, there is no way to synchronize sensor values with the frames in the live stream video. Garmin was contacted and they stated: “Each piece of data is made available as quickly as it can be. Sensors are almost instantaneous, but the video encoding takes time, as does the transfer of the video stream. Since network timing can be so unreliable and the data sources are completely independent, we would most likely have to embed the sensor data or timing information into the video stream. While technically possible, the VIRB firmware doesn’t have this capability. For now, we don’t have any plans to add additional metadata for synchronization of live data and video.”
•   Only one client can be connected to the live video stream; therefore, an S-Hub is required to make the video available to more than one viewer.

7.4.2.2     Grove Sensors

Grove is a sensor hardware connector designed for fast prototyping, so that no soldering nor breadboard configuration is needed for hardware prototyping. Seeedstudio, the Grove provider, offers sample code for each module, as well as detailed specifications and implementation guides. Hundreds of sensors are currently available. A Grove S-Hub driver was written by Compusult to allow for the plug and play autoconfiguration of supported sensors. The driver supports all of the Grove sensors in the table above.

The following issues were identified with the Grove Sensors.

• They are not inherently plug and play, so the driver needs to manually match up the sensor to the port.
• Some sensors randomly cause the base shield firmware to crash. New sensor data is then unavailable until reboot.
• Each new sensor type requires a driver update to maintain the effective “plug and play” capability.
• Some sensors return raw “analog” values instead of specific measurements. For example, the gas sensor just indicates a conductance value that does not directly correspond to a concentration in the air and needs to be processed to provide actionable information.

7.4.2.3     Z-Wave Sensors

Z-Wave is a wireless communication protocol designed for home automation. It uses a source-routed mesh network architecture and supports secure communications. An S-Hub driver was written to work with RaZberry, an add on for Raspberry Pi that turns it into a fully operational Z-Wave gateway.

Rasberry Pi firmware/software provides a way to retrieve in JSON either the entire Z-Wave stack, or changes to the stack since a moment in time, using an HTTP GET request. The S-Hub driver was written to parse the Z-Wave stack, process any changes, then update its sensor and observation data accordingly.

The following issues were identified with the Z-Wave sensors.

• The Z-Wave protocol is well defined; however, there still exist different interpretations of the protocol, so it takes some driver mediation magic to work with similar sensors from different vendors.
• The default configuration for most sensors is to conserve battery life by updating sensor readings infrequently; however, this can be adjusted for most sensors as needed.

7.4.2.4     Android Phone as a sensor platform

Android phones or tablets provide a range of valuable sensors on a computing platform that provides its own connectivity to a local or global network through Bluetooth, Wi-Fi and LTE connectivity. They can also be used, conveniently, for voice and text communications.

Sensors normally included in Android phones or tablets include: HD video/imagery camera, GPS and geo-orientation (acceleration and geomagnetic). The Android platform can support other sensors through Bluetooth, USB or Wi-Fi connections (e.g., laser rangefinder, IR camera, portable weather station, health monitors, etc.). Android platforms natively support Java applications and thus support software such as Botts Innovative Open Sensor Hub onboard.

There were no significant issues with using Android-based sensors, except that phone processing capabilities may get overwhelmed with supporting a large variety of sensing tasks. It may be perfectly appropriate to work with multiple Android devices to assure capacity and reliability.

7.4.2.5     Raspberry Pi “GeoCam”

The importance of being able to geolocate and georectify video and still imagery is becoming more and more recognized. It is important not only to know where the camera is, but also to recognize what direction it is looking and what actual location the camera is imaging. The Raspberry Pi (RPi) provides an excellent platform for building both prototype and operational sensor systems, including geospatially aware HD video cameras.

As part of the Pilot Extension, Botts Innovative provided four “GeoCams” that include an on-board OSH S-Hub and provide HD video, GPS location and geospatial orientation sensor outputs in a compact package that can run on attached batteries for up to 18 hours. They include a screw attachment for GoPro and Garmin mounting accessories. GeoCams function as independent (local) S-Hubs and can also stream observations using SOS-T through a Wi-Fi or ethernet connection to a field or cloud S-Hub.

The GeoCams are configured to run the OSH software automatically and register with other S-Hubs whenever they are turned on. The present GeoCams are prototypes and assembled by hand. They can also be modified individually at this point for particular needs with additional sensors or LTE communication through an onboad cellular modem (e.g., FONA). They may serve as model designs for future COTS products.

7.4.2.6     Angel Sensor Wristband

The Angel Wristband is designed to monitor the wearer’s heart-rate, body temperature, movement and oxygen concentration. The device communicates with other devices through Bluetooth. The Angel Wristbands deployed  in the Pilot Extension were integrated with OSH-based S-Hubs. There were substantial issues with this device in terms of sensor consistency. The measurements from the sensor were sporadic at best with heartbeat only being monitored when the person/patient was perfectly still. The O₂ sensors never reported any values to the S-Hub or through the Angel Sensor API.

7.4.2.7     Apple Watch and Android Watch

The Apple Watch does carry health sensors, but in the Pilot Extension was used as an output device. The iPhone / Apple Watch sensor alert app developed by Noblis acts as a STA client, connecting to a STA service to monitor sensor readings for alert conditions. The current app added the capability to select between different S-Hubs, including Botts cloud S-Hub, SensorUp field S-Hub and Compusult field S-Hub)

Samsung Watch / Android Wear devices were used as STA / MQTT clients and as heartrate monitors, connecting to SensorUp S-Hubs.

7.4.2.8     3DR Solo Drone Video 

The 3DR Solo quadcopter system provides an open hardware / software platform for supporting additional functionality. For the Pilot, software modifications were made to the Solo operating system to transmit gimbal settings, GPS location, geospatial orientation and HD video via Wi-Fi into an OSH-based S-Hub ground station. The streaming imagery could then be geo-rectified on demand allowing one to view the current camera view, as well as its map footprint within a Cesium3D-based browser client. The client application also included a window with traditional aircraft-type controls for altitude, attitude, direction and speed. Real-time geo-rectification of UAS video and still imagery greatly increases the value of UAV observations for providing situation awareness remotely, as well as connecting observations to other georeferenced information.

Issues with Solo Drone included the following.

• The 3DR Solo is designed principally to fly outdoors under GPS control. It provides a wealth of functionality for stabilized, constrained and even autonomous flight plans as long as a GPS fix is available. It does not, however, have an alternate means of indoor stabilization or navigation (e.g., optical flow sensors). Minus a GPS link and inside a building with an abundance of wind crosscurrents from broken windows (the case at Lorton), it proved extremely challenging to control. It is likely that reliable UAS operations would best be served by different craft that are specialized for indoor and outdoor conditions.
• The FAA has been evaluating regulations for commercial use of drone aircraft within the U.S. Understanding and obeying the rules of piloting drones for anything other than recreational use has been extremely difficult. This ambiguity made outdoor flights at the Lorton Facility problematic. The FAA has recently released new guidelines/rules for drone flight, which are set to take effect in August 2016. These new rulings should make the licensing requirements for recreational, commercial and research drone flight both simpler and more transparent. Combined with a now smaller restricted DC air space, this should allow outdoor flight at Lorton in the future and increase the opportunities to demonstrate the value of UAS operations in response activities.

7.5           Sensor Plug and Play

Sensor Plug and Play is achieved in the S-Hub architecture via the use of drivers. The purpose of the driver is to communicate with each sensor’s often local and/or proprietary protocols so as to make its data available using open SWE and IoT standards.

In each of the S-Hubs used in the Pilot activities, drivers were set up before deployment, or uploaded via an administrative interface to allow for the addition of new sensors and sensor types. Each S-Hub developer provided some sort of administrative interface to configure sensors.

The typical process for adding a sensor to an S-Hub is shown below:

Of course, if a particular sensor technology or implementation (as sensor interfaces are typically vendor specific) is not yet supported, it is typically necessary to develop a new driver. Once that driver is built and configured, it is a simple task to add a new sensor (either via simple configuration files or through an easy-to-use Graphical User Interface (GUI).

7.5.1.1     Configuring Sensors

The following image shows an example listing of configured drivers in the Compusult S-Hub.

Drivers can also include interface pages that display the configuration settings for the current driver; however, this is only required if the sensors are not built on a protocol that allows for true plug and play operation. A driver that adds support for Z-Wave sensors may not require a configuration page because the device will pair and automatically appear upon startup. A driver that adds support for Grove sensors, however, will require a configuration page because the Grove specification does not provide for automatic detection of new sensors. The following is an example configuration page for Grove sensors in the Compusult S-Hub.