Plexus OS is our ontological inference process that combines AI based sensor data & visual relationship detection (VRD) for threat detection and onward action in both military and civilian settings.
Plexus ontology defines the relationship between various objects in both military and civilian settings, operational conditions (RoE etc) & expected situation on description logic (DL).
Plexus VRD
Visual relationship detection methods can detect multiple interactions between objects and offer a comprehensive scene understanding of onsite images. Plexus VRD Is the first end-to-end relation detection network that can detect objects and relations simultaneously.
Plexus VRD involves two parts: an object detection module and a relation module. The Plexus VRD network first builds an object detection module which is a convolutional localization network and then builds a relation module that integrates feature extraction and visual translation embedding. An image is input into the object detection module and a group of detected objects is the output. Then, objects are fed into the relation module. In the end, the detected images with objects and the relationships between objects in the form of the subject–predicate–object triplet will be output.
The Plexus VRD method refers to the visual relationship as the subject–predicate–object triplet, where the predicate can be a verb, spatial (under), preposition (with), and comparative (higher). Inspired by Translation Embedding (TransE) Plexus VRD maps the features of objects and predicates in a low-dimensional space, in which the relation triplet is explained as a vector translation, e.g., person + wear ≈ helmet. For knowledge transfer in relation, Plexus VRD uses a unique feature extraction layer that extracts three kinds of object features: classeme (i.e., class probabilities), locations (i.e., bounding boxes coordinates and scales), and RoI visual features
Plexus Ontology
Plexus Ontology are built from the ground up by Roark using the following methodology:
- Determine the domain and scope.
- Enumerate terms that are most commonly used in the specific domains (military, civiliant etc.
- Categorize these terms and their class hierarchy relationships.
- Define the properties of classes and the facets of the slots.
- Create instances.
The ontology can address the semantic gap between visual information extracted from the images by computer vision methods and the textual information in safety rules. For example, the entity detected in a military setting by the deep learning object model is “drone, Russian Mine” while what is identified by humans from the ground and found in RoEs is “enemy drone”. Another example in a civilian setting is- entity = “human,knife,street” while what is identifed and found in the law is “offensive weapon”.
Thus, the representation of entities at the military/civilian setting is the domain of the ontology. The concepts describing different entities and their hierarchical relationships will figure into the ontology.
The terms or the entities are the particular entities in the regulations which can be a “thing” (e.g., equipment, building structure) or “personnel” (e.g., soldier). The terms emphasize the objects including building material, civilian machinery, combat machinery, clothing, weapon systems, transport vehicle, and civilians. Then, a top-down approach is adopted to start from the top concepts and gradually refine them.
These definitions are what power the Plexus Decision Support Systems which can be fully or part automated.
To improve operational efficiency and decision making awareness, Plexus is deployed both at the edge and on deployment specific MCP relational cloud servers.
