Driver and Passenger Behaviour Monitoring

Identified critical AI components (e.g., face detection, face recognition, face-landmarks detection, eye-gaze, head-pose estimation, body-pose detection, object detection, age and gender classification) crucial for the solution integration. Adapted state-of-the-art AI components and tailored them for the execution on embedded boards. Performed benchmarks of the chosen components and optimized critical sections of their execution.

Designed and implemented the embedded solution based on a scalable architecture orchestrating the AI components in real-time. Combined the basic AI components within a multi-tasking framework to define the data-flow graph of the application. Implemented several SW layers assigned for video signal processing and reasoning. Fulfilled design requirements with respect to the processing latency and the number of processed frames per second.

Identified performance and accuracy drawbacks related to the specific field of algorithm application, not necessarily well-aligned with a generic field, main-stream models, and the training data they are based-on. Prepared and defined dozens of data-collection scenarios to address the original model drawbacks. Performed data collection with real in-car persons to enlarge available data-sets and improve the in-the-field performance.

Refined and retrained models to meet better stringent KPRs. Used quantization and optimization techniques to find sweet spots in the performance-cost curves. Guaranteed optimal execution on target platforms, using heterogeneous computing resources available (CPUs, GPUs, and dedicated neural network processing units, also known as deep-learning accelerators).

Developed supporting applications for facilitating data-collection and annotation. Implemented data-base back-end to store the collected data and annotations. Concocted methods for maintaining the quality of the collected data. Built mobile applications and the back-end responsible for on-boarding drivers/passengers, allowing users to create relations between their identities and facial fingerprints.