1 前言

  在Ubuntu18.04实现的一个人眼监测小程序，使用Qt5.14、Dlib19.24、Opencv3.4.16实现。
  其主要实现思想是，首先通过Opencv获取摄像头数据,然后通过Dlib提取人脸68关键点的算法对所输入图片中的人脸进行关键点检测，检测出眼部的关键点后,在用Opnecv中的画图命令进行绘制，再利用Qt进行界面展示。

2 效果

  人眼监测小程序效果如下图所示，将画面中的人眼处用绿色的线进行框选。

3 Ubuntu18.04下Qt、Opencv、Dlib的配置

3.0 Ubuntu18.04的安装以及一些基本配置

  U此部分涉及到buntu18.04系统的安装、配置一些基础的应用软件等过程。
  (1)引导盘的制作;(2)安装ubuntu18.04系统;(3)安装搜狗输入法;(4)安装百度网盘;(5)安装Chrome浏览器;(6)安装激活Pycharm;(7)安装VScode;
  整体的大概流程参照这篇博客

3.1 Qt

  从Qt官网下载链接https://download.qt.io/archive/qt/处下载所对应版本,我这里直接下载的是https://download.qt.io/archive/qt/5.14/5.14.2/这个页面下的这个qt-opensource-linux-x64-5.14.2.run文件.下载下来后直接命令行运行,按照步骤走即可,注意一下安装Qt的具体位置.

3.2 Opencv

  从github上下载Opencv的源码进行编译,我这次没下新版本,因为之前有用过这个Opnecv3.4.16,所以就下了这个https://github.com/opencv/opencv/tree/3.4.16版本,没有什么特别的用意,如果只是做一些小demo,仅仅用到一些opencv的基础操作如读取图片等操作,下哪个版本的都可以.
  在用到cmake-gui配置编译选项的时候,勾选编译opencv_world,这样的话所有的库文件全部打包成一个,可以简便pro文件的写法,也不用去查某个函数到底对应哪个库,这个libopencv_world.so.3.4.16也就20多MB吧.(此处是相对而言的,也可以不编译为一个库)

3.3 Dlib

  从github上下载Dlib的源码https://github.com/davisking/dlib进行编译,过程与编译opencv时类似.

4 核心代码

  本项目下的文件层级如下图所示.

4.1 pro文件

QT       += core guigreaterThan(QT_MAJOR_VERSION, 4): QT += widgetsCONFIG += c++14# You can make your code fail to compile if it uses deprecated APIs.
# In order to do so, uncomment the following line.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0SOURCES += \\camera.cpp \\fatiguedetect.cpp \\main.cpp \\widget.cppHEADERS += \\camera.h \\fatiguedetect.h \\widget.hFORMS += \\widget.ui#opencv库的路径opencv3.4.16
INCLUDEPATH += /home/ai/software/opencv3416_installed/include
LIBS += /home/ai/software/opencv3416_installed/lib/libopencv_world.so# dlib19.24库的路径
INCLUDEPATH += /home/ai/software/dlib_installed/include
LIBS += /home/ai/software/dlib_installed/lib/libdlib.a# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += targetDISTFILES +=

4.2 Widget.cpp

#include "widget.h"
#include "ui_widget.h"
#include <iostream>Widget::Widget(QWidget *parent): QWidget(parent), ui(new Ui::Widget)
{ui->setupUi(this);camera_thread = new Camera();camera_thread->set_cam_number(0);connect(camera_thread, SIGNAL(sendPicture(QImage)),this,SLOT(receive(QImage)));camera_thread->open_camera();
}Widget::~Widget()
{delete ui;
}void Widget::receive(QImage img)
{QImage scaled_img = img.scaled(ui->label->width(), ui->label->height());ui->label->setPixmap(QPixmap::fromImage(scaled_img));
}

4.3 fatiguedetect.cpp

#include "fatiguedetect.h"// 常量定义
const double close_standard = 0.45;
const int eye_l_len = 60;// 判断状态的类
class JudgeCondition : public QObject {Q_OBJECTpublic:explicit JudgeCondition(QObject *parent = nullptr) : QObject(parent), condition(0), heavy_flag(0) {};void judge(std::deque<int> &left_eye_l, std::deque<int> &right_eye_l){int left_sum = std::accumulate(left_eye_l.begin(), left_eye_l.end(), 0);int right_sum = std::accumulate(right_eye_l.begin(), right_eye_l.end(), 0);if (left_sum == eye_l_len && right_sum == eye_l_len) {++heavy_flag;if (heavy_flag > 100) {condition = 3;} else {condition = 2;}} else {heavy_flag = 0;condition = 1;}emit sendCondition(condition);};signals:void sendCondition(int);private:int condition;  // 状态：0=图像缺失 1=清醒 2=疲劳 3=重度疲劳int heavy_flag;  // 重度疲劳标志位
};FatigueDetect::FatigueDetect(QObject *parent) : QObject(parent),noImageFlag(0),noImage(0)
{cout << "[INFO] loading facial landmark predictor..." << endl;dlib::deserialize("./model.dat") >> predictor;          //是直接调试运行,则在build文件夹内;如果为release则与.exe在同一文件夹内detector = dlib::get_frontal_face_detector();
}cv::Mat FatigueDetect::detect(cv::Mat frame)
{std::vector<std::vector<cv::Point>> pos;cv::Mat resized_frame;cv::resize(frame, resized_frame, cv::Size(720, 720 * frame.rows / frame.cols));cv::Mat gray;
//    cv::cvtColor(resized_frame, gray, cv::COLOR_BGR2GRAY);cv_image<bgr_pixel> dlib_img(resized_frame);std::vector<dlib::rectangle> rects = detector(dlib_img, 0);    //这里放入detector的必须是dlib类型参数，不能是cv::Mat，不报错，但编译时出“问题”if (rects.size() == 0){noImage = 1;emit noSignal(noImageFlag);}for (dlib::rectangle rect : rects)    //for(类型 变量名称 : 容器/数组等内容){noImage = 0;dlib::full_object_detection shape = predictor(dlib_img, rect);std::vector<cv::Point> pos_;for (int i = 36; i <= 41; i++)   //左眼关键点标识{cv::Point point(shape.part(i).x(), shape.part(i).y());pos_.push_back(point);}for (int i = 42; i <= 47; i++)   //右眼关键点标识{cv::Point point(shape.part(i).x(), shape.part(i).y());pos_.push_back(point);}pos.push_back(pos_);draw_point_and_line(pos_, resized_frame);}cv::Mat resized_back_frame;cv::resize(resized_frame, resized_back_frame, cv::Size(frame.cols, frame.rows));return resized_back_frame;
}void FatigueDetect::draw_point_and_line(std::vector<cv::Point> pos_, cv::Mat &image)
{int linewidth = 1;for (int i = 0; i < 5; i++){cv::line(image, pos_[i], pos_[i + 1], cv::Scalar(0, 255, 0), linewidth);}cv::line(image, pos_[0], pos_[5], cv::Scalar(0, 255, 0), linewidth);for (int i = 6; i < 11; i++){cv::line(image, pos_[i], pos_[i + 1], cv::Scalar(0, 255, 0), linewidth);}cv::line(image, pos_[6], pos_[11], cv::Scalar(0, 255, 0), linewidth);for (cv::Point point : pos_){cv::circle(image, point, linewidth, cv::Scalar(255, 0, 0), -1);}
}

5 资源下载

本案例中涉及到的工程文件到此处下载。