Output Format
The two main projects that provide output files (that are not just videos) are FaceLandmarkImg and FeatureExtraction and their output is explained in detail in this page
The detected landmarks in an image are output in a text file, with a postfix _det_n if there are multiple faces in an image:
version: 1
npoints: 68
{
603.825 193.492
604.934 209.987
...
662.5 252.346
654.784 251.901
}
Each line corresponds to a facial landmark location in pixels (x y), the landmark indices follow the following scheme:
If -opdir <dir> is specified then the format changes to:
version: 1
npoints: 68
{
-34.4666 11.5525 501.305
-35.9184 33.9751 502.034
-35.361 56.7779 503.278
...
40.2726 77.8179 448.076
33.2769 76.5671 447.281
}
pose: eul_x, eul_y, eul_z:
{
-0.193585 -0.0835944 0.0948636
}
gaze: dir_x_1, dir_y_1, dir_z_1, dir_x_2, dir_y_2, dir_z_2:
{
0.0604244 0.125688 -0.990228 -0.137919 0.204113 -0.969183
}
Where the landmarks are no longer in pixel values but in milimeters and we also report head pose and gaze (this however needs accurate estimates of fx,fy,cx,cy. This functionality is useful for batch image processing where the camera is the same and we want to know pose and gaze.
The main output of feature extraction is a comma separated value file with facial landmarks, head pose, facial action units and eye gaze. Its format is as follows:
frame, timestamp, confidence, success, gaze_0_x, gaze_0_y, gaze_0_z, gaze_1_x, gaze_1_y, gaze_2_z, pose_Tx, pose_Ty, pose_Tz, pose_Rx, pose_Ry, pose_Rz, x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7, x_8, x_9, x_10, x_11, x_12, x_13, x_14, x_15, x_16, x_17, x_18, x_19, x_20, x_21, x_22, x_23, x_24, x_25, x_26, x_27, x_28, x_29, x_30, x_31, x_32, x_33, x_34, x_35, x_36, x_37, x_38, x_39, x_40, x_41, x_42, x_43, x_44, x_45, x_46, x_47, x_48, x_49, x_50, x_51, x_52, x_53, x_54, x_55, x_56, x_57, x_58, x_59, x_60, x_61, x_62, x_63, x_64, x_65, x_66, x_67, y_0, y_1, y_2, y_3, y_4, y_5, y_6, y_7, y_8, y_9, y_10, y_11, y_12, y_13, y_14, y_15, y_16, y_17, y_18, y_19, y_20, y_21, y_22, y_23, y_24, y_25, y_26, y_27, y_28, y_29, y_30, y_31, y_32, y_33, y_34, y_35, y_36, y_37, y_38, y_39, y_40, y_41, y_42, y_43, y_44, y_45, y_46, y_47, y_48, y_49, y_50, y_51, y_52, y_53, y_54, y_55, y_56, y_57, y_58, y_59, y_60, y_61, y_62, y_63, y_64, y_65, y_66, y_67, X_0, X_1, X_2, X_3, X_4, X_5, X_6, X_7, X_8, X_9, X_10, X_11, X_12, X_13, X_14, X_15, X_16, X_17, X_18, X_19, X_20, X_21, X_22, X_23, X_24, X_25, X_26, X_27, X_28, X_29, X_30, X_31, X_32, X_33, X_34, X_35, X_36, X_37, X_38, X_39, X_40, X_41, X_42, X_43, X_44, X_45, X_46, X_47, X_48, X_49, X_50, X_51, X_52, X_53, X_54, X_55, X_56, X_57, X_58, X_59, X_60, X_61, X_62, X_63, X_64, X_65, X_66, X_67, Y_0, Y_1, Y_2, Y_3, Y_4, Y_5, Y_6, Y_7, Y_8, Y_9, Y_10, Y_11, Y_12, Y_13, Y_14, Y_15, Y_16, Y_17, Y_18, Y_19, Y_20, Y_21, Y_22, Y_23, Y_24, Y_25, Y_26, Y_27, Y_28, Y_29, Y_30, Y_31, Y_32, Y_33, Y_34, Y_35, Y_36, Y_37, Y_38, Y_39, Y_40, Y_41, Y_42, Y_43, Y_44, Y_45, Y_46, Y_47, Y_48, Y_49, Y_50, Y_51, Y_52, Y_53, Y_54, Y_55, Y_56, Y_57, Y_58, Y_59, Y_60, Y_61, Y_62, Y_63, Y_64, Y_65, Y_66, Y_67, Z_0, Z_1, Z_2, Z_3, Z_4, Z_5, Z_6, Z_7, Z_8, Z_9, Z_10, Z_11, Z_12, Z_13, Z_14, Z_15, Z_16, Z_17, Z_18, Z_19, Z_20, Z_21, Z_22, Z_23, Z_24, Z_25, Z_26, Z_27, Z_28, Z_29, Z_30, Z_31, Z_32, Z_33, Z_34, Z_35, Z_36, Z_37, Z_38, Z_39, Z_40, Z_41, Z_42, Z_43, Z_44, Z_45, Z_46, Z_47, Z_48, Z_49, Z_50, Z_51, Z_52, Z_53, Z_54, Z_55, Z_56, Z_57, Z_58, Z_59, Z_60, Z_61, Z_62, Z_63, Z_64, Z_65, Z_66, Z_67, p_scale, p_rx, p_ry, p_rz, p_tx, p_ty, p_0, p_1, p_2, p_3, p_4, p_5, p_6, p_7, p_8, p_9, p_10, p_11, p_12, p_13, p_14, p_15, p_16, p_17, p_18, p_19, p_20, p_21, p_22, p_23, p_24, p_25, p_26, p_27, p_28, p_29, p_30, p_31, p_32, p_33, AU01_r, AU02_r, AU04_r, AU05_r, AU06_r, AU09_r, AU10_r, AU12_r, AU14_r, AU15_r, AU17_r, AU20_r, AU25_r, AU26_r, AU04_c, AU12_c, AU15_c, AU23_c, AU28_c, AU45_c
1, 0, 0.94848, 1, 0.10917, 0.147619, -0.983001, -0.166114, 0.136956, -0.97655, 21.184, -4.04434, 422.08, 0.0528253, -0.0474321, 0.0169401, 129.32, 129.568, 130.737, 132.621, 136.572, 143.131, 151.472, 161.39, 172.219, 182.459, 191.094, 198.564, 204.427, 208.15, 210.221, 211.544, 212.038, 137.679, 143.604, 151.613, 159.659, 166.93, 180.113, 187.491, 194.915, 202.123, 206.965, 173.757, 173.999, 174.255, 174.513, 164.595, 169.096, 173.678, 178.155, 182.141, 146.832, 151.662, 157.183, 161.887, 156.979, 151.443, 183.759, 188.823, 194.114, 198.327, 194.444, 189.226, 155.318, 162.665, 169.103, 173.317, 178.143, 183.948, 189.568, 183.938, 178.302, 173.212, 168.623, 162.257, 157.973, 168.889, 173.214, 178.095, 187.128, 178.053, 173.151, 168.76, 96.3795, 107.458, 118.622, 129.446, 139.556, 148.479, 155.831, 161.454, 163.038, 161.598, 156.413, 149.47, 140.829, 130.685, 119.815, 108.843, 97.7375, 88.3902, 83.379, 82.0184, 83.1791, 86.0683, 85.8879, 83.1019, 82.3679, 84.0608, 89.0695, 93.5734, 100.678, 107.762, 115.118, 120.167, 121.755, 123.095, 121.833, 120.453, 94.8765, 92.2265, 92.3677, 95.7136, 96.5624, 96.597, 95.9942, 92.5987, 92.7628, 95.4758, 96.9835, 96.9262, 133.507, 131.346, 130.399, 131.494, 130.532, 131.774, 133.82, 138.237, 140.029, 140.445, 140.003, 138.113, 133.83, 133.706, 134.202, 133.716, 134.111, 135.027, 135.524, 135.051, -58.3457, -57.8686, -55.7015, -52.0705, -44.2213, -31.3873, -15.4704, 2.43908, 21.2727, 39.7591, 57.0255, 72.6207, 84.8269, 92.2786, 96.0504, 98.3322, 99.4596, -39.2106, -28.2826, -14.2539, -0.571417, 11.4866, 33.4085, 46.1934, 59.6174, 73.1567, 83.0303, 22.8051, 22.7966, 22.7769, 22.7809, 7.55478, 14.8233, 22.1575, 29.6144, 36.4299, -22.7817, -14.3064, -4.82294, 3.22773, -5.15778, -14.6417, 40.7882, 49.5531, 58.8734, 66.6969, 59.3376, 50.1574, -8.06112, 4.43767, 14.947, 21.8374, 29.8506, 40.1354, 51.1408, 40.0907, 30.0283, 21.5851, 14.1216, 3.76024, -3.46833, 14.6581, 21.7447, 29.9099, 46.5786, 29.7515, 21.5704, 14.4138, -44.9202, -23.849, -2.62326, 17.9654, 36.9141, 52.9898, 65.0022, 72.7413, 74.9274, 73.6421, 66.7804, 55.4954, 39.7695, 20.4769, -0.353364, -21.2845, -42.5497, -55.527, -63.1692, -64.5493, -61.6804, -56.2403, -56.6602, -62.0003, -64.2568, -62.4176, -54.6825, -43.8063, -31.4651, -19.5541, -7.66358, 0.274791, 2.86038, 5.01436, 2.98958, 0.745632, -43.4648, -47.6557, -47.3021, -41.5453, -40.0151, -40.0447, -41.2118, -47.1081, -47.0056, -42.6776, -39.6515, -39.5986, 23.2559, 18.8953, 17.0763, 18.8488, 17.3286, 19.7329, 23.9027, 30.5422, 32.8614, 33.4028, 32.7583, 30.1806, 23.664, 22.6006, 23.3707, 22.6715, 24.2276, 24.7644, 25.4641, 24.7658, 475.436, 475.394, 475.868, 475.46, 471.895, 465.17, 453.538, 438.689, 435.239, 442.584, 458.493, 470.782, 477.338, 479.125, 478.136, 476.929, 477.818, 439.159, 431.236, 424.872, 418.787, 414.364, 415.25, 420.077, 426.875, 434.189, 441.978, 414.415, 407.107, 399.446, 392.419, 411.019, 407.402, 404.993, 407.801, 411.345, 432.512, 428.968, 427.96, 427.66, 426.826, 427.773, 429.186, 429.799, 431.446, 435.057, 430.685, 429.042, 430.446, 416.351, 410.516, 409.959, 411.321, 418.989, 432.401, 418.688, 410.179, 408.44, 409.407, 416.563, 427.753, 412.239, 411.386, 413.232, 429.244, 411.998, 410.066, 411.369, 0.579649, 0.0810897, -0.0438177, 0.0165988, 172.634, 117.523, 6.75643, 3.98344, -0.913317, 0.0951292, -7.52669, 9.41583, -0.982634, -0.479738, 1.05969, -0.755182, -1.9985, -0.206534, -2.59672, -0.208004, 0.222584, 0.116356, 0.133241, 0.994562, 0.0731596, -1.20472, -0.238291, -0.344064, -0.00294826, 0.0520344, -0.0141846, 0.030905, -0.269735, -0.0120041, 0.00161123, -0.00588343, 0.00292991, -0.0331269, -0.0246774, -0.0311158,0,0,0,0,0,0,0,0,0.552199,0,0.781291,0,0,0,0,1,0,0,0,0
2, 0.04, 0.953299, 1, 0.107503, 0.150627, -0.982728, -0.165798, 0.140331, -0.976124, 21.3134, -4.05283, 421.827, 0.0619525, -0.0490535, 0.0168232, 129.336, 129.567, 130.75, 132.65, 136.563, 143.075, 151.445, 161.462, 172.38, 182.676, 191.325, 198.773, 204.549, 208.192, 210.265, 211.618, 212.133, 137.856, 143.729, 151.727, 159.777, 167.044, 180.122, 187.493, 194.902, 202.075, 206.89, 173.806, 174.052, 174.313, 174.572, 164.818, 169.246, 173.749, 178.158, 182.088, 146.974, 151.765, 157.282, 161.952, 157.09, 151.557, 183.829, 188.841, 194.121, 198.291, 194.443, 189.236, 155.974, 163.107, 169.344, 173.436, 178.093, 183.736, 189.175, 183.746, 178.285, 173.366, 168.913, 162.75, 158.641, 169.142, 173.336, 178.049, 186.737, 178.019, 173.289, 169.038, 95.9927, 107.028, 118.141, 128.947, 139.12, 148.149, 155.628, 161.367, 162.976, 161.47, 156.173, 149.133, 140.443, 130.307, 119.476, 108.535, 97.4481, 88.2293, 83.2484, 81.9124, 83.1303, 86.0446, 85.8819, 83.0908, 82.3074, 83.9467, 88.8832, 93.5991, 100.657, 107.693, 115.004, 120.21, 121.774, 123.079, 121.855, 120.476, 94.8033, 92.1472, 92.2812, 95.6349, 96.5423, 96.5816, 95.9065, 92.5106, 92.6841, 95.3944, 96.9555, 96.896, 134.007, 131.723, 130.671, 131.765, 130.799, 132.118, 134.269, 138.531, 140.243, 140.649, 140.224, 138.43, 134.267, 134.042, 134.509, 134.044, 134.517, 135.211, 135.696, 135.234, -58.2086, -57.8092, -55.6541, -52.0168, -44.2455, -31.5006, -15.5284, 2.56799, 21.5799, 40.1942, 57.4893, 73.0478, 85.0984, 92.3773, 96.1133, 98.3753, 99.4681, -38.8064, -27.9958, -14.0245, -0.372126, 11.6562, 33.38, 46.1293, 59.5017, 72.95, 82.7599, 22.8533, 22.8575, 22.851, 22.8607, 7.91679, 15.0663, 22.2783, 29.6165, 36.3292, -22.4956, -14.1035, -4.64386, 3.33442, -4.96069, -14.4253, 40.8605, 49.5092, 58.8087, 66.5768, 59.2829, 50.121, -6.91503, 5.16541, 15.3299, 22.019, 29.7463, 39.7302, 50.3622, 39.7334, 30.011, 21.8481, 14.6001, 4.57635, -2.319, 15.0647, 21.9356, 29.8184, 45.8087, 29.6861, 21.7936, 14.8643, -45.5718, -24.6405, -3.53652, 17.0162, 36.0957, 52.39, 64.6725, 72.6753, 74.9137, 73.5076, 66.387, 54.8869, 39.0508, 19.7564, -1.00258, -21.8502, -43.0288, -55.6755, -63.236, -64.5682, -61.6266, -56.187, -56.5968, -61.9274, -64.2598, -62.5092, -54.9206, -43.7003, -31.4643, -19.6496, -7.83844, 0.344816, 2.89131, 4.98971, 3.02566, 0.783229, -43.5146, -47.6997, -47.3576, -41.6108, -39.989, -40.0125, -41.3143, -47.1893, -47.0799, -42.7818, -39.6634, -39.608, 24.056, 19.4857, 17.5062, 19.2801, 17.7546, 20.2844, 24.6317, 31.007, 33.2248, 33.7511, 33.1268, 30.6736, 24.3459, 23.1395, 23.8642, 23.2014, 24.872, 25.0598, 25.7411, 25.0555, 474.563, 474.886, 475.676, 475.469, 471.954, 465.285, 453.8, 439.211, 435.789, 443.135, 458.818, 470.999, 477.556, 479.211, 478.032, 476.455, 476.996, 438.105, 430.158, 423.814, 417.867, 413.682, 414.712, 419.458, 426.209, 433.45, 441.246, 413.815, 406.66, 399.141, 392.209, 410.76, 407.373, 405.085, 407.758, 411.182, 431.749, 428.141, 427.126, 426.951, 426.182, 427.149, 428.686, 429.158, 430.884, 434.676, 430.292, 428.583, 429.35, 415.56, 410.144, 409.697, 411.012, 418.464, 431.546, 418.321, 410.322, 408.639, 409.504, 416.087, 426.62, 411.968, 411.198, 413.014, 428.331, 411.867, 409.997, 411.171, 0.580147, 0.0901969, -0.0454037, 0.0164686, 172.717, 117.509, 7.10181, 4.50381, -4.01826, 1.54338, -8.07686, 7.79325, 0.242849, -0.430568, 2.11593, -0.569905, -1.95438, -0.186302, -2.62639, -0.66344, 0.287365, 0.0960486, 0.060306, 1.01928, 0.0387424, -1.19855, -0.197185, -0.298684, -0.0910158, -0.125318, -0.0463423, 0.0533237, -0.265746, -0.001024, 0.000431798, -0.0062599, -0.00415493, -0.0304479, -0.0233709, -0.0273382,0,0,0,0,0,0,0,0,0.915043,0,0.829461,0,0,0,1,1,0,0,0,0
3, 0.08, 0.955743, 1, 0.10642, 0.153098, -0.982464, -0.165396, 0.142754, -0.975841, 20.9377, -3.801, 420.152, 0.077066, -0.0420931, 0.0169737, 129.401, 129.594, 130.767, 132.676, 136.57, 143.019, 151.292, 161.226, 172.174, 182.559, 191.292, 198.798, 204.585, 208.236, 210.338, 211.708, 212.235, 137.757, 143.558, 151.557, 159.649, 166.934, 179.846, 187.29, 194.788, 202.024, 206.814, 173.596, 173.791, 174.003, 174.215, 164.574, 168.959, 173.436, 177.864, 181.811, 146.849, 151.61, 157.125, 161.785, 156.941, 151.406, 183.701, 188.703, 193.995, 198.159, 194.303, 189.084, 155.877, 162.898, 169.084, 173.14, 177.757, 183.416, 188.87, 183.418, 177.926, 173.048, 168.628, 162.526, 158.533, 168.889, 173.044, 177.718, 186.426, 177.674, 172.983, 168.774, 95.6202, 106.721, 117.895, 128.763, 139.03, 148.158, 155.763, 161.625, 163.27, 161.7, 156.224, 148.998, 140.187, 130.005, 119.171, 108.22, 97.0928, 88.0227, 83.0285, 81.6829, 82.9415, 85.9184, 85.7633, 82.9075, 82.0903, 83.7578, 88.7132, 93.5322, 100.648, 107.74, 115.111, 120.397, 121.981, 123.297, 122.083, 120.68, 94.6611, 91.9945, 92.1351, 95.5051, 96.4325, 96.4753, 95.7805, 92.3638, 92.5401, 95.2671, 96.8642, 96.7984, 134.514, 132.31, 131.265, 132.377, 131.409, 132.715, 134.797, 139.187, 140.946, 141.35, 140.907, 139.043, 134.785, 134.619, 135.093, 134.633, 135.065, 135.898, 136.382, 135.898, -57.7963, -57.5128, -55.4276, -51.8225, -44.1074, -31.5218, -15.7691, 2.1492, 21.1698, 39.8704, 57.2123, 72.7664, 84.7293, 91.9476, 95.6769, 97.8994, 98.9726, -38.7815, -28.1283, -14.2234, -0.582882, 11.3971, 32.7018, 45.4661, 58.8913, 72.3502, 82.0745, 22.3586, 22.2893, 22.2157, 22.161, 7.47888, 14.5276, 21.6631, 28.9891, 35.6869, -22.5903, -14.2881, -4.88292, 3.03171, -5.18665, -14.6055, 40.3788, 48.9382, 58.1852, 65.9022, 58.6493, 49.5365, -7.05484, 4.79916, 14.8453, 21.4504, 29.0745, 39.0269, 49.6099, 39.047, 29.3435, 21.2744, 14.0979, 4.19211, -2.49391, 14.595, 21.3746, 29.1584, 45.0794, 29.0195, 21.2248, 14.3849, -46.049, -25.1181, -3.99074, 16.6187, 35.8241, 52.2684, 64.7628, 72.9841, 75.2452, 73.6986, 66.2311, 54.3871, 38.3621, 19.0716, -1.57547, -22.3033, -43.4037, -55.7537, -63.2509, -64.5494, -61.5294, -56.0224, -56.4147, -61.7967, -64.1762, -62.3953, -54.8522, -43.5264, -31.2778, -19.4503, -7.62195, 0.649278, 3.21165, 5.31534, 3.37995, 1.11266, -43.5276, -47.692, -47.3305, -41.5955, -39.9576, -39.9819, -41.2621, -47.1186, -46.9998, -42.7146, -39.5565, -39.5174, 24.8362, 20.3861, 18.4102, 20.2054, 18.6809, 21.1924, 25.4271, 31.9927, 34.2885, 34.8108, 34.1632, 31.6067, 25.1329, 24.0016, 24.7326, 24.0811, 25.6989, 26.1037, 26.7825, 26.0654, 472.205, 472.875, 474.009, 474.141, 470.63, 464.069, 452.717, 438.346, 434.746, 441.841, 457.091, 468.883, 475.095, 476.549, 475.169, 473.324, 473.69, 435.885, 427.701, 421.153, 415.082, 410.943, 411.946, 416.504, 423.218, 430.406, 438.301, 411.126, 404.058, 396.624, 389.759, 408.75, 405.374, 403.087, 405.693, 409.045, 429.454, 425.738, 424.643, 424.533, 423.864, 424.892, 425.918, 426.24, 427.894, 431.76, 427.438, 425.804, 427.791, 414.009, 408.56, 408.111, 409.332, 416.673, 429.592, 416.845, 409.241, 407.624, 408.511, 414.931, 424.986, 410.466, 409.675, 411.426, 426.47, 410.485, 408.706, 409.877, 0.58296, 0.104298, -0.037298, 0.0164906, 172.547, 117.629, 5.15587, 4.96086, -4.71681, 3.75421, -7.39546, 7.4205, 0.815089, -0.930302, 3.05524, 0.635019, -1.79681, -0.262913, -2.66956, -1.19047, -0.168866, 0.441451, 0.0106802, 0.836421, -0.0732082, -1.31846, -0.009313, -0.427927, -0.152835, -0.169329, 0.00582317, -0.00231911, -0.302436, -0.00821701, -0.00528087, 0.00686356, -0.0275763, -0.0160663, -0.0206939, -0.0243734,0,0,0,0.585777,0,0,0,0,0.897447,0,0,0,0,0,0,1,0,0,0,0
4, 0.12, 0.963871, 1, 0.106936, 0.154229, -0.982231, -0.164536, 0.145976, -0.97551, 20.4166, -3.87284, 418.836, 0.0831857, -0.0378122, 0.0119711, 129.144, 129.391, 130.663, 132.677, 136.624, 143.065, 151.28, 161.147, 172.084, 182.496, 191.261, 198.78, 204.543, 208.15, 210.23, 211.575, 212.05, 137.348, 143.094, 151.069, 159.147, 166.416, 179.388, 186.845, 194.366, 201.622, 206.425, 173.146, 173.35, 173.57, 173.794, 164.306, 168.655, 173.097, 177.508, 181.441, 146.413, 151.144, 156.68, 161.364, 156.515, 150.961, 183.329, 188.301, 193.62, 197.804, 193.941, 188.694, 155.989, 162.766, 168.859, 172.893, 177.457, 183.089, 188.434, 183.124, 177.679, 172.837, 168.435, 162.43, 158.656, 168.69, 172.813, 177.439, 185.967, 177.405, 172.753, 168.574, 95.3783, 106.561, 117.817, 128.737, 139.072, 148.242, 155.899, 161.8, 163.445, 161.821, 156.208, 148.816, 139.888, 129.629, 118.745, 107.729, 96.5486, 87.8055, 82.8152, 81.471, 82.7331, 85.6975, 85.5142, 82.6243, 81.7406, 83.3467, 88.2501, 93.3341, 100.469, 107.574, 114.96, 120.346, 121.903, 123.189, 121.972, 120.548, 94.4962, 91.7979, 91.9157, 95.2878, 96.2555, 96.3212, 95.4683, 92.021, 92.1838, 94.8937, 96.5396, 96.4907, 134.93, 132.47, 131.274, 132.386, 131.388, 132.787, 135.047, 139.58, 141.426, 141.852, 141.42, 139.522, 135.15, 134.745, 135.186, 134.732, 135.304, 136.217, 136.719, 136.244, -58.0438, -57.7087, -55.4857, -51.7169, -43.9191, -31.3727, -15.7612, 2.0087, 20.9785, 39.6772, 56.9978, 72.5033, 84.3436, 91.4036, 95.0215, 97.1298, 98.0419, -39.3322, -28.8012, -14.9813, -1.41038, 10.5049, 31.8226, 44.5371, 57.9182, 71.3196, 80.9994, 21.5338, 21.4946, 21.453, 21.4328, 7.01797, 13.9888, 21.0505, 28.315, 34.9537, -23.2538, -15.0221, -5.6169, 2.30756, -5.88643, -15.3085, 39.5821, 48.0401, 57.2876, 65.0072, 57.7844, 48.6685, -6.83714, 4.56332, 14.4231, 20.9665, 28.4698, 38.3203, 48.6254, 38.432, 28.8791, 20.8921, 13.7578, 4.02238, -2.27471, 14.221, 20.9252, 28.5985, 44.0923, 28.4991, 20.7979, 14.0224, -46.3159, -25.3367, -4.128, 16.5375, 35.8332, 52.3181, 64.8855, 73.184, 75.4207, 73.7597, 66.0177, 53.8737, 37.6595, 18.2782, -2.37357, -23.1091, -44.1732, -55.9018, -63.3495, -64.6288, -61.615, -56.161, -56.6028, -62.0088, -64.4808, -62.8062, -55.3957, -43.6786, -31.4468, -19.644, -7.83114, 0.563514, 3.07607, 5.12585, 3.18886, 0.89298, -43.6502, -47.8403, -47.5113, -41.7962, -40.1093, -40.1018, -41.6224, -47.4935, -47.3978, -43.1723, -39.9416, -39.8749, 25.4465, 20.5698, 18.3556, 20.1422, 18.5721, 21.2223, 25.733, 32.5414, 35.0005, 35.5642, 34.9361, 32.3131, 25.6484, 24.13, 24.8013, 24.1595, 25.9861, 26.5539, 27.2651, 26.5642, 470.274, 471.331, 472.833, 473.199, 469.711, 463.123, 451.86, 437.701, 433.999, 440.928, 455.82, 467.397, 473.388, 474.575, 472.933, 470.82, 470.901, 434.094, 425.91, 419.352, 413.336, 409.306, 410.335, 414.766, 421.339, 428.38, 436.188, 409.5, 402.532, 395.218, 388.432, 407.42, 404.087, 401.832, 404.316, 407.564, 427.879, 424.085, 422.935, 422.83, 422.301, 423.394, 424.17, 424.368, 425.991, 429.895, 425.628, 424.033, 426.107, 412.374, 407.018, 406.555, 407.71, 414.922, 427.534, 415.492, 408.386, 406.883, 407.754, 413.801, 423.254, 409.126, 408.293, 409.981, 424.509, 409.361, 407.693, 408.843, 0.585041, 0.110759, -0.0323022, 0.0113965, 172.276, 117.569, 3.39147, 5.48401, -5.82276, 5.7464, -7.71578, 5.88891, 1.78466, -0.592173, 3.80212, 0.150893, -1.38067, -0.384341, -2.18357, -1.15011, -0.0751731, 0.465033, 0.123919, 0.467066, -0.0636545, -1.50489, 0.0495619, -0.382291, -0.102015, -0.140306, 0.0301354, 0.0119626, -0.296017, 0.00944414, -0.00441807, 0.00994939, -0.03512, -0.0213233, -0.0229672, -0.0172276,0,0,0,0.761611,0,0,0,0,0.859176,0,0,0,0,0,1,1,0,0,0,1
...
...
204, 8.12, 0.952574, 1, 0.121502, 0.17169, -0.97763, -0.143931, 0.155402, -0.97731, 12.8349, -5.59191, 410.328, 0.107749, -0.058971, 0.0502874, 124.423, 124.206, 125.189, 127.019, 130.81, 136.986, 144.871, 154.67, 165.76, 176.364, 185.205, 192.765, 198.453, 202.191, 204.701, 206.547, 207.38, 134.016, 140.133, 148.326, 156.506, 163.684, 176.924, 184.35, 191.732, 198.696, 202.84, 170.352, 170.333, 170.369, 170.413, 160.385, 164.655, 169.032, 173.437, 177.298, 142.986, 147.962, 153.561, 158.117, 153.19, 147.555, 179.885, 185.047, 190.369, 194.303, 190.481, 185.208, 151.2, 157.83, 164.439, 168.367, 172.788, 178.289, 182.76, 177.82, 172.299, 167.523, 163.095, 156.868, 153.929, 164.094, 168.118, 172.56, 180.245, 172.316, 167.75, 163.613, 90.7048, 102.714, 114.894, 126.637, 137.949, 148.032, 156.631, 163.26, 165.364, 163.631, 157.274, 148.879, 139.271, 128.646, 117.732, 106.652, 95.4064, 84.1519, 79.2084, 78.1078, 79.7568, 83.1749, 83.5542, 80.9658, 80.2812, 82.0667, 87.1396, 91.0586, 98.2875, 105.457, 112.959, 118.162, 119.885, 121.381, 120.315, 118.953, 91.1784, 88.6154, 88.9671, 92.5478, 93.3091, 93.1254, 93.5461, 90.3313, 90.7103, 93.5923, 95.0917, 94.802, 134.062, 130.64, 129.237, 130.557, 129.695, 131.766, 135.318, 141.588, 144.028, 144.359, 143.74, 140.793, 134.273, 133.083, 133.62, 133.43, 135.495, 138.078, 138.506, 137.853, -64.8277, -65.5949, -64.161, -60.9109, -53.5648, -41.7385, -26.877, -9.2042, 9.87829, 28.5026, 45.2353, 60.0876, 71.1772, 78.048, 82.2624, 85.2714, 86.7794, -43.6395, -32.8012, -19.0088, -5.62194, 5.88637, 27.2162, 39.6153, 52.4579, 65.0991, 73.379, 16.5598, 16.2532, 16.0267, 15.8265, 0.612278, 7.35013, 14.1969, 21.2636, 27.6129, -28.2466, -19.8299, -10.5916, -3.09777, -11.1908, -20.4795, 33.0152, 41.5994, 50.6684, 57.7878, 50.8295, 41.8479, -14.6597, -3.49515, 7.05118, 13.2859, 20.3825, 29.6938, 38.1083, 29.1565, 19.8406, 12.0906, 4.97844, -5.09143, -10.0578, 6.55086, 12.9695, 20.1658, 33.6984, 19.8808, 12.4544, 5.81502, -53.3805, -31.6783, -9.41097, 12.2573, 32.9367, 50.8391, 65.0743, 74.7071, 77.7971, 75.9963, 66.8945, 52.9602, 35.6703, 15.9935, -4.17446, -24.4526, -45.0452, -60.2068, -67.3474, -68.2115, -64.7472, -58.8435, -58.6091, -63.5065, -65.6617, -63.8166, -56.2858, -46.296, -34.1534, -22.4783, -10.7005, -2.92296, -0.18142, 2.17019, 0.499203, -1.67149, -47.8494, -51.6986, -51.0428, -45.1676, -43.8579, -44.2264, -43.9206, -49.2762, -48.8674, -44.4865, -41.5367, -41.8321, 23.4261, 17.136, 14.6729, 16.7639, 15.4535, 19.1032, 25.6484, 35.3208, 38.7598, 39.147, 38.1877, 33.8, 23.6442, 20.9363, 21.7583, 21.5635, 25.7923, 29.1829, 29.7385, 28.7325, 455.539, 458.139, 460.781, 461.714, 458.764, 453.398, 444.118, 431.729, 428.738, 435.451, 448.667, 458.469, 462.749, 462.473, 460.067, 457.984, 457.895, 419.875, 412.753, 407.066, 402.224, 399.48, 402.029, 406.735, 413.292, 420.584, 428.219, 399.911, 393.246, 386.418, 379.956, 397.658, 394.762, 392.961, 395.618, 399.086, 415.048, 411.815, 411.2, 411.329, 410.794, 411.415, 415.066, 415.22, 417.104, 421.152, 416.896, 415.034, 416.489, 402.647, 397.135, 396.991, 398.476, 405.901, 418.595, 409.035, 403.281, 401.767, 402.138, 406.381, 414.142, 400.053, 399.388, 401.396, 416.129, 403.568, 401.731, 402.343, 0.598208, 0.136351, -0.053165, 0.0496707, 167.876, 116.389, -13.2689, 13.6493, -1.02892, 14.3784, -10.5895, -2.9426, 6.2489, -2.82006, 5.56716, -3.2069, 0.360861, -0.513787, -1.28229, -0.941657, 1.25147, 0.0729805, 1.28252, 0.780729, -0.533899, -0.698888, 0.809073, -0.755656, 0.0321429, 0.0375847, 0.10858, 0.308759, 0.0772421, -0.0557184, -0.0312446, -0.00364208, -0.0493718, -0.00840708, -0.013717, 0.00441885,0,0,0,0,0,0.583841,0,0,0,0,0,0,2.76711,0.790546,0,0,0,1,0,0
205, 8.16, 0.943502, 1, 0.119735, 0.171356, -0.977906, -0.141987, 0.154965, -0.977663, 13.3425, -6.40826, 410.029, 0.0806764, -0.0697097, 0.0482961, 124.486, 124.252, 125.217, 127.038, 130.833, 137.022, 144.961, 154.903, 166.055, 176.638, 185.385, 192.861, 198.514, 202.261, 204.773, 206.598, 207.407, 134.295, 140.436, 148.657, 156.836, 163.983, 177.144, 184.535, 191.864, 198.783, 202.935, 170.605, 170.671, 170.785, 170.901, 160.762, 165.072, 169.495, 173.902, 177.752, 143.257, 148.209, 153.793, 158.343, 153.444, 147.829, 180.137, 185.263, 190.544, 194.433, 190.682, 185.455, 151.681, 158.442, 164.953, 168.876, 173.295, 178.673, 183.105, 178.271, 172.891, 168.138, 163.741, 157.567, 154.428, 164.603, 168.626, 173.064, 180.638, 172.853, 168.306, 164.182, 91.0214, 102.987, 115.135, 126.847, 138.101, 148.116, 156.591, 163.054, 165.089, 163.481, 157.366, 149.133, 139.554, 128.863, 117.816, 106.614, 95.2744, 83.855, 78.8175, 77.6544, 79.2258, 82.5482, 82.8669, 80.3027, 79.6619, 81.5165, 86.6856, 90.3551, 97.574, 104.74, 112.233, 117.593, 119.307, 120.781, 119.707, 118.348, 90.8022, 88.1684, 88.4913, 92.027, 92.8203, 92.6803, 92.9791, 89.7532, 90.1293, 93.0418, 94.4966, 94.2085, 133.686, 130.224, 128.796, 130.083, 129.227, 131.271, 134.808, 140.566, 142.765, 143.083, 142.495, 139.836, 133.868, 132.578, 133.092, 132.893, 134.958, 137.01, 137.427, 136.802, -64.7382, -65.4541, -63.9829, -60.708, -53.3381, -41.5021, -26.5929, -8.75591, 10.328, 28.8486, 45.416, 60.1852, 71.3281, 78.3458, 82.6714, 85.6934, 87.2226, -43.2659, -32.3908, -18.5333, -5.10962, 6.39056, 27.6794, 40.0855, 52.9093, 65.5315, 73.8434, 17.0189, 16.8272, 16.6984, 16.5816, 1.21236, 8.00656, 14.921, 22.0026, 28.3581, -27.8489, -19.4695, -10.2356, -2.73267, -10.7948, -20.0668, 33.5316, 42.1028, 51.1382, 58.1974, 51.3232, 42.3829, -13.8222, -2.50517, 7.8642, 14.0885, 21.1879, 30.3096, 38.654, 29.8141, 20.7193, 13.0256, 5.99305, -3.93975, -9.20443, 7.35589, 13.7624, 20.9548, 34.3077, 20.6938, 13.3089, 6.71097, -52.8242, -31.1508, -8.94923, 12.6099, 33.1017, 50.7809, 64.7007, 73.9557, 76.911, 75.3902, 66.8523, 53.3569, 36.2136, 16.4308, -4.03298, -24.6165, -45.4914, -60.8382, -68.1848, -69.1889, -65.8564, -60.0833, -59.9537, -64.8587, -66.9801, -65.0251, -57.2969, -47.5757, -35.3623, -23.6268, -11.8157, -3.8292, -1.09356, 1.22785, -0.464457, -2.63968, -48.5654, -52.5601, -51.9563, -46.1358, -44.7531, -45.0424, -44.9936, -50.408, -50.0112, -45.564, -42.6612, -42.9431, 22.7395, 16.4402, 13.9663, 16.0044, 14.7051, 18.2949, 24.7743, 33.5577, 36.5908, 36.9477, 36.0386, 32.118, 22.9081, 20.099, 20.889, 20.6815, 24.865, 27.3873, 27.9229, 26.9612, 455.718, 457.749, 459.875, 460.438, 457.173, 451.532, 442.055, 429.433, 426.435, 433.468, 447.274, 457.881, 463.006, 463.463, 461.618, 459.752, 459.962, 420.793, 413.919, 408.478, 403.787, 401.071, 403.641, 408.458, 415.117, 422.422, 429.971, 401.213, 394.212, 387.066, 380.294, 397.679, 394.668, 392.847, 395.674, 399.36, 415.831, 412.798, 412.237, 412.325, 411.641, 412.179, 416.285, 416.64, 418.564, 422.543, 418.192, 416.253, 415.375, 402.005, 396.972, 396.835, 398.429, 405.789, 418.246, 407.933, 401.833, 400.164, 400.515, 404.794, 412.97, 399.495, 398.885, 401.007, 415.585, 402.523, 400.559, 401.158, 0.598085, 0.11082, -0.0664424, 0.0479842, 168.181, 115.939, -11.9401, 12.1435, -2.53836, 14.045, -11.2765, -0.732664, 6.17934, -3.12388, 5.21925, -3.14636, 0.102961, -0.0954346, -1.78584, -1.12418, 1.58407, -0.0888273, 0.826367, 0.700256, -0.351087, -0.623023, 0.66073, -0.450211, -0.0178976, -0.175885, 0.205149, 0.308204, 0.0731635, -0.0274786, -0.0243863, -0.00930196, -0.0267284, -0.0130051, -0.0129202, 0.000738565,0.720321,0,0,0.522199,0,0.992203,0,0,0,0,0,0,3.04877,0.744715,1,0,0,1,0,0
The header specifies the meaning of each column. The explanation of each:
Basic
frame the number of the frame processed
timestamp the timer of video being processed in seconds
confidence how confident is the tracker in current landmark detection estimage
success is the track successful (is there a face in the frame or do we think we tracked it well)
Gaze
gaze_0_x, gaze_0_y, gaze_0_z Eye gaze direction vector in world coordinates for eye 0 (normalized)
gaze_1_x, gaze_1_y, gaze_2_z Eye gaze direction vector in world coordinates for eye 0 (normalized)
Pose
pose_Tx, pose_Ty, pose_Tz the location of the head with respect to camera in milometers (positive Z is away from the camera)
pose_Rx, pose_Ry, pose_Rz Rotation is in radians around X,Y,Z axes with the convention R = Rx * Ry * Rz, left-handed positive sign. The rotation can be either in world or camera coordinates (for visualisation we want rotation with respect to world coordinates). This is controlled by world_coord <1/0> flag
Landmarks locations in 2D
x_0, x_1, ... x_66, x_67, y_0,...y_67 location of 2D landmarks in pixels, the landmark index can be seen above (just subtract 1 to get from a Matlab to a C++ standard)
Landmarks locations in 3D
X_0, ... X_67, Y_0,...Y_67, Z_0,...Z_67 location of 3D landmarks in milimeters, the landmark index can be seen above (just subtract 1 to get from a Matlab to a C++ standard). For this to be accurate need to have good estimates for fx,fy,cx,cy
Rigid and non-rigid shape parameters
Parameters of a point distribution model (PDM) that describe the rigid face shape (location, scale and rotation) and non-rigid face shape (deformation due to expression and identity). For more details look at chapter 4.2 of my Thesis for more details.
p_scale, p_rx, p_ry, p_rz, p_tx, p_ty - scale, rotation and translation terms of the PDM
p_0, p_1, ... p_33 - non-rigid shape parameters
Facial Action Units
Facial Action Units (AUs) are a way to describe human facial expression, more details on how they are defined can be found here
The system can detect the intensity (from 0 to 5) of 14 AUs:
AU01_r, AU02_r, AU04_r, AU05_r, AU06_r, AU09_r, AU10_r, AU12_r, AU14_r, AU15_r, AU17_r, AU20_r, AU25_r, AU26_r
And the presense (0 absent, 1 present) of 6 AUs:
AU04_c, AU12_c, AU15_c, AU23_c, AU28_c, AU45_c
Similarity aligned faces and HOG
The code also allows for extraction of aligned faces (left) and HOG features (right). The HOG features are written as a binary file (for space consideration) and can be read in using the ./matlab_runners/Demos/Read_HOG_files.m script to Matlab, see ./matlab_runners/Demos/feature_extraction_demo_vid.m for an example (only on Windows at the moment). Once the script is finished, the HOG features are stored in hog_data variable