Output Format
The two projects that process data are FaceLandmarkImg and FeatureExtraction and their output is explained in detail in this page. FaceLandmarkImg is intended for individual images and FeatureExtraction for sequence analysis (image sequence/webcam/video). Their output format is the same with the exception that FeatureExtraction assumes that there is only one face in the sequence, so each data row/observation corresponds to a frame, while FaceLandmarkImg can process multiple faces in the same image and each data row/observation corresponds to a face.
Part of OpenFace processing outputs a comma separated value file of the following format:
frame, timestamp, confidence, success, gaze_0_x, gaze_0_y, gaze_0_z, gaze_1_x, gaze_1_y, gaze_1_z, pose_Tx, pose_Ty, pose_Tz, pose_Rx, pose_Ry, pose_Rz, x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7, x_8, x_9, x_10, x_11, x_12, x_13, x_14, x_15, x_16, x_17, x_18, x_19, x_20, x_21, x_22, x_23, x_24, x_25, x_26, x_27, x_28, x_29, x_30, x_31, x_32, x_33, x_34, x_35, x_36, x_37, x_38, x_39, x_40, x_41, x_42, x_43, x_44, x_45, x_46, x_47, x_48, x_49, x_50, x_51, x_52, x_53, x_54, x_55, x_56, x_57, x_58, x_59, x_60, x_61, x_62, x_63, x_64, x_65, x_66, x_67, y_0, y_1, y_2, y_3, y_4, y_5, y_6, y_7, y_8, y_9, y_10, y_11, y_12, y_13, y_14, y_15, y_16, y_17, y_18, y_19, y_20, y_21, y_22, y_23, y_24, y_25, y_26, y_27, y_28, y_29, y_30, y_31, y_32, y_33, y_34, y_35, y_36, y_37, y_38, y_39, y_40, y_41, y_42, y_43, y_44, y_45, y_46, y_47, y_48, y_49, y_50, y_51, y_52, y_53, y_54, y_55, y_56, y_57, y_58, y_59, y_60, y_61, y_62, y_63, y_64, y_65, y_66, y_67, X_0, X_1, X_2, X_3, X_4, X_5, X_6, X_7, X_8, X_9, X_10, X_11, X_12, X_13, X_14, X_15, X_16, X_17, X_18, X_19, X_20, X_21, X_22, X_23, X_24, X_25, X_26, X_27, X_28, X_29, X_30, X_31, X_32, X_33, X_34, X_35, X_36, X_37, X_38, X_39, X_40, X_41, X_42, X_43, X_44, X_45, X_46, X_47, X_48, X_49, X_50, X_51, X_52, X_53, X_54, X_55, X_56, X_57, X_58, X_59, X_60, X_61, X_62, X_63, X_64, X_65, X_66, X_67, Y_0, Y_1, Y_2, Y_3, Y_4, Y_5, Y_6, Y_7, Y_8, Y_9, Y_10, Y_11, Y_12, Y_13, Y_14, Y_15, Y_16, Y_17, Y_18, Y_19, Y_20, Y_21, Y_22, Y_23, Y_24, Y_25, Y_26, Y_27, Y_28, Y_29, Y_30, Y_31, Y_32, Y_33, Y_34, Y_35, Y_36, Y_37, Y_38, Y_39, Y_40, Y_41, Y_42, Y_43, Y_44, Y_45, Y_46, Y_47, Y_48, Y_49, Y_50, Y_51, Y_52, Y_53, Y_54, Y_55, Y_56, Y_57, Y_58, Y_59, Y_60, Y_61, Y_62, Y_63, Y_64, Y_65, Y_66, Y_67, Z_0, Z_1, Z_2, Z_3, Z_4, Z_5, Z_6, Z_7, Z_8, Z_9, Z_10, Z_11, Z_12, Z_13, Z_14, Z_15, Z_16, Z_17, Z_18, Z_19, Z_20, Z_21, Z_22, Z_23, Z_24, Z_25, Z_26, Z_27, Z_28, Z_29, Z_30, Z_31, Z_32, Z_33, Z_34, Z_35, Z_36, Z_37, Z_38, Z_39, Z_40, Z_41, Z_42, Z_43, Z_44, Z_45, Z_46, Z_47, Z_48, Z_49, Z_50, Z_51, Z_52, Z_53, Z_54, Z_55, Z_56, Z_57, Z_58, Z_59, Z_60, Z_61, Z_62, Z_63, Z_64, Z_65, Z_66, Z_67, p_scale, p_rx, p_ry, p_rz, p_tx, p_ty, p_0, p_1, p_2, p_3, p_4, p_5, p_6, p_7, p_8, p_9, p_10, p_11, p_12, p_13, p_14, p_15, p_16, p_17, p_18, p_19, p_20, p_21, p_22, p_23, p_24, p_25, p_26, p_27, p_28, p_29, p_30, p_31, p_32, p_33, AU01_r, AU02_r, AU04_r, AU05_r, AU06_r, AU09_r, AU10_r, AU12_r, AU14_r, AU15_r, AU17_r, AU20_r, AU25_r, AU26_r, AU04_c, AU12_c, AU15_c, AU23_c, AU28_c, AU45_c
1, 0, 0.94848, 1, 0.10917, 0.147619, -0.983001, -0.166114, 0.136956, -0.97655, 21.184, -4.04434, 422.08, 0.0528253, -0.0474321, 0.0169401, 129.32, 129.568, 130.737, 132.621, 136.572, 143.131, 151.472, 161.39, 172.219, 182.459, 191.094, 198.564, 204.427, 208.15, 210.221, 211.544, 212.038, 137.679, 143.604, 151.613, 159.659, 166.93, 180.113, 187.491, 194.915, 202.123, 206.965, 173.757, 173.999, 174.255, 174.513, 164.595, 169.096, 173.678, 178.155, 182.141, 146.832, 151.662, 157.183, 161.887, 156.979, 151.443, 183.759, 188.823, 194.114, 198.327, 194.444, 189.226, 155.318, 162.665, 169.103, 173.317, 178.143, 183.948, 189.568, 183.938, 178.302, 173.212, 168.623, 162.257, 157.973, 168.889, 173.214, 178.095, 187.128, 178.053, 173.151, 168.76, 96.3795, 107.458, 118.622, 129.446, 139.556, 148.479, 155.831, 161.454, 163.038, 161.598, 156.413, 149.47, 140.829, 130.685, 119.815, 108.843, 97.7375, 88.3902, 83.379, 82.0184, 83.1791, 86.0683, 85.8879, 83.1019, 82.3679, 84.0608, 89.0695, 93.5734, 100.678, 107.762, 115.118, 120.167, 121.755, 123.095, 121.833, 120.453, 94.8765, 92.2265, 92.3677, 95.7136, 96.5624, 96.597, 95.9942, 92.5987, 92.7628, 95.4758, 96.9835, 96.9262, 133.507, 131.346, 130.399, 131.494, 130.532, 131.774, 133.82, 138.237, 140.029, 140.445, 140.003, 138.113, 133.83, 133.706, 134.202, 133.716, 134.111, 135.027, 135.524, 135.051, -58.3457, -57.8686, -55.7015, -52.0705, -44.2213, -31.3873, -15.4704, 2.43908, 21.2727, 39.7591, 57.0255, 72.6207, 84.8269, 92.2786, 96.0504, 98.3322, 99.4596, -39.2106, -28.2826, -14.2539, -0.571417, 11.4866, 33.4085, 46.1934, 59.6174, 73.1567, 83.0303, 22.8051, 22.7966, 22.7769, 22.7809, 7.55478, 14.8233, 22.1575, 29.6144, 36.4299, -22.7817, -14.3064, -4.82294, 3.22773, -5.15778, -14.6417, 40.7882, 49.5531, 58.8734, 66.6969, 59.3376, 50.1574, -8.06112, 4.43767, 14.947, 21.8374, 29.8506, 40.1354, 51.1408, 40.0907, 30.0283, 21.5851, 14.1216, 3.76024, -3.46833, 14.6581, 21.7447, 29.9099, 46.5786, 29.7515, 21.5704, 14.4138, -44.9202, -23.849, -2.62326, 17.9654, 36.9141, 52.9898, 65.0022, 72.7413, 74.9274, 73.6421, 66.7804, 55.4954, 39.7695, 20.4769, -0.353364, -21.2845, -42.5497, -55.527, -63.1692, -64.5493, -61.6804, -56.2403, -56.6602, -62.0003, -64.2568, -62.4176, -54.6825, -43.8063, -31.4651, -19.5541, -7.66358, 0.274791, 2.86038, 5.01436, 2.98958, 0.745632, -43.4648, -47.6557, -47.3021, -41.5453, -40.0151, -40.0447, -41.2118, -47.1081, -47.0056, -42.6776, -39.6515, -39.5986, 23.2559, 18.8953, 17.0763, 18.8488, 17.3286, 19.7329, 23.9027, 30.5422, 32.8614, 33.4028, 32.7583, 30.1806, 23.664, 22.6006, 23.3707, 22.6715, 24.2276, 24.7644, 25.4641, 24.7658, 475.436, 475.394, 475.868, 475.46, 471.895, 465.17, 453.538, 438.689, 435.239, 442.584, 458.493, 470.782, 477.338, 479.125, 478.136, 476.929, 477.818, 439.159, 431.236, 424.872, 418.787, 414.364, 415.25, 420.077, 426.875, 434.189, 441.978, 414.415, 407.107, 399.446, 392.419, 411.019, 407.402, 404.993, 407.801, 411.345, 432.512, 428.968, 427.96, 427.66, 426.826, 427.773, 429.186, 429.799, 431.446, 435.057, 430.685, 429.042, 430.446, 416.351, 410.516, 409.959, 411.321, 418.989, 432.401, 418.688, 410.179, 408.44, 409.407, 416.563, 427.753, 412.239, 411.386, 413.232, 429.244, 411.998, 410.066, 411.369, 0.579649, 0.0810897, -0.0438177, 0.0165988, 172.634, 117.523, 6.75643, 3.98344, -0.913317, 0.0951292, -7.52669, 9.41583, -0.982634, -0.479738, 1.05969, -0.755182, -1.9985, -0.206534, -2.59672, -0.208004, 0.222584, 0.116356, 0.133241, 0.994562, 0.0731596, -1.20472, -0.238291, -0.344064, -0.00294826, 0.0520344, -0.0141846, 0.030905, -0.269735, -0.0120041, 0.00161123, -0.00588343, 0.00292991, -0.0331269, -0.0246774, -0.0311158,0,0,0,0,0,0,0,0,0.552199,0,0.781291,0,0,0,0,1,0,0,0,0
2, 0.04, 0.953299, 1, 0.107503, 0.150627, -0.982728, -0.165798, 0.140331, -0.976124, 21.3134, -4.05283, 421.827, 0.0619525, -0.0490535, 0.0168232, 129.336, 129.567, 130.75, 132.65, 136.563, 143.075, 151.445, 161.462, 172.38, 182.676, 191.325, 198.773, 204.549, 208.192, 210.265, 211.618, 212.133, 137.856, 143.729, 151.727, 159.777, 167.044, 180.122, 187.493, 194.902, 202.075, 206.89, 173.806, 174.052, 174.313, 174.572, 164.818, 169.246, 173.749, 178.158, 182.088, 146.974, 151.765, 157.282, 161.952, 157.09, 151.557, 183.829, 188.841, 194.121, 198.291, 194.443, 189.236, 155.974, 163.107, 169.344, 173.436, 178.093, 183.736, 189.175, 183.746, 178.285, 173.366, 168.913, 162.75, 158.641, 169.142, 173.336, 178.049, 186.737, 178.019, 173.289, 169.038, 95.9927, 107.028, 118.141, 128.947, 139.12, 148.149, 155.628, 161.367, 162.976, 161.47, 156.173, 149.133, 140.443, 130.307, 119.476, 108.535, 97.4481, 88.2293, 83.2484, 81.9124, 83.1303, 86.0446, 85.8819, 83.0908, 82.3074, 83.9467, 88.8832, 93.5991, 100.657, 107.693, 115.004, 120.21, 121.774, 123.079, 121.855, 120.476, 94.8033, 92.1472, 92.2812, 95.6349, 96.5423, 96.5816, 95.9065, 92.5106, 92.6841, 95.3944, 96.9555, 96.896, 134.007, 131.723, 130.671, 131.765, 130.799, 132.118, 134.269, 138.531, 140.243, 140.649, 140.224, 138.43, 134.267, 134.042, 134.509, 134.044, 134.517, 135.211, 135.696, 135.234, -58.2086, -57.8092, -55.6541, -52.0168, -44.2455, -31.5006, -15.5284, 2.56799, 21.5799, 40.1942, 57.4893, 73.0478, 85.0984, 92.3773, 96.1133, 98.3753, 99.4681, -38.8064, -27.9958, -14.0245, -0.372126, 11.6562, 33.38, 46.1293, 59.5017, 72.95, 82.7599, 22.8533, 22.8575, 22.851, 22.8607, 7.91679, 15.0663, 22.2783, 29.6165, 36.3292, -22.4956, -14.1035, -4.64386, 3.33442, -4.96069, -14.4253, 40.8605, 49.5092, 58.8087, 66.5768, 59.2829, 50.121, -6.91503, 5.16541, 15.3299, 22.019, 29.7463, 39.7302, 50.3622, 39.7334, 30.011, 21.8481, 14.6001, 4.57635, -2.319, 15.0647, 21.9356, 29.8184, 45.8087, 29.6861, 21.7936, 14.8643, -45.5718, -24.6405, -3.53652, 17.0162, 36.0957, 52.39, 64.6725, 72.6753, 74.9137, 73.5076, 66.387, 54.8869, 39.0508, 19.7564, -1.00258, -21.8502, -43.0288, -55.6755, -63.236, -64.5682, -61.6266, -56.187, -56.5968, -61.9274, -64.2598, -62.5092, -54.9206, -43.7003, -31.4643, -19.6496, -7.83844, 0.344816, 2.89131, 4.98971, 3.02566, 0.783229, -43.5146, -47.6997, -47.3576, -41.6108, -39.989, -40.0125, -41.3143, -47.1893, -47.0799, -42.7818, -39.6634, -39.608, 24.056, 19.4857, 17.5062, 19.2801, 17.7546, 20.2844, 24.6317, 31.007, 33.2248, 33.7511, 33.1268, 30.6736, 24.3459, 23.1395, 23.8642, 23.2014, 24.872, 25.0598, 25.7411, 25.0555, 474.563, 474.886, 475.676, 475.469, 471.954, 465.285, 453.8, 439.211, 435.789, 443.135, 458.818, 470.999, 477.556, 479.211, 478.032, 476.455, 476.996, 438.105, 430.158, 423.814, 417.867, 413.682, 414.712, 419.458, 426.209, 433.45, 441.246, 413.815, 406.66, 399.141, 392.209, 410.76, 407.373, 405.085, 407.758, 411.182, 431.749, 428.141, 427.126, 426.951, 426.182, 427.149, 428.686, 429.158, 430.884, 434.676, 430.292, 428.583, 429.35, 415.56, 410.144, 409.697, 411.012, 418.464, 431.546, 418.321, 410.322, 408.639, 409.504, 416.087, 426.62, 411.968, 411.198, 413.014, 428.331, 411.867, 409.997, 411.171, 0.580147, 0.0901969, -0.0454037, 0.0164686, 172.717, 117.509, 7.10181, 4.50381, -4.01826, 1.54338, -8.07686, 7.79325, 0.242849, -0.430568, 2.11593, -0.569905, -1.95438, -0.186302, -2.62639, -0.66344, 0.287365, 0.0960486, 0.060306, 1.01928, 0.0387424, -1.19855, -0.197185, -0.298684, -0.0910158, -0.125318, -0.0463423, 0.0533237, -0.265746, -0.001024, 0.000431798, -0.0062599, -0.00415493, -0.0304479, -0.0233709, -0.0273382,0,0,0,0,0,0,0,0,0.915043,0,0.829461,0,0,0,1,1,0,0,0,0
3, 0.08, 0.955743, 1, 0.10642, 0.153098, -0.982464, -0.165396, 0.142754, -0.975841, 20.9377, -3.801, 420.152, 0.077066, -0.0420931, 0.0169737, 129.401, 129.594, 130.767, 132.676, 136.57, 143.019, 151.292, 161.226, 172.174, 182.559, 191.292, 198.798, 204.585, 208.236, 210.338, 211.708, 212.235, 137.757, 143.558, 151.557, 159.649, 166.934, 179.846, 187.29, 194.788, 202.024, 206.814, 173.596, 173.791, 174.003, 174.215, 164.574, 168.959, 173.436, 177.864, 181.811, 146.849, 151.61, 157.125, 161.785, 156.941, 151.406, 183.701, 188.703, 193.995, 198.159, 194.303, 189.084, 155.877, 162.898, 169.084, 173.14, 177.757, 183.416, 188.87, 183.418, 177.926, 173.048, 168.628, 162.526, 158.533, 168.889, 173.044, 177.718, 186.426, 177.674, 172.983, 168.774, 95.6202, 106.721, 117.895, 128.763, 139.03, 148.158, 155.763, 161.625, 163.27, 161.7, 156.224, 148.998, 140.187, 130.005, 119.171, 108.22, 97.0928, 88.0227, 83.0285, 81.6829, 82.9415, 85.9184, 85.7633, 82.9075, 82.0903, 83.7578, 88.7132, 93.5322, 100.648, 107.74, 115.111, 120.397, 121.981, 123.297, 122.083, 120.68, 94.6611, 91.9945, 92.1351, 95.5051, 96.4325, 96.4753, 95.7805, 92.3638, 92.5401, 95.2671, 96.8642, 96.7984, 134.514, 132.31, 131.265, 132.377, 131.409, 132.715, 134.797, 139.187, 140.946, 141.35, 140.907, 139.043, 134.785, 134.619, 135.093, 134.633, 135.065, 135.898, 136.382, 135.898, -57.7963, -57.5128, -55.4276, -51.8225, -44.1074, -31.5218, -15.7691, 2.1492, 21.1698, 39.8704, 57.2123, 72.7664, 84.7293, 91.9476, 95.6769, 97.8994, 98.9726, -38.7815, -28.1283, -14.2234, -0.582882, 11.3971, 32.7018, 45.4661, 58.8913, 72.3502, 82.0745, 22.3586, 22.2893, 22.2157, 22.161, 7.47888, 14.5276, 21.6631, 28.9891, 35.6869, -22.5903, -14.2881, -4.88292, 3.03171, -5.18665, -14.6055, 40.3788, 48.9382, 58.1852, 65.9022, 58.6493, 49.5365, -7.05484, 4.79916, 14.8453, 21.4504, 29.0745, 39.0269, 49.6099, 39.047, 29.3435, 21.2744, 14.0979, 4.19211, -2.49391, 14.595, 21.3746, 29.1584, 45.0794, 29.0195, 21.2248, 14.3849, -46.049, -25.1181, -3.99074, 16.6187, 35.8241, 52.2684, 64.7628, 72.9841, 75.2452, 73.6986, 66.2311, 54.3871, 38.3621, 19.0716, -1.57547, -22.3033, -43.4037, -55.7537, -63.2509, -64.5494, -61.5294, -56.0224, -56.4147, -61.7967, -64.1762, -62.3953, -54.8522, -43.5264, -31.2778, -19.4503, -7.62195, 0.649278, 3.21165, 5.31534, 3.37995, 1.11266, -43.5276, -47.692, -47.3305, -41.5955, -39.9576, -39.9819, -41.2621, -47.1186, -46.9998, -42.7146, -39.5565, -39.5174, 24.8362, 20.3861, 18.4102, 20.2054, 18.6809, 21.1924, 25.4271, 31.9927, 34.2885, 34.8108, 34.1632, 31.6067, 25.1329, 24.0016, 24.7326, 24.0811, 25.6989, 26.1037, 26.7825, 26.0654, 472.205, 472.875, 474.009, 474.141, 470.63, 464.069, 452.717, 438.346, 434.746, 441.841, 457.091, 468.883, 475.095, 476.549, 475.169, 473.324, 473.69, 435.885, 427.701, 421.153, 415.082, 410.943, 411.946, 416.504, 423.218, 430.406, 438.301, 411.126, 404.058, 396.624, 389.759, 408.75, 405.374, 403.087, 405.693, 409.045, 429.454, 425.738, 424.643, 424.533, 423.864, 424.892, 425.918, 426.24, 427.894, 431.76, 427.438, 425.804, 427.791, 414.009, 408.56, 408.111, 409.332, 416.673, 429.592, 416.845, 409.241, 407.624, 408.511, 414.931, 424.986, 410.466, 409.675, 411.426, 426.47, 410.485, 408.706, 409.877, 0.58296, 0.104298, -0.037298, 0.0164906, 172.547, 117.629, 5.15587, 4.96086, -4.71681, 3.75421, -7.39546, 7.4205, 0.815089, -0.930302, 3.05524, 0.635019, -1.79681, -0.262913, -2.66956, -1.19047, -0.168866, 0.441451, 0.0106802, 0.836421, -0.0732082, -1.31846, -0.009313, -0.427927, -0.152835, -0.169329, 0.00582317, -0.00231911, -0.302436, -0.00821701, -0.00528087, 0.00686356, -0.0275763, -0.0160663, -0.0206939, -0.0243734,0,0,0,0.585777,0,0,0,0,0.897447,0,0,0,0,0,0,1,0,0,0,0
4, 0.12, 0.963871, 1, 0.106936, 0.154229, -0.982231, -0.164536, 0.145976, -0.97551, 20.4166, -3.87284, 418.836, 0.0831857, -0.0378122, 0.0119711, 129.144, 129.391, 130.663, 132.677, 136.624, 143.065, 151.28, 161.147, 172.084, 182.496, 191.261, 198.78, 204.543, 208.15, 210.23, 211.575, 212.05, 137.348, 143.094, 151.069, 159.147, 166.416, 179.388, 186.845, 194.366, 201.622, 206.425, 173.146, 173.35, 173.57, 173.794, 164.306, 168.655, 173.097, 177.508, 181.441, 146.413, 151.144, 156.68, 161.364, 156.515, 150.961, 183.329, 188.301, 193.62, 197.804, 193.941, 188.694, 155.989, 162.766, 168.859, 172.893, 177.457, 183.089, 188.434, 183.124, 177.679, 172.837, 168.435, 162.43, 158.656, 168.69, 172.813, 177.439, 185.967, 177.405, 172.753, 168.574, 95.3783, 106.561, 117.817, 128.737, 139.072, 148.242, 155.899, 161.8, 163.445, 161.821, 156.208, 148.816, 139.888, 129.629, 118.745, 107.729, 96.5486, 87.8055, 82.8152, 81.471, 82.7331, 85.6975, 85.5142, 82.6243, 81.7406, 83.3467, 88.2501, 93.3341, 100.469, 107.574, 114.96, 120.346, 121.903, 123.189, 121.972, 120.548, 94.4962, 91.7979, 91.9157, 95.2878, 96.2555, 96.3212, 95.4683, 92.021, 92.1838, 94.8937, 96.5396, 96.4907, 134.93, 132.47, 131.274, 132.386, 131.388, 132.787, 135.047, 139.58, 141.426, 141.852, 141.42, 139.522, 135.15, 134.745, 135.186, 134.732, 135.304, 136.217, 136.719, 136.244, -58.0438, -57.7087, -55.4857, -51.7169, -43.9191, -31.3727, -15.7612, 2.0087, 20.9785, 39.6772, 56.9978, 72.5033, 84.3436, 91.4036, 95.0215, 97.1298, 98.0419, -39.3322, -28.8012, -14.9813, -1.41038, 10.5049, 31.8226, 44.5371, 57.9182, 71.3196, 80.9994, 21.5338, 21.4946, 21.453, 21.4328, 7.01797, 13.9888, 21.0505, 28.315, 34.9537, -23.2538, -15.0221, -5.6169, 2.30756, -5.88643, -15.3085, 39.5821, 48.0401, 57.2876, 65.0072, 57.7844, 48.6685, -6.83714, 4.56332, 14.4231, 20.9665, 28.4698, 38.3203, 48.6254, 38.432, 28.8791, 20.8921, 13.7578, 4.02238, -2.27471, 14.221, 20.9252, 28.5985, 44.0923, 28.4991, 20.7979, 14.0224, -46.3159, -25.3367, -4.128, 16.5375, 35.8332, 52.3181, 64.8855, 73.184, 75.4207, 73.7597, 66.0177, 53.8737, 37.6595, 18.2782, -2.37357, -23.1091, -44.1732, -55.9018, -63.3495, -64.6288, -61.615, -56.161, -56.6028, -62.0088, -64.4808, -62.8062, -55.3957, -43.6786, -31.4468, -19.644, -7.83114, 0.563514, 3.07607, 5.12585, 3.18886, 0.89298, -43.6502, -47.8403, -47.5113, -41.7962, -40.1093, -40.1018, -41.6224, -47.4935, -47.3978, -43.1723, -39.9416, -39.8749, 25.4465, 20.5698, 18.3556, 20.1422, 18.5721, 21.2223, 25.733, 32.5414, 35.0005, 35.5642, 34.9361, 32.3131, 25.6484, 24.13, 24.8013, 24.1595, 25.9861, 26.5539, 27.2651, 26.5642, 470.274, 471.331, 472.833, 473.199, 469.711, 463.123, 451.86, 437.701, 433.999, 440.928, 455.82, 467.397, 473.388, 474.575, 472.933, 470.82, 470.901, 434.094, 425.91, 419.352, 413.336, 409.306, 410.335, 414.766, 421.339, 428.38, 436.188, 409.5, 402.532, 395.218, 388.432, 407.42, 404.087, 401.832, 404.316, 407.564, 427.879, 424.085, 422.935, 422.83, 422.301, 423.394, 424.17, 424.368, 425.991, 429.895, 425.628, 424.033, 426.107, 412.374, 407.018, 406.555, 407.71, 414.922, 427.534, 415.492, 408.386, 406.883, 407.754, 413.801, 423.254, 409.126, 408.293, 409.981, 424.509, 409.361, 407.693, 408.843, 0.585041, 0.110759, -0.0323022, 0.0113965, 172.276, 117.569, 3.39147, 5.48401, -5.82276, 5.7464, -7.71578, 5.88891, 1.78466, -0.592173, 3.80212, 0.150893, -1.38067, -0.384341, -2.18357, -1.15011, -0.0751731, 0.465033, 0.123919, 0.467066, -0.0636545, -1.50489, 0.0495619, -0.382291, -0.102015, -0.140306, 0.0301354, 0.0119626, -0.296017, 0.00944414, -0.00441807, 0.00994939, -0.03512, -0.0213233, -0.0229672, -0.0172276,0,0,0,0.761611,0,0,0,0,0.859176,0,0,0,0,0,1,1,0,0,0,1
...
...
204, 8.12, 0.952574, 1, 0.121502, 0.17169, -0.97763, -0.143931, 0.155402, -0.97731, 12.8349, -5.59191, 410.328, 0.107749, -0.058971, 0.0502874, 124.423, 124.206, 125.189, 127.019, 130.81, 136.986, 144.871, 154.67, 165.76, 176.364, 185.205, 192.765, 198.453, 202.191, 204.701, 206.547, 207.38, 134.016, 140.133, 148.326, 156.506, 163.684, 176.924, 184.35, 191.732, 198.696, 202.84, 170.352, 170.333, 170.369, 170.413, 160.385, 164.655, 169.032, 173.437, 177.298, 142.986, 147.962, 153.561, 158.117, 153.19, 147.555, 179.885, 185.047, 190.369, 194.303, 190.481, 185.208, 151.2, 157.83, 164.439, 168.367, 172.788, 178.289, 182.76, 177.82, 172.299, 167.523, 163.095, 156.868, 153.929, 164.094, 168.118, 172.56, 180.245, 172.316, 167.75, 163.613, 90.7048, 102.714, 114.894, 126.637, 137.949, 148.032, 156.631, 163.26, 165.364, 163.631, 157.274, 148.879, 139.271, 128.646, 117.732, 106.652, 95.4064, 84.1519, 79.2084, 78.1078, 79.7568, 83.1749, 83.5542, 80.9658, 80.2812, 82.0667, 87.1396, 91.0586, 98.2875, 105.457, 112.959, 118.162, 119.885, 121.381, 120.315, 118.953, 91.1784, 88.6154, 88.9671, 92.5478, 93.3091, 93.1254, 93.5461, 90.3313, 90.7103, 93.5923, 95.0917, 94.802, 134.062, 130.64, 129.237, 130.557, 129.695, 131.766, 135.318, 141.588, 144.028, 144.359, 143.74, 140.793, 134.273, 133.083, 133.62, 133.43, 135.495, 138.078, 138.506, 137.853, -64.8277, -65.5949, -64.161, -60.9109, -53.5648, -41.7385, -26.877, -9.2042, 9.87829, 28.5026, 45.2353, 60.0876, 71.1772, 78.048, 82.2624, 85.2714, 86.7794, -43.6395, -32.8012, -19.0088, -5.62194, 5.88637, 27.2162, 39.6153, 52.4579, 65.0991, 73.379, 16.5598, 16.2532, 16.0267, 15.8265, 0.612278, 7.35013, 14.1969, 21.2636, 27.6129, -28.2466, -19.8299, -10.5916, -3.09777, -11.1908, -20.4795, 33.0152, 41.5994, 50.6684, 57.7878, 50.8295, 41.8479, -14.6597, -3.49515, 7.05118, 13.2859, 20.3825, 29.6938, 38.1083, 29.1565, 19.8406, 12.0906, 4.97844, -5.09143, -10.0578, 6.55086, 12.9695, 20.1658, 33.6984, 19.8808, 12.4544, 5.81502, -53.3805, -31.6783, -9.41097, 12.2573, 32.9367, 50.8391, 65.0743, 74.7071, 77.7971, 75.9963, 66.8945, 52.9602, 35.6703, 15.9935, -4.17446, -24.4526, -45.0452, -60.2068, -67.3474, -68.2115, -64.7472, -58.8435, -58.6091, -63.5065, -65.6617, -63.8166, -56.2858, -46.296, -34.1534, -22.4783, -10.7005, -2.92296, -0.18142, 2.17019, 0.499203, -1.67149, -47.8494, -51.6986, -51.0428, -45.1676, -43.8579, -44.2264, -43.9206, -49.2762, -48.8674, -44.4865, -41.5367, -41.8321, 23.4261, 17.136, 14.6729, 16.7639, 15.4535, 19.1032, 25.6484, 35.3208, 38.7598, 39.147, 38.1877, 33.8, 23.6442, 20.9363, 21.7583, 21.5635, 25.7923, 29.1829, 29.7385, 28.7325, 455.539, 458.139, 460.781, 461.714, 458.764, 453.398, 444.118, 431.729, 428.738, 435.451, 448.667, 458.469, 462.749, 462.473, 460.067, 457.984, 457.895, 419.875, 412.753, 407.066, 402.224, 399.48, 402.029, 406.735, 413.292, 420.584, 428.219, 399.911, 393.246, 386.418, 379.956, 397.658, 394.762, 392.961, 395.618, 399.086, 415.048, 411.815, 411.2, 411.329, 410.794, 411.415, 415.066, 415.22, 417.104, 421.152, 416.896, 415.034, 416.489, 402.647, 397.135, 396.991, 398.476, 405.901, 418.595, 409.035, 403.281, 401.767, 402.138, 406.381, 414.142, 400.053, 399.388, 401.396, 416.129, 403.568, 401.731, 402.343, 0.598208, 0.136351, -0.053165, 0.0496707, 167.876, 116.389, -13.2689, 13.6493, -1.02892, 14.3784, -10.5895, -2.9426, 6.2489, -2.82006, 5.56716, -3.2069, 0.360861, -0.513787, -1.28229, -0.941657, 1.25147, 0.0729805, 1.28252, 0.780729, -0.533899, -0.698888, 0.809073, -0.755656, 0.0321429, 0.0375847, 0.10858, 0.308759, 0.0772421, -0.0557184, -0.0312446, -0.00364208, -0.0493718, -0.00840708, -0.013717, 0.00441885,0,0,0,0,0,0.583841,0,0,0,0,0,0,2.76711,0.790546,0,0,0,1,0,0
205, 8.16, 0.943502, 1, 0.119735, 0.171356, -0.977906, -0.141987, 0.154965, -0.977663, 13.3425, -6.40826, 410.029, 0.0806764, -0.0697097, 0.0482961, 124.486, 124.252, 125.217, 127.038, 130.833, 137.022, 144.961, 154.903, 166.055, 176.638, 185.385, 192.861, 198.514, 202.261, 204.773, 206.598, 207.407, 134.295, 140.436, 148.657, 156.836, 163.983, 177.144, 184.535, 191.864, 198.783, 202.935, 170.605, 170.671, 170.785, 170.901, 160.762, 165.072, 169.495, 173.902, 177.752, 143.257, 148.209, 153.793, 158.343, 153.444, 147.829, 180.137, 185.263, 190.544, 194.433, 190.682, 185.455, 151.681, 158.442, 164.953, 168.876, 173.295, 178.673, 183.105, 178.271, 172.891, 168.138, 163.741, 157.567, 154.428, 164.603, 168.626, 173.064, 180.638, 172.853, 168.306, 164.182, 91.0214, 102.987, 115.135, 126.847, 138.101, 148.116, 156.591, 163.054, 165.089, 163.481, 157.366, 149.133, 139.554, 128.863, 117.816, 106.614, 95.2744, 83.855, 78.8175, 77.6544, 79.2258, 82.5482, 82.8669, 80.3027, 79.6619, 81.5165, 86.6856, 90.3551, 97.574, 104.74, 112.233, 117.593, 119.307, 120.781, 119.707, 118.348, 90.8022, 88.1684, 88.4913, 92.027, 92.8203, 92.6803, 92.9791, 89.7532, 90.1293, 93.0418, 94.4966, 94.2085, 133.686, 130.224, 128.796, 130.083, 129.227, 131.271, 134.808, 140.566, 142.765, 143.083, 142.495, 139.836, 133.868, 132.578, 133.092, 132.893, 134.958, 137.01, 137.427, 136.802, -64.7382, -65.4541, -63.9829, -60.708, -53.3381, -41.5021, -26.5929, -8.75591, 10.328, 28.8486, 45.416, 60.1852, 71.3281, 78.3458, 82.6714, 85.6934, 87.2226, -43.2659, -32.3908, -18.5333, -5.10962, 6.39056, 27.6794, 40.0855, 52.9093, 65.5315, 73.8434, 17.0189, 16.8272, 16.6984, 16.5816, 1.21236, 8.00656, 14.921, 22.0026, 28.3581, -27.8489, -19.4695, -10.2356, -2.73267, -10.7948, -20.0668, 33.5316, 42.1028, 51.1382, 58.1974, 51.3232, 42.3829, -13.8222, -2.50517, 7.8642, 14.0885, 21.1879, 30.3096, 38.654, 29.8141, 20.7193, 13.0256, 5.99305, -3.93975, -9.20443, 7.35589, 13.7624, 20.9548, 34.3077, 20.6938, 13.3089, 6.71097, -52.8242, -31.1508, -8.94923, 12.6099, 33.1017, 50.7809, 64.7007, 73.9557, 76.911, 75.3902, 66.8523, 53.3569, 36.2136, 16.4308, -4.03298, -24.6165, -45.4914, -60.8382, -68.1848, -69.1889, -65.8564, -60.0833, -59.9537, -64.8587, -66.9801, -65.0251, -57.2969, -47.5757, -35.3623, -23.6268, -11.8157, -3.8292, -1.09356, 1.22785, -0.464457, -2.63968, -48.5654, -52.5601, -51.9563, -46.1358, -44.7531, -45.0424, -44.9936, -50.408, -50.0112, -45.564, -42.6612, -42.9431, 22.7395, 16.4402, 13.9663, 16.0044, 14.7051, 18.2949, 24.7743, 33.5577, 36.5908, 36.9477, 36.0386, 32.118, 22.9081, 20.099, 20.889, 20.6815, 24.865, 27.3873, 27.9229, 26.9612, 455.718, 457.749, 459.875, 460.438, 457.173, 451.532, 442.055, 429.433, 426.435, 433.468, 447.274, 457.881, 463.006, 463.463, 461.618, 459.752, 459.962, 420.793, 413.919, 408.478, 403.787, 401.071, 403.641, 408.458, 415.117, 422.422, 429.971, 401.213, 394.212, 387.066, 380.294, 397.679, 394.668, 392.847, 395.674, 399.36, 415.831, 412.798, 412.237, 412.325, 411.641, 412.179, 416.285, 416.64, 418.564, 422.543, 418.192, 416.253, 415.375, 402.005, 396.972, 396.835, 398.429, 405.789, 418.246, 407.933, 401.833, 400.164, 400.515, 404.794, 412.97, 399.495, 398.885, 401.007, 415.585, 402.523, 400.559, 401.158, 0.598085, 0.11082, -0.0664424, 0.0479842, 168.181, 115.939, -11.9401, 12.1435, -2.53836, 14.045, -11.2765, -0.732664, 6.17934, -3.12388, 5.21925, -3.14636, 0.102961, -0.0954346, -1.78584, -1.12418, 1.58407, -0.0888273, 0.826367, 0.700256, -0.351087, -0.623023, 0.66073, -0.450211, -0.0178976, -0.175885, 0.205149, 0.308204, 0.0731635, -0.0274786, -0.0243863, -0.00930196, -0.0267284, -0.0130051, -0.0129202, 0.000738565,0.720321,0,0,0.522199,0,0.992203,0,0,0,0,0,0,3.04877,0.744715,1,0,0,1,0,0
The header specifies the meaning of each column. The explanation of each:
Basic
frame/face the number of the frame (in case of sequences) and the face (in case of individual images) processed
timestamp the timer of video being processed in seconds
confidence how confident is the tracker in current landmark detection estimage
success is the track successful (is there a face in the frame or do we think we tracked it well)
Gaze related
gaze_0_x, gaze_0_y, gaze_0_z Eye gaze direction vector in world coordinates for eye 0 (normalized)
gaze_1_x, gaze_1_y, gaze_1_z Eye gaze direction vector in world coordinates for eye 0 (normalized)
gaze_angle_x, gaze_angle_y Eye gaze direction in radians in world coordinates for both eyes and converted into more easy to use format than gaze vectors
eye_lmk_x_0, eye_lmk_x_1,... eye_lmk_x55, eye_lmk_y_1,... eye_lmk_y_55 location of 2D eye region landmarks in pixels. The landmark index can be found below
eye_lmk_X_0, eye_lmk_X_1,... eye_lmk_X55, eye_lmk_Y_0,... eye_lmk_Z_55 location of 3D eye region landmarks in pixels. The landmark index can be found below
Pose
pose_Tx, pose_Ty, pose_Tz the location of the head with respect to camera in millimetre (positive Z is away from the camera)
pose_Rx, pose_Ry, pose_Rz Rotation is in radians around X,Y,Z axes with the convention R = Rx * Ry * Rz, left-handed positive sign. This can be seen as pitch (Rx), yaw (Ry), and roll (Rz). The rotation is in world coordinates with camera being the origin.
Lines in au intensities and au occurrences correspond to predicted Action Unit presence and intensities respectively. For more details see here
Where the landmarks are no longer in pixel values but in millimetres and we also report head pose and gaze (this however needs accurate estimates of fx,fy,cx,cy. This functionality is useful for batch image processing where the camera is the same and we want to know pose and gaze.
Landmarks locations in 2D
x_0, x_1, ... x_66, x_67, y_0,...y_67 location of 2D landmarks in pixels, the landmark index can be seen below
Landmarks locations in 3D
X_0, ... X_67, Y_0,...Y_67, Z_0,...Z_67 location of 3D landmarks in millimetres, the landmark index can be seen below. For this to be accurate need to have good estimates for fx,fy,cx,cy
Rigid and non-rigid shape parameters
Parameters of a point distribution model (PDM) that describe the rigid face shape (location, scale and rotation) and non-rigid face shape (deformation due to expression and identity). For more details look at chapter 4.2 of my Thesis for more details.
p_scale, p_rx, p_ry, p_rz, p_tx, p_ty - scale, rotation and translation terms of the PDM
p_0, p_1, ... p_33 - non-rigid shape parameters
Facial Action Units
Facial Action Units (AUs) are a way to describe human facial expression, more details on Action Units can be found here
The system can detect the intensity (from 0 to 5) of 17 AUs:
AU01_r, AU02_r, AU04_r, AU05_r, AU06_r, AU07_r, AU09_r, AU10_r, AU12_r, AU14_r, AU15_r, AU17_r, AU20_r, AU23_r, AU25_r, AU26_r, AU45_r
And the presense (0 absent, 1 present) of 18 AUs:
AU01_c, AU02_c, AU04_c, AU05_c, AU06_c, AU07_c, AU09_c, AU10_c, AU12_c, AU14_c, AU15_c, AU17_c, AU20_c, AU23_c, AU25_c, AU26_c, AU28_c, AU45_c
The executables also allow for extraction of aligned faces (left) and HOG features (right). The HOG features are written as a binary file (for space consideration) and can be read in using the ./matlab_runners/Demos/Read_HOG_files.m script to Matlab, see ./matlab_runners/Demos/feature_extraction_demo_vid.m for an example. Once the script is finished, the HOG features are stored in hog_data variable
It is also possible to control the size of the similarity aligned faces, using the -simsize <pixel width and height> and -simscale <scale> command line arguments, you can get the following results:
Default parameters - size 112 and scale 0.7
Smaller scale - 0.5
Larger output - size 200 px
The executables also output the visualization of landmarks, head pose and eye gaze tracking as a video or an image.
Finally, a meta file is produced that contains the information about input and output.