Skip to content

Refactor capture and transformation handles #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 84 commits into from
Sep 17, 2020
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
2535591
Simple test
shagren Jul 23, 2020
29716b2
More functions supported
shagren Aug 4, 2020
009148a
Merge pull request #1 from shagren/threading-gil-releasing
shagren Aug 4, 2020
68e9505
Fixes, Example
shagren Aug 5, 2020
0b2ab2f
Merge pull request #2 from shagren/threading-gil-releasing
shagren Aug 5, 2020
21f38a2
Merge remote-tracking branch 'upstream/develop' into develop
shagren Aug 10, 2020
516bfb1
Merge remote-tracking branch 'upstream/develop' into develop
shagren Aug 10, 2020
1b0c0b1
Merge remote-tracking branch 'upstream/develop' into develop
shagren Aug 19, 2020
3fa1349
Merge remote-tracking branch 'upstream/develop' into develop
shagren Aug 26, 2020
889072f
Add code quality tool
shagren Aug 26, 2020
6411a4a
Add .mypy_cache to .gitignore
shagren Aug 26, 2020
58005f0
Make line length 120 chars
shagren Aug 31, 2020
c94b0da
Initial playback support
shagren Aug 28, 2020
1e88ed5
Seek support
shagren Aug 28, 2020
d0264c2
Add tests
shagren Aug 28, 2020
44982bb
Format tests too
shagren Aug 28, 2020
fa2323d
Remove commented code
shagren Aug 28, 2020
04b0639
Smaller asset.
shagren Aug 31, 2020
52c6a64
Fix build
shagren Sep 1, 2020
e66dad4
Remove debug
shagren Sep 1, 2020
4af8758
Validate if open() method called twice
shagren Sep 1, 2020
9a7765a
Remove unused file placeholder
shagren Sep 1, 2020
146536f
Better typing
shagren Sep 1, 2020
da8fdd7
Merge branch 'develop' into mkv-support-3
lpasselin Sep 1, 2020
1264175
Merge branch 'develop' into mkv-support-3
shagren Sep 3, 2020
60f892b
Merge develop
shagren Sep 3, 2020
b5ce7b2
CI changes
shagren Sep 3, 2020
f6efd0a
CI changes
shagren Sep 3, 2020
152a38a
CI changes
shagren Sep 3, 2020
904aa25
CI changes
shagren Sep 3, 2020
ec25eac
CI changes
shagren Sep 3, 2020
04ce188
CI changes
shagren Sep 3, 2020
f6ca706
Some changes in tests definitions
shagren Sep 3, 2020
8009436
Fix Maklefile
shagren Sep 3, 2020
cbacd61
Fix
shagren Sep 3, 2020
ce95f04
Update playback_seek_timestamp function description
shagren Sep 3, 2020
71b8871
Typofix
shagren Sep 3, 2020
f769915
Update readme with small details.
lpasselin Sep 3, 2020
e5e5f98
rename _thread_safe to thread_safe
lpasselin Sep 3, 2020
e0f54a1
reformat line for readability
lpasselin Sep 3, 2020
36e63c9
force ubuntu-18.04. Version supported by SDK
lpasselin Sep 3, 2020
1c09d70
WIP
shagren Sep 1, 2020
1d938f9
WIP
shagren Sep 1, 2020
f091db3
Fix build
shagren Sep 1, 2020
11cabd3
Rebase mkv-support-3
shagren Sep 1, 2020
79f3960
Refactor tests
shagren Sep 1, 2020
063eacf
WIP: color controls
shagren Sep 2, 2020
337eed6
More color-module tests
shagren Sep 3, 2020
9f9125d
Refactor device_get_color_control_capabilities()
shagren Sep 3, 2020
c9f22ea
WIP: IMU support
shagren Sep 3, 2020
d6599fe
Merge remote-tracking branch 'upstream/develop' into refactor-transfo…
shagren Sep 3, 2020
0569641
CI fix
shagren Sep 3, 2020
cdec3e7
start/stop cameras support
shagren Sep 3, 2020
bed83b1
device_get_capture support
shagren Sep 3, 2020
c707908
Remove debug lines
shagren Sep 3, 2020
29d50b2
support of get_capture, get_imu_sample
shagren Sep 4, 2020
c4e4448
WIP: Calibration
shagren Sep 4, 2020
b1e2a11
WIP: Capture
shagren Sep 4, 2020
444e4b1
Support device_get_raw_calibration
shagren Sep 7, 2020
471b236
Merge branch 'develop' into refactor-transformation
shagren Sep 7, 2020
5ed8db0
Support creating calibration from json file
shagren Sep 7, 2020
b447310
convert_3d_to_3d support
shagren Sep 7, 2020
4757c0b
calibration_2d_to_3d support
shagren Sep 8, 2020
8dfb34d
WIP: Transformations support
shagren Sep 8, 2020
0798eba
Transformation functions
shagren Sep 9, 2020
b2ad1d5
Refactor transformation
shagren Sep 9, 2020
97ef4bc
Refactor examples
shagren Sep 10, 2020
2be3314
Add benchmark example
shagren Sep 10, 2020
8289354
Fix tests
shagren Sep 10, 2020
ac6ca02
Better playback example
shagren Sep 10, 2020
55145a5
Fix playback example
shagren Sep 10, 2020
7dd0034
Rollback some text changes
shagren Sep 10, 2020
5dfd125
rename capsule_xxxx_name to CAPSULE_XXXX_NAME
shagren Sep 11, 2020
8ede0c7
Text fix
shagren Sep 11, 2020
af0a144
Typo fix
shagren Sep 11, 2020
de36550
Refactor examples
shagren Sep 11, 2020
cc99d74
CR changes
shagren Sep 11, 2020
622092e
CR changes
shagren Sep 11, 2020
2013b63
CR changes
shagren Sep 11, 2020
8538aca
CR changes
shagren Sep 11, 2020
c95b9db
CR changes
shagren Sep 11, 2020
7cb043d
add py.typed to distribution
shagren Sep 15, 2020
9e37a8f
remove not required _start_imu() call
lpasselin Sep 17, 2020
d58ba19
fix PytestAssertRewriteWarning
lpasselin Sep 17, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions example/bench.py → example/benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ def __call__(self, parser, namespace, values, option_string=None):

def parse_args() -> Namespace:
parser = ArgumentParser(
description="Bench camera captures transfer speed. \n"
description="Camera captures transfer speed benchmark. \n"
"You can check if you USB controller/cable has enough performance."
)
parser.add_argument("--device-id", type=int, default=0, help="Device ID, from zero. Default: 0")
Expand All @@ -74,7 +74,7 @@ def parse_args() -> Namespace:
help="Color sensor resoultion. Default: 720P",
)
parser.add_argument(
"--color-color_format",
"--color-format",
type=ImageFormat,
action=EnumActionTuned,
default=ImageFormat.COLOR_BGRA32,
Expand Down Expand Up @@ -115,7 +115,7 @@ def parse_args() -> Namespace:

def bench(config: Config, device_id: int):
device = PyK4A(config=config, device_id=device_id)
device.connect()
device.start()
depth = color = depth_period = color_period = 0
print("Press CTRL-C top stop benchmark")
started_at = started_at_period = monotonic()
Expand All @@ -138,7 +138,7 @@ def bench(config: Config, device_id: int):
except KeyboardInterrupt:
break
elapsed = monotonic() - started_at
device.disconnect()
device.stop()
print()
print(f"Result: Color: {color / elapsed:0.2f} FPS, Depth: {depth / elapsed: 0.2f} FPS")

Expand Down
28 changes: 6 additions & 22 deletions example/color_formats.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,44 +2,28 @@
import numpy as np

import pyk4a
from helpers import convert_to_bgra_if_required
from pyk4a import Config, PyK4A


def get_color_image_size(config, imshow=True):
if imshow:
cv2.namedWindow("k4a")
k4a = PyK4A(config)
k4a.connect()
k4a.start()
count = 0
while count < 60:
capture = k4a.get_capture()
if np.any(capture.color):
count += 1
if imshow:
cv2.imshow("k4a", convert_to_bgra_if_required(k4a, capture.color))
cv2.imshow("k4a", convert_to_bgra_if_required(config.color_format, capture.color))
cv2.waitKey(10)
cv2.destroyAllWindows()
k4a.disconnect()
k4a.stop()
return capture.color.nbytes


def convert_to_bgra_if_required(k4a, img_color):
# examples for all possible pyk4a.ColorFormats
if k4a._config.color_format == pyk4a.ImageFormat.COLOR_MJPG:
img_color = cv2.imdecode(img_color, cv2.IMREAD_COLOR)
elif k4a._config.color_format == pyk4a.ImageFormat.COLOR_NV12:
img_color = cv2.cvtColor(img_color, cv2.COLOR_YUV2BGRA_NV12)
# this also works and it explains how the COLOR_NV12 color color_format is stored in memory
# h, w = color_image.shape[0:2]
# h = h // 3 * 2
# luminance = color_image[:h]
# chroma = color_image[h:, :w//2]
# color_image = cv2.cvtColorTwoPlane(luminance, chroma, cv2.COLOR_YUV2BGRA_NV12)
elif k4a._config.color_format == pyk4a.ImageFormat.COLOR_YUY2:
img_color = cv2.cvtColor(img_color, cv2.COLOR_YUV2BGRA_YUY2)
return img_color


if __name__ == "__main__":
imshow = True
config_BGRA32 = Config(color_format=pyk4a.ImageFormat.COLOR_BGRA32)
Expand All @@ -52,8 +36,8 @@ def convert_to_bgra_if_required(k4a, img_color):
nbytes_NV12 = get_color_image_size(config_NV12, imshow=imshow)
nbytes_YUY2 = get_color_image_size(config_YUY2, imshow=imshow)

print(f"{nbytes_BGRA32} {nbytes_MJPG}")
print(f"BGRA32 is {nbytes_BGRA32/nbytes_MJPG} larger")
print(f"BGRA32: {nbytes_BGRA32}, MJPG: {nbytes_MJPG}, NV12: {nbytes_NV12}, YUY2: {nbytes_YUY2}")
print(f"BGRA32 is {nbytes_BGRA32/nbytes_MJPG:0.2f} larger than MJPG")

# output:
# nbytes_BGRA32=3686400 nbytes_MJPG=229693
Expand Down
37 changes: 37 additions & 0 deletions example/helpers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
from typing import Optional, Tuple

import cv2
import numpy as np

from pyk4a import ImageFormat


def convert_to_bgra_if_required(color_format: ImageFormat, color_image):
# examples for all possible pyk4a.ColorFormats
if color_format == ImageFormat.COLOR_MJPG:
color_image = cv2.imdecode(color_image, cv2.IMREAD_COLOR)
elif color_format == ImageFormat.COLOR_NV12:
color_image = cv2.cvtColor(color_image, cv2.COLOR_YUV2BGRA_NV12)
# this also works and it explains how the COLOR_NV12 color color_format is stored in memory
# h, w = color_image.shape[0:2]
# h = h // 3 * 2
# luminance = color_image[:h]
# chroma = color_image[h:, :w//2]
# color_image = cv2.cvtColorTwoPlane(luminance, chroma, cv2.COLOR_YUV2BGRA_NV12)
elif color_format == ImageFormat.COLOR_YUY2:
color_image = cv2.cvtColor(color_image, cv2.COLOR_YUV2BGRA_YUY2)
return color_image


def colorize(
image: np.ndarray,
clipping_range: Tuple[Optional[int], Optional[int]] = (None, None),
colormap: int = cv2.COLORMAP_HSV,
) -> np.ndarray:
if clipping_range[0] or clipping_range[1]:
img = image.clip(clipping_range[0], clipping_range[1])
else:
img = image.copy()
img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
img = cv2.applyColorMap(img, colormap)
return img
4 changes: 2 additions & 2 deletions example/imu.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def main():
synchronized_images_only=True,
)
)
k4a.connect()
k4a.start()
k4a._start_imu()

plt.ion()
Expand Down Expand Up @@ -75,7 +75,7 @@ def main():
fig.canvas.flush_events()

k4a._stop_imu()
k4a.disconnect()
k4a.stop()


if __name__ == "__main__":
Expand Down
36 changes: 2 additions & 34 deletions example/playback.py
Original file line number Diff line number Diff line change
@@ -1,41 +1,9 @@
from argparse import ArgumentParser
from typing import Optional, Tuple

import cv2
import numpy as np

from pyk4a import ImageFormat, PyK4APlayback


def colorize(
image: np.ndarray,
clipping_range: Tuple[Optional[int], Optional[int]] = (None, None),
colormap: int = cv2.COLORMAP_HSV,
) -> np.ndarray:
if clipping_range[0] or clipping_range[1]:
img = image.clip(clipping_range[0], clipping_range[1])
else:
img = image.copy()
img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
img = cv2.applyColorMap(img, colormap)
return img


def convert_to_bgra_if_required(color_format, color_image):
# examples for all possible pyk4a.ColorFormats
if color_format == ImageFormat.COLOR_MJPG:
color_image = cv2.imdecode(color_image, cv2.IMREAD_COLOR)
elif color_format == ImageFormat.COLOR_NV12:
color_image = cv2.cvtColor(color_image, cv2.COLOR_YUV2BGRA_NV12)
# this also works and it explains how the COLOR_NV12 color color_format is stored in memory
# h, w = color_image.shape[0:2]
# h = h // 3 * 2
# luminance = color_image[:h]
# chroma = color_image[h:, :w//2]
# color_image = cv2.cvtColorTwoPlane(luminance, chroma, cv2.COLOR_YUV2BGRA_NV12)
elif color_format == ImageFormat.COLOR_YUY2:
color_image = cv2.cvtColor(color_image, cv2.COLOR_YUV2BGRA_YUY2)
return color_image
from helpers import colorize, convert_to_bgra_if_required
from pyk4a import PyK4APlayback


def info(playback: PyK4APlayback):
Expand Down
4 changes: 2 additions & 2 deletions example/threads.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,13 @@ def __init__(self, device_id=0, thread_safe: bool = True):
def run(self) -> None:
print("Start run")
camera = PyK4A(device_id=self._device_id, thread_safe=self.thread_safe)
camera.connect()
camera.start()
while not self._halt:
capture = camera.get_capture()
assert capture.depth is not None
self._count += 1
sleep(0.1)
camera.disconnect()
camera.stop()
del camera
print("Stop run")

Expand Down
4 changes: 2 additions & 2 deletions example/viewer.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ def main():
synchronized_images_only=True,
)
)
k4a.connect()
k4a.start()

# getters and setters directly get and set on device
k4a.whitebalance = 4500
Expand All @@ -29,7 +29,7 @@ def main():
if key != -1:
cv2.destroyAllWindows()
break
k4a.disconnect()
k4a.stop()


if __name__ == "__main__":
Expand Down
13 changes: 4 additions & 9 deletions example/viewer_depth.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import numpy as np

import pyk4a
from helpers import colorize
from pyk4a import Config, PyK4A


Expand All @@ -13,7 +14,7 @@ def main():
synchronized_images_only=False,
)
)
k4a.connect()
k4a.start()

# getters and setters directly get and set on device
k4a.whitebalance = 4500
Expand All @@ -24,18 +25,12 @@ def main():
while True:
capture = k4a.get_capture()
if np.any(capture.depth):
# Clip 5000 mm(5 meters)
clipped_depth = capture.depth.clip(None, 5000)
# normalize and convert to 8bit
normalized_depth = cv2.normalize(clipped_depth, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
# coloring image by choosed color map
colored_depth = cv2.applyColorMap(normalized_depth, cv2.COLORMAP_HSV)
cv2.imshow("k4a", colored_depth)
cv2.imshow("k4a", colorize(capture.depth, (None, 5000), cv2.COLORMAP_HSV))
key = cv2.waitKey(10)
if key != -1:
cv2.destroyAllWindows()
break
k4a.disconnect()
k4a.stop()


if __name__ == "__main__":
Expand Down
4 changes: 2 additions & 2 deletions example/viewer_point_cloud.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def main():
synchronized_images_only=True,
)
)
k4a.connect()
k4a.start()

# getters and setters directly get and set on device
k4a.whitebalance = 4500
Expand Down Expand Up @@ -47,7 +47,7 @@ def main():
ax.view_init(elev=-90, azim=-90)
plt.show()

k4a.disconnect()
k4a.stop()


if __name__ == "__main__":
Expand Down
22 changes: 3 additions & 19 deletions example/viewer_transformation.py
Original file line number Diff line number Diff line change
@@ -1,29 +1,13 @@
from typing import Optional, Tuple

import cv2
import numpy as np

import pyk4a
from helpers import colorize
from pyk4a import Config, PyK4A


def colorize(
image: np.ndarray,
clipping_range: Tuple[Optional[int], Optional[int]] = (None, None),
colormap: int = cv2.COLORMAP_HSV,
) -> np.ndarray:
if clipping_range[0] or clipping_range[1]:
img = image.clip(clipping_range[0], clipping_range[1])
else:
img = image.copy()
img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
img = cv2.applyColorMap(img, colormap)
return img


def main():
k4a = PyK4A(Config(color_resolution=pyk4a.ColorResolution.RES_720P, depth_mode=pyk4a.DepthMode.NFOV_UNBINNED,))
k4a.connect()
k4a.start()

while True:
capture = k4a.get_capture()
Expand All @@ -43,7 +27,7 @@ def main():
cv2.destroyAllWindows()
break

k4a.disconnect()
k4a.stop()


if __name__ == "__main__":
Expand Down
10 changes: 4 additions & 6 deletions pyk4a/calibration.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ class Calibration:
def __init__(
self, handle: object, depth_mode: DepthMode, color_resolution: ColorResolution, thread_safe: bool = True
):
self._handle = handle
self._calibration_handle = handle
self._transformation_handle: Optional[object] = None
self.thread_safe = thread_safe
self._depth_mode = depth_mode
Expand Down Expand Up @@ -59,9 +59,8 @@ def _convert_3d_to_3d(
:param target_camera The target camera.
:return The 3D coordinates in millimeters representing a point in target camera.
"""
# Device needs to be running for the functions to work
res, target_point_3d = k4a_module.calibration_3d_to_3d(
self._handle, self.thread_safe, source_point_3d, source_camera, target_camera,
self._calibration_handle, self.thread_safe, source_point_3d, source_camera, target_camera,
)

_verify_error(res)
Expand Down Expand Up @@ -89,9 +88,8 @@ def _convert_2d_to_3d(
:param target_camera The target camera.
:return The 3D coordinates in mm representing a point in target camera.
"""
# Device needs to be running for the functions to work
res, valid, target_point_3d = k4a_module.calibration_2d_to_3d(
self._handle, self.thread_safe, source_pixel_2d, source_depth, source_camera, target_camera,
self._calibration_handle, self.thread_safe, source_pixel_2d, source_depth, source_camera, target_camera,
)

_verify_error(res)
Expand All @@ -117,7 +115,7 @@ def convert_2d_to_3d(
@property
def transformation_handle(self) -> object:
if not self._transformation_handle:
handle = k4a_module.transformation_create(self._handle, self.thread_safe)
handle = k4a_module.transformation_create(self._calibration_handle, self.thread_safe)
if not handle:
raise K4AException("Cannot create transformation handle")
self._transformation_handle = handle
Expand Down
2 changes: 1 addition & 1 deletion pyk4a/errors.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from .result import Result
from .results import Result


class K4AException(Exception):
Expand Down
16 changes: 1 addition & 15 deletions pyk4a/playback.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,21 +15,7 @@
from .capture import PyK4ACapture
from .config import FPS, ColorResolution, DepthMode, ImageFormat, WiredSyncMode
from .errors import K4AException, _verify_error
from .result import Result


# k4a_buffer_result_t
class BufferResult(IntEnum):
Success = 0
Failed = 1
TooSmall = 2


# k4a_stream_result_t
class StreamResult(IntEnum):
Success = 0
Failed = 1
EOF = 2
from .results import Result, StreamResult


class SeekOrigin(IntEnum):
Expand Down
Loading