Add a depth camera example

[srmainwaring]

🎉 New feature

This PR adds an example that demonstrates how to use the DepthCamera.

Summary

This is a standalone example that demonstrates how to render a scene using a depth camera. It runs using either the ogre or ogre2 render engines.

The code for converting the depth camera image to a RGB grayscale format is adapted from ign-sensors/src/DepthCameraSensor.cc.

Test it

The tutorial explains how to build and run the example in detail. Once built the example is run from the command line:

# run with the default render engine (ogre2)
./depth_camera

# run with the ogre render engine
./depth_camera ogre

# run with the ogre2 render engine and metal render system
./depth_camera ogre2 metal

The example accepts an additional command line argument to specify the render system. This will only take effect once #463 is merged. The fallback behaviour is to use the default render system GL3+.

Checklist

• Signed all commits for DCO
• Added tests
• Added example and/or tutorial
• Updated documentation (as needed)
• Updated migration guide (as needed)
• codecheck passed (See contributing)
• All tests passed (See test coverage)
• While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge

[codecov]

Codecov Report

Merging #467 (13c90a5) into ign-rendering6 (5a5cdf1) will not change coverage.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           ign-rendering6     #467   +/-   ##
===============================================
  Coverage           53.58%   53.58%           
===============================================
  Files                 194      194           
  Lines               19669    19669           
===============================================
  Hits                10539    10539           
  Misses               9130     9130           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a5cdf1...13c90a5. Read the comment docs.

[iche033]

the demo works for me. I tested this on Ubuntu with both ogre and ogre2. View control does not work well with a depth camera though. This is expected as we also ran into this problem with the segmentation camera demo. I think this can be fixed later.

I just have some minor suggestions about adding more info in the tutorial about the RGBA FLOAT texture format.

[srmainwaring]

View control does not work well with a depth camera though.

I'll take a quick look at this. I copied the mouse handling from the segmentation camera example, but realise this has some comments about it effectiveness. I think the mouse handling in the ogre2demo works well, so I'll look into updating this example based on that.

Update

Ok - I see it's the same code as used in ogre2demo reduced to one camera. So no quick solution there. If it is acceptable to come back to this in the camera examples where the ray query is raising an exception I'll leave it as is.

The problem with the mouse handling stems from the OgreRayQuery::SetFromCamera method which attempts to cast a CameraPtr to an OgreCameraPtr so it can extract the internal ogreCamera pointer. However since an OgreDepthCamera is not an OgreCamera this cast fails, even though both objects contain a protected data member Ogre::Camera *ogreCamera.

For example we could fix the mouse handling for ogre using:

diff --git a/ogre/include/ignition/rendering/ogre/OgreThermalCamera.hh b/ogre/include/ignition/rendering/ogre/OgreThermalCamera.hh
index 35e763af..a4be9ac8 100644
--- a/ogre/include/ignition/rendering/ogre/OgreThermalCamera.hh
+++ b/ogre/include/ignition/rendering/ogre/OgreThermalCamera.hh
@@ -112,6 +112,7 @@ namespace ignition
       private: std::unique_ptr<OgreThermalCameraPrivate> dataPtr;
 
       private: friend class OgreScene;
+      private: friend class OgreRayQuery;
     };
     }
   }
diff --git a/ogre/src/OgreRayQuery.cc b/ogre/src/OgreRayQuery.cc
index a6ffe43d..cd451414 100644
--- a/ogre/src/OgreRayQuery.cc
+++ b/ogre/src/OgreRayQuery.cc
@@ -22,8 +22,10 @@
 #include "ignition/rendering/ogre/OgreIncludes.hh"
 #include "ignition/rendering/ogre/OgreCamera.hh"
 #include "ignition/rendering/ogre/OgreConversions.hh"
+#include "ignition/rendering/ogre/OgreDepthCamera.hh"
 #include "ignition/rendering/ogre/OgreRayQuery.hh"
 #include "ignition/rendering/ogre/OgreScene.hh"
+#include "ignition/rendering/ogre/OgreThermalCamera.hh"
 
 class ignition::rendering::OgreRayQueryPrivate
 {
@@ -51,9 +53,29 @@ void OgreRayQuery::SetFromCamera(const CameraPtr &_camera,
 {
   // convert to nomalized screen pos for ogre
   math::Vector2d screenPos((_coord.X() + 1.0) / 2.0, (_coord.Y() - 1.0) / -2.0);
-  OgreCameraPtr camera = std::dynamic_pointer_cast<OgreCamera>(_camera);
-  Ogre::Ray ray =
+
+  // TODO(anyone) check the cast is valid, and deal with other camera types.
+  Ogre::Ray ray;
+  
+  if (OgreCameraPtr camera = std::dynamic_pointer_cast<OgreCamera>(_camera))
+  {
+    ray =
+      camera->ogreCamera->getCameraToViewportRay(screenPos.X(), screenPos.Y());
+  }
+  else if (OgreDepthCameraPtr camera = std::dynamic_pointer_cast<OgreDepthCamera>(_camera))
+  {
+    ray =
       camera->ogreCamera->getCameraToViewportRay(screenPos.X(), screenPos.Y());
+  }
+  else if (OgreThermalCameraPtr camera = std::dynamic_pointer_cast<OgreThermalCamera>(_camera))
+  {
+    ray =
+      camera->ogreCamera->getCameraToViewportRay(screenPos.X(), screenPos.Y());
+  }
+  else
+  {
+    // raise exception
+  }
 
   this->origin = OgreConversions::Convert(ray.getOrigin());
   this->direction = OgreConversions::Convert(ray.getDirection());

at the cost of introducing multiple dynamic_pointer_casts.

[srmainwaring]

View control does not work well with a depth camera though. This is expected as we also ran into this problem with the segmentation camera demo. I think this can be fixed later.

Here's a commit containing this approach to fixing the view control for the depth, segmentation and thermal cameras: srmainwaring@2972a98

[iche033]

looks good to me :)