Augmented Reality


Here’s another of my small projects. Basically it’s just a small Augmented Reality program, it loads OBJ file and show the 3d object on a given marker using OpenCV.

Here’s the code which have been extended to a crappy ‘swat the mosquito’ game : But you can still salvage the code and see how the whole thing works.


Kinect-based Medical Image Viewer


This is from one of my master project. So what happened is that, My supervisor and I talked about how to leverage the use of Kinect in operating rooms, providing a hands free solution (thus more sterile) for surgeons when viewing thru X-Rays or MRI images. It was my first foray into OpenCV and OpenNI (open source SDK for Kinect).

In nutshell the application operates like this:

  • Use OpenNI to obtain skeleton and joint data from Kinect, from which I know the positions of wrists, shoulder, etc.
  • Since we know the position of wrist, I can roughly approximate where the palms are. So rather than trying to detect from the whole image, I detect the appearance (or the lack of) fingers in that region.
  • OpenCV is used to extract the hand region, apply polylines, and check the convexity of that shape. Thus I can figure out whether I am making a knuckle or open palm gesture.
  • The images is shown using OpenGL (and if I remembered correctly I used VTK / DICOM libraries to extract the MRI images)
  • I added some logic to emulate finger gestures on mobile phones, but this time using the palms instead. Each closed palm gesture is treated as if you’re holding down a mouse button, and open palm means letting the button go.

Here’s the first prototype to show that I could detect fingertips

Then it was extended so that it could manipulate images