Problem showing image captured by Kinecy using OpenCv

Nov 26, 2012 at 12:38 PM
Edited Nov 26, 2012 at 12:39 PM

I found some examples that show how to communicate PyKinect with Python OpenCV bindings, and tried to write a very simple one, which grabs an image from Kinect and shows it in a named window using OpenCV. The code is as follows:

import cv
import array
from pykinect import nui
if __name__ == '__main__':
	window = 'Kinect RGB'
	cv.NamedWindow(window)
	image = cv.CreateImage((640,480), cv.IPL_DEPTH_8U, 4)
	matrix = array.array('c')
	matrix.fromstring(image.tostring())
	address = matrix.buffer_info()[0]
	
	def video_frame_ready(frame):
		frame.image.copy_bits(address)
		cv.SetData(image, matrix.tostring())
		cv.ShowImage(window, image)
	
	kinect = nui.Runtime()
	kinect.video_frame_ready += video_frame_ready
	kinect.video_stream.open(nui.ImageStreamType.Video,
				2,
				nui.ImageResolution.Resolution640x480,
				nui.ImageType.Color)
	while cv.WaitKey(30) < 0:
		pass
	kinect.close()
	cv.DestroyAllWindows()

As you can see the code is very simple, however, when I try to run the same script, changing just the name of the window say to "Kinect", the scripts hangs and becomes nonresponsive. I have tested that on Visual Studio, Spyder and using the Python interpreter directly with the same results. Any ideas on what's going on here?

Coordinator
Nov 28, 2012 at 5:45 PM

I think the issue here is that video_frame_ready is calling into cv from a different thread from where you initialized opencv or created the window.  I changed this so that it just copies the frame into address (the copy needs to happen in video_frame_ready because the frame will be freed after the call), and then does the SetData/ShowImage inside of the while loop such as:

import cv
import array
from pykinect import nui
if __name__ == '__main__':
  window = 'Kinect RGB'
  cv.NamedWindow(window) 
 
  image = cv.CreateImage((640,480), cv.IPL_DEPTH_8U, 4)
  matrix = array.array('c')
  matrix.fromstring(image.tostring())
  address = matrix.buffer_info()[0]
 
  cur_image = None
  def video_frame_ready(frame):
    frame.image.copy_bits(address)

  kinect = nui.Runtime()
  kinect.video_frame_ready += video_frame_ready
  kinect.video_stream.open(nui.ImageStreamType.Video,
        2,
        nui.ImageResolution.Resolution640x480,
        nui.ImageType.Color)
 
  while cv.WaitKey(30) < 0:
    cv.SetData(image, matrix.tostring())
    cv.ShowImage(window, image)
   
  print('done waiting')
  kinect.close()
  cv.DestroyAllWindows()

 

And it looks like it's basically working.

Nov 29, 2012 at 9:11 AM
Edited Nov 29, 2012 at 12:14 PM

It sure works, thanks! 

Also, why to mimic the .NET event model instead of using Python signaling?