Python + opencv + ….. wait for it …+ Kinect

11 Dec

Yes, I am back again after a long time. At present I am working in implementing SLAM (which I’ll post in a few days). So, I started out with my environment Linux. Installed openKinect libraries for Kinect. There are other libraries as well but I have tried this one first and it installed smoothly no errors. Now I wrote a small piece of code to get started, yes in Python. The freenect Library has python wrappers, which are also easy to install. Following is the basic code for the newbies.

#######################################################################

import freenect
import cv
import numpy as np
cv.NamedWindow(‘RGB’)

def video_kin(dev,video, timestamp):
global run
print timestamp
cv.ShowImage(‘RGB’,cv.fromarray(video))
cv.WaitKey(10)

freenect.runloop(video=video_kin)

###############################################################

Explanation of Code :

The last Line of the code freenect is the actual backbone. This is the library provided by freenect people for python and has one of the method runloop. This will fetch the video data frame as numpy array. So , in order to show that image we need to first convert it to CvMat or other format.Yeah and most importantly don’t forget to run the code as sudo,  kinect won,t work unless you are a priveleged person.

If you have problems feel free to comment, I’ll try to resolve.

Advertisements

Installing Kinect Drivers in Ubuntu 12.04

10 Oct

After searching a lot on how to use kinect in ubuntu finall found the result. Follow the steps and you can find Kinect up and running in a few minutes.

Install the following one-by-one :

  1. sudo apt-get install git-core cmake freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev
  2. git clone git://github.com/OpenKinect/libfreenect.git

  3. cd libfreenect

  4. mkdir build

  5. cd build

  6. cmake ..

  7. make

  8. sudo make install

  9. sudo ldconfig /usr/local/lib64/

  10. sudo glview

  11. sudo adduser $USER video

 

Lucas Kanade ALgorithm

8 May

As the name suggests it was proposed by Lucas and Kanade and widely known as  LK tracking algorithm. This algorithm is used for tracking of objects in video as well as for stereo correspondence problem. It can be applied on all those use cases where the following assumptions hold true for a larger amount of time .

This algorithm is based on 3 assumptions

a. Brightness Constancy Equation : It means the pixel in concern follow the following equation  Ex*u + Ey*v = Et. Here, Ex is the derivative of E wrt x and similarly for Ey and Et. While ‘u’ and ‘v’ are velocity in x and y direction respectively wrt frame. Now question arises, what is E. Well E image Irradiance. For further info on what is E just Google it. 😉

b. Spatial Coherence :  It means that object does not break apart in next frame.So all the pixels of the object will have same velocity.

c. Last but most important, for video object tracking the object should move with very small distance in successive frames.

Based above assumptions, we now will derive the whole algorithm.Hey wait, I have cleared initial doubts now derive yourself.For reference go through relevant  paper published.

The following is its implementation.

####################################################################

import cv
import cv2
import numpy

# some constants and default parameters
lk_params = dict(winSize=(15,15),maxLevel=2,criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT,10,0.03))
subpix_params = dict(zeroZone=(-1,-1),winSize=(10,10),criteria = (cv2.TERM_CRITERIA_COUNT | cv2.TERM_CRITERIA_EPS,20,0.03))
feature_params = dict(maxCorners=500,qualityLevel=0.01,minDistance=5)
# setup video capture
cap = cv2.VideoCapture(0)
ret,im = cap.read()

gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
fourcc = cv.CV_FOURCC(‘I’,’4′,’2′,’0′)

while True:
ret,im = cap.read()
gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)

features = cv2.goodFeaturesToTrack(gray, **feature_params)
tracks = [[p] for p in features.reshape((-1,2))]

prev_gray = gray
if features != []:
ret,im_new = cap.read()
gray=cv2.cvtColor(im_new,cv2.COLOR_BGR2GRAY)
tmp = numpy.float32(features).reshape(-1, 1, 2)
features,status,track_error = cv2.calcOpticalFlowPyrLK(prev_gray,gray,tmp,None,**lk_params)
features = [p for (st,p) in zip(status,features) if st]
features = numpy.array(features).reshape((-1,2))
for i,f in enumerate(features):
tracks[i].append(f)
ndx = [i for (i,st) in enumerate(status) if not st]
ndx.reverse() #remove from back
for i in ndx:
tracks.pop(i)
prev_gray = gray
for point in features:
cv2.circle(im,(int(point[0]),int(point[1])),3,(0,255,0),-1)

cv2.imshow(‘LKtrack’,im)

cv2.waitKey(1)
#################################################################################

PS: check indentation

S.U.R.F.

2 May

Speed Up Robust Feature extraction algorithm. For those who all are hearing this algorithm for the first time please visit wiki and understand a bit about it.

Now, from name you can guess its something about features. Here I am gonna tell you about its implementation.The following is the code in python using Opencv.

############################################################

import cv2

cap = cv2.VideoCapture(0)
ret,img = cap.read()
#im2 = cv2.imread(‘ex1.jpg’)
#im = cv2.cvtColor(im2,cv2.COLOR_BGR2GRAY)
while True:
ret,img = cap.read()
im= cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
surfDetector = cv2.FeatureDetector_create(“SURF”)
surf = cv2.DescriptorExtractor_create(“SURF”)
keypoints = surfDetector.detect(im)
(l,d)= surf.compute(im,keypoints)

for kp in l:
x = int(kp.pt[0])
y = int(kp.pt[1])
cv2.circle(im,(x,y),2,(0,0,255))

cv2.imshow(“features”,im)
cv2.waitKey(1)

#############################################################

Android and Open CV

22 Jan

Open CV cannot only be ported on pc running Ubuntu or windows but also there are people doing projects porting Open CV on different platforms.One of the most famous is porting Open CV to android and develop application using it.So, this post is regarding “Porting Open CV on Android”

For those who doesn’t even slightest bit of what Android is and how the applications are developed over Dont worry just read a little bit from Here.By this point I assume that you know basics of Android. 🙂

Here is one of the detailed process of how to port. This from one the student of Stanford University.Actually Stanford University started this project a long back and have a dedicated work force over it. Please follow through the process. Later on I will be uploading a smaller process of porting but currently this is the  standard way.

Application of Android and open CV are great and quite challenging. But motivation behind doing  is the portability. 😉

Install open CV in ubuntu:”Simplest way”

22 Jan

I have been searching through the wide domain of Internet on how to install open CV and found many blogs with atleast 8-10 steps.If you miss one or if it doesnt work its really frustrating 😛 . 

So, after this search found the simplest way

1. sudo apt-get install libopencv-*

if you want python integration also

2. sudo apt-get install python-opencv

3. sudo apt-get install python-numpy  

# I have installed opencv in ubuntu 11.10 running on beagleboard-xm so if you find problem post it as comment. I’ll try solving it.

Install OpenCV in windows

5 Jan

Well, I am not an OS dependent guy.So, always switch from ubuntu-> windows -> Debian and lots more.Why restrict ourselves for developing also.

Being said that, in this post there is link to install OpenCv in windows for python. This link is the easiest way to install, instead of downloading visual studio C++ and the combining it with OpenCV  

http://opencvpython.blogspot.in/2012/05/install-opencv-in-windows-for-python.html” 

I don’t own anything and I am internet wanderer.