Cropping the results from UIImagePickerController

I recently had to write some code to use the UIImagePickerControllerCropRect when picking or taking a photo. Looking around the web there were some pretty crazy coding examples that seemed to be unnecessarily complicated so I knocked up my own quick solution.

// ger the original image along with it's size
UIImage *image = [info objectForKey:UIImagePickerControllerOriginalImage];
CGSize size = image.size;
    
// crop the crop rect that the user selected
CGRect cropRect = [[info objectForKey:UIImagePickerControllerCropRect] 
                  CGRectValue];

// create a graphics context of the correct size    
UIGraphicsBeginImageContext(cropRect.size);
CGContextRef context = UIGraphicsGetCurrentContext();

// correct for image orientation    
UIImageOrientation orientation = [image imageOrientation];
if(orientation == UIImageOrientationUp) {
  CGContextTranslateCTM(context, 0, size.height);
  CGContextScaleCTM(context, 1, -1);                
  cropRect = CGRectMake(cropRect.origin.x, 
                        -cropRect.origin.y, 
                        cropRect.size.width, 
                        cropRect.size.height);
} else if(orientation == UIImageOrientationRight) {
  CGContextScaleCTM(context, 1.0, -1.0);
  CGContextRotateCTM(context, -M_PI/2);
  size = CGSizeMake(size.height, size.width);
  cropRect = CGRectMake(cropRect.origin.y, 
                        cropRect.origin.x, 
                        cropRect.size.height, 
                        cropRect.size.width);
} else if(orientation == UIImageOrientationDown) {
  CGContextTranslateCTM(context, size.width, 0);
  CGContextScaleCTM(context, -1, 1);        
  cropRect = CGRectMake(-cropRect.origin.x, 
                        cropRect.origin.y, 
                        cropRect.size.width, 
                        cropRect.size.height);
}
// draw the image in the correct place
CGContextTranslateCTM(context, -cropRect.origin.x, -cropRect.origin.y);
CGContextDrawImage(context, 
                   CGRectMake(0,0, size.width, size.height), 
                   image.CGImage);
// and pull out the cropped image
UIImage *croppedImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();

GameKit and GameCenter

Had lots of fun at iOSDevUK. Excellent conference with some really interesting talks. And I even managed to give a talk! My slides along with notes can be downloaded from here. It's all about making real time multiplayer games using GameKit and GameCenter.

Tanks! will soon be removed from the US app store

I'm sad to say that in the next couple of days I'll be removing both the free and paid versions of Tanks! from the US app store.

I received an email from Apple last night saying that Atari had complained about the game saying that it infringed on their copyright. They appear to be of the opinion that they hold the rights to produce any game that involves driving a tank around...

You can read the letter that Atari sent to apple here.

Obviously I don't believe that Atari have a leg to stand on in this matter and taking the game down in no way implies any acceptance of their claims. But the amount of money that comes in from the game (<$1 a day) does not justify paying for expensive lawyers.

Messing around with the XBOX Kinect

Just got back from a trip up to Scotland to visit my good friends Rob and Judy for the comedy festival.

Since we had a bit of spare time on of the days, Rob and I decided that it would be good fun to try and hook his Kinect up to one of my games. Since I was part through porting the old space invaders game (Aliens Invade) from the iPhone to the Mac app store it seemed like a good opportunity to have a bit of a play.

We initially tried processing the raw depth data ourselves and using this to control the game. This was quite good for controlling the position, but we struggled to find a good algorithm for detecting when the user wanted to fire so decided that we needed a more powerful solution. Fortunately there is OpenNI and NITE which give you person tracking and pose detection.

It works pretty well - and is great fun! Just run left and right to avoid the bullets and wave your hands in the air to fire.

iAds Report

Thought it might be interesting to share some stats on how iAds are performing.

Heart Rate - Free has now been live for 2-3 weeks. Interestingly it somehow ended up in the wrong category, going into Entertainment when it was supposed to go into Health and Fitness. I don't know what impact that has had on downloads or advertising revenue.

Over the two weeks it's been into the top 100 entertainment apps in various countries (it's doing particularly well in Hong Kong for some reason). In the US it got to around 60 ish before dropping out of the charts. I think it's now hovering around the 200 mark. To get into the top 100 free entertainment apps required over 1500 downloads a day in the US.

Worldwide the application has been downloaded around 40-50,000 times. On its best performing day there were just under 9,500 downloads world wide. Currently it's doing around 1,000 downloads a day, but there is a definitely downward trend.

When I was building the app I had already decided that it should be free so added AdMob adverts and then at the last moment decided that I might as well add iAds to the mix.

As iAds are not available in all countries and having heard that fill rates could be very low I set the app up so that it would first try to get an iAd and if that failed it would pull down an AdMob ad. That way as iAds were rolled out to more countries the app would pick them automatically.

The iAd reporting front end is a little bit clunky in how much you can slice and dice the data. What I'd really like to show is revenue broken out by time for the US - which is where iAds are mostly available (UK and France only started having inventory a few days ago). But here are the top 10 countries by ad requests since the app went live.

Country Name	Revenue	eCPM	Requests	Impressions	Fill Rate	CTR
Hong Kong	$0.00	$0.00	284868	0	0.00%	0.00%
Unknown	$0.00	$0.00	97132	0	0.00%	0.00%
United States	$1,230.03	$29.75	83133	41348	49.74%	2.81%
China	$0.00	$0.00	77595	0	0.00%	0.00%
Israel	$0.00	$0.00	70080	0	0.00%	0.00%
France	$14.29	$8.87	58591	1612	2.75%	4.03%
Canada	$0.36	$2.86	48669	126	0.26%	2.38%
UK/GB	$10.31	$7.38	24280	1397	5.75%	3.29%
Korea, Republic of	$0.00	$0.00	20992	0	0.00%	0.00%

Fill rates for the US are actually pretty reasonable at just under 50%. eCPM is amazing at amost $30 when you compare it with the stats from AdMob over the same period of time:

Revenu	eCPM	Request	Impressions	Fill Rate	CTR through rate
$675.38	$2.22	304,627	303,980	99.79%	4.51%

The iAds are generating almost 15 times the eCPM.

From its peak of $160 in a day revenue has declined quite rapidly. There's almost a direct correlation between the number of downloads in the US and amount of revenue. One problem with the heart rate app is that it doesn't really have anything currently to make the user come back to it very often. So the ad revenue tends to be tightly coupled to new users.

The peak at the end of the chart corresponds to using the money generated from the AdMob ads on advertising the app in the US. This is definitely leading to more downloads, but it's not enough to keep the app in the charts in the entertainment category.

Heart Rate - Free is the first reasonably successful free app I've produced - I'm not a big fan of adverts in apps, but there does seem to be a compelling case for it. It will be interesting to see if the revenue continues when the app is really out in the wastelands and only getting 1 or 2 downloads a day.

Heart Rate Free - How it works

I've had a few emails asking for information on how my app Heart Rate - Free works.

There's nothing particularly clever about it, it just uses the camera to pick up the slight change in the colour of the light coming from the flash as the blood flows in and out of the finger.

You can make a few modifications to the code posted in the Augmented Reality Post or download the demo code.

Setting up the capture session is pretty much the same as usual. The main differences are that we don't need to preview the image (it's not very interesting to look at...) and that we need to turn on the flash to illuminate the finger.

// switch on the flash in torch mode
if([camera isTorchModeSupported:AVCaptureTorchModeOn]) {
 [camera lockForConfiguration:nil];
 camera.torchMode=AVCaptureTorchModeOn;
 [camera unlockForConfiguration];
}

We'll also set the frame size to low as we don't need high resolution images:

[session setSessionPreset:AVCaptureSessionPresetLow];

The capture setup code looks like this now:

// Create the AVCapture Session
session = [[AVCaptureSession alloc] init];
 
// Get the default camera device
AVCaptureDevice* camera = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
if([camera isTorchModeSupported:AVCaptureTorchModeOn]) {
 [camera lockForConfiguration:nil];
 camera.torchMode=AVCaptureTorchModeOn;
 [camera unlockForConfiguration];
}
// Create a AVCaptureInput with the camera device
NSError *error=nil;
AVCaptureInput* cameraInput = [[AVCaptureDeviceInput alloc] initWithDevice:camera error:&error];
if (cameraInput == nil) {
 NSLog(@"Error to create camera capture:%@",error);
}
 
// Set the output
AVCaptureVideoDataOutput* videoOutput = [[AVCaptureVideoDataOutput alloc] init];
 
// create a queue to run the capture on
dispatch_queue_t captureQueue=dispatch_queue_create("catpureQueue", NULL);
 
// setup our delegate
[videoOutput setSampleBufferDelegate:self queue:captureQueue];
 
// configure the pixel format
videoOutput.videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA], (id)kCVPixelBufferPixelFormatTypeKey,
         nil];
// cap the framerate
videoOutput.minFrameDuration=CMTimeMake(1, 10);
// and the size of the frames we want
[session setSessionPreset:AVCaptureSessionPresetLow];
 
// Add the input and output
[session addInput:cameraInput];
[session addOutput:videoOutput];
 
// Start the session
[session startRunning];

In out capture callback we'll pull out the RGB values from the preview image:

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
 // this is the image buffer
 CVImageBufferRef cvimgRef = CMSampleBufferGetImageBuffer(sampleBuffer);
 // Lock the image buffer
 CVPixelBufferLockBaseAddress(cvimgRef,0);
 // access the data
 int width=CVPixelBufferGetWidth(cvimgRef);
 int height=CVPixelBufferGetHeight(cvimgRef);
 // get the raw image bytes
 uint8_t *buf=(uint8_t *) CVPixelBufferGetBaseAddress(cvimgRef);
 size_t bprow=CVPixelBufferGetBytesPerRow(cvimgRef);
 // get the average red green and blue values from the image
 float r=0,g=0,b=0;
 for(int y=0; y<height; y++) {
  for(int x=0; x<width*4; x+=4) {
   b+=buf[x];
   g+=buf[x+1];
   r+=buf[x+2];
  }
  buf+=bprow;
 }
 r/=255*(float) (width*height);
 g/=255*(float) (width*height);
 b/=255*(float) (width*height);

 NSLog(@"%f,%f,%f", r, g, b);
}

If we run this and plot the values for red and green (blue is pretty uninteresting as it's almost zero) we can see that there's definitely some kind of change when the heart beats:

The peaks in both the red and green values correspond to your pulse. We can combine the values into something a lot more convenient if we convert the RGB values to HSV (hue, saturation, value) and just use the value of hue.

We can now see a definite pattern that corresponds to the heart beat. You'll probably want to do some filtering on the results from this - I'd suggest running it through a simple notch filter (highpass and lowpass filter) to get rid of DC component and apply a bit of smoothing to isolate the heart beat:

You can then pass these values into whatever mechanism you plan to use to detect and measure the beats.

Here's the sample project in action:

You can download it from here. You'll need to replace the filtering with something a bit more sensible, the code I've put there is purely for demo purposes.

Heart Rate Free - just released on the iPhone

Saw an app that does this on the android platform so thought I 'd give it a go on iPhone.


It measures your heart rate by looking at the changes in skin colour as the blood flows in. It's only available on the iPhone 4 as it needs the flash in torch mode to work.

You can get it here - it's free! http://itunes.apple.com/us/app/heart-rate-free/id401450171?mt=8

I'll do a blog post soon on how it works. It's surprisingly simple.