The standard image capture in API in the iPhone SDK is the UIImagePickerController. There is much discussion on the web about how this can be customized via subclassing, both from the viewpoint of technical feasibility, and from the viewpoint of being allowed onto the AppStore. It is generally accepted that going direct to private frameworks is unacceptable, even though this arguably can give a better user experience. Phanfare had their app pulled from their AppStore for using the PhotoLibrary private framework, and returned with a new version that instead customizes the UIImagePickerController experience.

I took an in-depth look at the view structure that the standard UIImagePickerController creates. The first is obtained by looking at self.view and is not particularly interesting. It consists of a UITransitionController and a UINavigationController and is clearly the meta-system for switching between the capture and preview screens.

The second view hierarchy is created by PLCameraController, and can be obtained by implementing willShowViewController in the UINavigationControllerDelegate. The first callback to this comes when UIImagePickerController wants to display a PLCameraController. The view hierarchy is much more interesting:

uiimagepickercontroller1

Several of these classes are undocumented, but that doesn’t prevent us from manipulating them with standard APIs. Experimentation showed that the UIImageView highlighted in red is the important one – it’s the camera preview pane. You can remove every other view, and just retain this one, and you will get a full-screen camera view. You can also move the TPPushButton (which represents the cancel button) out of the hierarchy and place it elsewhere on the screen. Doing the same with the TPCameraPushButton causes a crash when you activate the button, because the PLCameraController tries to modify the buttons, and fails because the view structure has changed.

You can also add further views (as Phanfare did) to overlay extra details. Clearly there is yet more scope for customizing other aspects of the hierarchy – this requires more experimentation, and we’ll report on this as we learn more.

77 thoughts on “Views of UIImagePickerController

  1. Pingback: Views of UIImagePickerController « JongAm's blog

  2. Has anyone figured out a solution to this trouble? Rolando, if you have a legal way of doing this or anyone else that is reasonably fast I would be willing to pay for the solution. The new 3.1 routines are too slow. UIGetScreenImage() works fast enough for our usage and uses little memory but it is part of the private api. email: delay at pobox dot com if anyone has any code they want to give away or sell that solves this trouble.

    Reply
  3. Well the new 3.1 OS certainly makes it a lot nicer to change the UI for the UIImagePickerController

    OverlayViewController *oController = [[OverlayViewController alloc] initWithNibName:@”OverlayView” bundle:nil];
    oController.tabController = self;
    self.overlayController = oController;
    [oController release];
    UIImagePickerController *uip = [[[UIImagePickerController alloc] init] autorelease];
    uip.delegate = self;
    uip.sourceType = UIImagePickerControllerSourceTypeCamera;
    uip.showsCameraControls = NO;
    uip.cameraOverlayView = overlayController.view;

    Currently trying to use the takePicture method in a timer to feed a barcode reader. takePicture seems to be taking a picture though it’s a 2MP (3G iPhone for ref), I resize that to 320×480 and feed it to the barcode reader but then it won’t recognize it. For some reason UIGetScreenImage() works really well though. I would like to stay away from using that, so any suggestions?

    Reply
  4. I was playing around with PLCameraController. After updates to 3.0 same source code doesnot work. Not yet figure out the issue. Any ideas?

    Reply
  5. Hey is this hierarchy is same for all the iphone os version!!! I had iphone 3.0 os my hierarchy is different from yours!!! what can i do with iphone 3.0??

    Reply
    • No. The hierarchy has changed in iPhone OS 3.0. And remember that the behaviour is different on the 3GS, which has auto-focus, than on older devices.

      Reply
  6. Hi all

    We have been browsing around for some time looking for info about the problems your discussing.
    Today we came across a newly updated app: RedLaser, a barcode reader.

    In their newest version on AppStore they have realtime scanning of the image from the camera, with custom overlays. Has anyone found new possibilities in latest SDK 3.0? Looks like they found a great approach in RedLaser, which passed the appstore review.

    Reply
  7. Pingback: UIImagePickerController in 3.0 at Under The Bridge

  8. Pingback: /home/ahhong » Blog Archive » [iPhone] Uncovering the Secrets of UIImagePickerController Part 1

  9. Has anyone made any progress in obtaining raw image data from the camera? I know Rolando has successfully achieved this, but we’ve yet to hear from him.

    @Rolando – Are you able to share your methods now? Also, did your app get rejected from the App Store? Could you share you experience with us?

    Also, for those of you who have published apps (or attempted to) that have manipulated the PLCameraView, could you please share your experiences with Apple? Has anyone figured out a way to do this without getting rejected by Apple?

    As Ben has mentioned, there are obviously people out there getting away with this. I would hate to see all my hard work go to waste b/c of a moody app reviewer. Somehow I feel like the only way to slip these apps through the reviewers is to make friends with one of the evangelists over in Cupertino.

    Has anyone made contact with these guys? Furthermore, has anyone gotten specific feedback regarding policy on manipulating PLCameraView?

    My implementation provides a subclass for UIImagePickerController and does not use any “undocumented” API’s so to speak. It seems ridiculous to me that these apps are getting rejected for hiding UI elements using calls like:

    [[[view subviews] objectAtIndex:0] setHidden:YES]

    Thoughts anyone?

    Reply
  10. Hi rolando,

    I just want to confirm are you able to grab the screen after displaying the camera preview on it?

    In my application I want video stream from camera before displaying it on screen. I will process that stream and then display on screen. Is this possible?

    You can mail me at vishalnandanwar@gmail.com

    Vishal N

    Reply
  11. Clearly some apps have been rejected – but there are plenty of apps out there that mess with the views, QuadCamera in particular – and also Microsoft Tag Reader. QuadCamera doesn’t seem to have any issues getting accepted, they are up to something like version 1.8 now, suggesting that they have been through the testing process 9 times.

    Reply
  12. Yes mine just got rejected too since I called the “undocumented API:s in UIImagePickerController” (I am just hiding the ugly labels).

    There are indications in the 3.0 release notes that the camera view is being changed, so maybe they are getting tougher on this issue.

    Could we hope for a better API in later versions of 3.0? (but that is probably overoptimistic).

    At least everyone show add a request for a better API in the bugreporter.

    Reply
  13. @Tom, ours did also.

    Does anyone have any experience getting an app accepted that hides the “Take a Picture” overlay or changes its text?

    The feedback from Apple seems to tell us that touching the PLCameraView object at all is against the rules.

    Reply
  14. Any word from Rolando?

    I’m starting to see a ton of proof of concept augmented reality apps – so people are definitely managing to access the video stream.

    I really want to add this feature to my Sudoku Grab application (http://www.youtube.com/watch?v=oImMJ6p6mKE). I can definitely make my puzzle detection and solver code work in real time so I could superimpose the solved puzzle on top of the image.

    At the moment there’s no point in doing the optimization as it works fast enough (in the latest version <1second) for non real time.

    If anyone can help them drop me an email chris at cmgresearch.com

    There’s a promo code for my app in it (worth a huge $1!)

    Reply
  15. @Tom:

    You tweaked the view structure and your app got rejected? Can you share the details? What did you do, and what did Apple say?

    Reply
  16. Has anybody had any success getting an app that tweaked PLCameraView approved by Apple? I’ve seen apps that clearly do, but ours got rejected…

    Reply
  17. Codz, QuadCam uses UIGetScreenImage() – I’ve been in contact with the developer.

    Has anyone found an alternative to UIGetScreenImage / renderInContext ?

    Reply

Leave a reply

required

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>