Camera Usage and Features, from iOS* to Windows* 8 Store App

Download Article

Download Camera Usage and Features, from iOS* to Windows* 8 Store App [PDF 1.26MB]

Abstract

Modern applications take advantage of camera features to enable new types of usage models and enhanced user experiences. In this article we will cover how to use the camera feature in a Windows* 8 Store app to take pictures and how to convert the picture into base64 encoding, which can then be transmitted as JSON data over the network to a backend server or persisted into a local database. We will also discuss how iOS* camera usage compares to Windows 8 and some tips on migrating your code.

Overview

The camera is one of the most used features in mobile device applications. Users can capture photos or record videos with different configuration settings. All of the modern OS platforms like iOS and Windows 8, provide APIs and platform services to make camera usage seamless for the end user and easy to use for the programmer. This gives users a well-known user interface to interact with the camera, common across all apps. Some apps may create a fully customized camera usage depending on their requirements, but for most applications the default usage is recommended.

In this article we will cover how iOS developers can port their camera-related code to a Windows 8 Store app. We will discuss how to invoke the default camera UI control, process the result using async programming patterns, and use the Image control to display the picture in a Windows Store app UI.

We will also cover how to encode and decode the picture to/from base64 format, which can be very useful in line of business apps that may require transmitting the image over the network as JSON data for RESTful backend services.

Camera Controls and UI

The default camera UI controls come with some standard options that users can tweak. They usually handle all user interactions with the camera UI, including the touch gestures, finally returning the result (captured picture or video) to the code invoking it (via delegates or asynchronous callbacks).

Depending on the API settings, the camera controls let users edit the picture (e.g., crop) and customize as needed. For advanced editing and customization, we may have to develop our own camera UI control as the default control capabilities are limited.

These controls also provide a way for users to play with different device camera capabilities like flash, brightness, and other standard features depending on the platform.

The code required to use these controls is usually simple and easy to integrate as long as you stick to the platform recommended API usage patterns.

A Healthcare Windows Store App

Like we did in several other articles in this forum, we will build the case study around a healthcare Windows Store application. We will extend it with the capability to take a picture and update the patient’s profile.

Some of the previous articles include:

The application allows the user to login to the system, view the list of patients (Figure 1), and access patient medical records, such as the profiles, doctor’s notes, lab test results, vital graphs, etc.

Figure 1: The “Patients” page of the Healthcare Line of Business app provides a list of all patients. Selecting an individual patient provides access to the patient’s medical records.

In Figure 1 all the profile pictures are shown as generic avatars. With the camera feature enabled, we will be able to update those images to the most recent profile picture of each patient as needed.

Migrating Camera related code from iOS to Windows 8

The iOS platform provides you with both a default camera UI control option and a more advanced custom solution that has greater flexibility. UIImagePickerController API gives us the simple, default, common camera usage for taking pictures and the AV foundation framework has the fully customizable solution. This document has more details:

http://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/CameraAndPhotoLib_TopicsForIOS/Introduction/Introduction.html

iOS developers looking to port camera-related features in their apps to Windows 8 Store apps can use the default camera UI and control, CameraCaptureUI, which is part of Windows.Media.Capture namespace.

http://msdn.microsoft.com/EN-US/library/windows/apps/windows.media.capture.cameracaptureui.aspx

Using the Windows 8 camera UI and some tips on delegating the code

In this article we assume basic familiarity with Windows Store app development. Similar to other modern OSs (e.g., iOS), Windows 8 Store apps must declare camera capability in the app manifest. Double clicking on the Package.appxmanifest in your Visual Studio* 2012 project should bring up a manifest UI that you can use to tweak the settings. Figure 2 shows the webcam capability enabled in the manifest, which lets us use the camera feature.

Figure 2: App manifest showing the webcam package capability (captured from Visual Studio* 2012)

Our project should now be ready to access the camera feature.

When the app accesses the camera for the first time, Windows 8 brings up a user permission dialog allowing the user to either block or enable camera access. Figure 3 shows an example.

Figure 3: Camera permission dialog (capture from Windows* 8)

As discussed previously, Windows Store apps can use the system-provided default camera UI control (CameraCaptureUI class under Windows.Media.Capture namespace) for most of the use cases. This has an additional advantage in that users will already be familiar with the UI controls, gestures, and usage.

The app usually invokes the camera control in response to some kind of user action—a button click or a touch gesture.

Figure 4 shows the app bar with a camera button labeled “Photo.”

Figure 4: Camera button and icon (captured from Windows* 8)

We can use the Windows standard icon (PhotoAppBarButtonStyle) for styling our camera button. Figure 5 shows the sample XAML code for a camera button as part of the app bar.

<Button x:Name="button_take_photo" Grid.Column="1" HorizontalAlignment="Left" Margin="0,0,0,0"
Style="{StaticResource PhotoAppBarButtonStyle}" Height="87" VerticalAlignment="Top" 
DataContext="{Binding Patient}" Command="{Binding PatientPicCmd}" RenderTransformOrigin="0.5,0.5" 
Visibility="Visible"/>

Figure 5: XAML code for camera button and default icon as part of app bar ++

In Windows 8 the button click input events from all sources (touch, mouse, etc.) are automatically routed to the same input handler, in this case the Command property (PatientPicCmd binding). The PatientPicCmd command is covered later in this article.

It is usually recommended to design your code in an MVVM pattern for XAML-based apps. Please refer to the article below for more details on MVVM:

http://msdn.microsoft.com/en-us/magazine/dd419663.aspx

This allows us to implement our button event handler logic inside the patient view model and use the same view model and logic in any UI page of the app by simply binding to properties. Figure 6 shows another UI page where we have another camera button. No need for re-implementing the button click handler in XAML code behind, as we simply bind to the same patient view model class (which has our camera button click handler logic).

Figure 6: Another UI page with camera button (captured from Windows* 8)

Invoking the camera dialog is straightforward in Windows 8. We just have to create an instance of CameraCaptureUI class, configure the instance with any custom settings, and then show the camera UI dialog with CaptureFileAsync method of the instance. Since this method is implemented as an async pattern, we will need to use the await keyword to invoke it.

For more details on asynchronous programming in Windows 8 Store apps, please refer to the following article.

http://msdn.microsoft.com/EN-US/library/vstudio/hh191443.aspx

Figure 7 shows sample code for invoking the default camera UI in Windows 8.

private async void DoPatientPicCmd(object sender)
        {
            try
            {
                CameraCaptureUI dialog = new CameraCaptureUI();
               
                dialog.PhotoSettings.CroppedAspectRatio = new Size(4, 3);

                // fix the photo size to 300x300
                //dialog.PhotoSettings.CroppedSizeInPixels = new Size(300, 300);

                StorageFile photo = await dialog.CaptureFileAsync(CameraCaptureUIMode.Photo);
                if (photo == null) return;
              
                // custom process the photo
            }
            catch (Exception ex)
            {
// custom exception handling code

            }
        }

Figure 7: Sample code for invoking camera UI dialog ++

The CameraCaptureUI class has two main properties: PhotoSettings and VideoSettings. We can invoke the Camera Dialog in Photo mode only by specifying “CameraCaptureUIMode.Photo” when invoking the dialog.

PhotoSettings is an instance of the CameraCaptureUIPhotoCaptureSettings class and can be used to specify options such as cropping, aspect ratio, max resolution, and format. Please refer to the following article for more details:

http://msdn.microsoft.com/EN-US/library/windows/apps/windows.media.capture.cameracaptureuiphotocapturesettings.aspx

In our sample code we enabled cropping with an aspect ratio 4:3. Figure 8 shows the default Camera UI dialog. It has the video option disabled since we invoked the control with photo mode. Other options are change camera (to switch to back, front, or other cameras), camera options (e.g., resolution), and timer functionality.

Figure 8: Default camera UI dialog (captured from Windows* 8)

Users can either single tap or click anywhere on the screen to take a photo. Depending on the configuration, the app brings up another dialog allowing users to crop or edit the picture. The edit options are basic and limited. For advanced editing, a custom camera UI control is recommended (or you can add a process for editing later).

Figure 9: Camera UI control allows basic editing (captured from Windows* 8)

After users are satisfied with their photos, they can click ok to accept, or retake as needed. After a successful action (photo is captured and OK is selected), the camera UI dialog returns the photo captured as an image file (instance of “StorageFile” class in Windows.Storage namespace), which can then be processed further or persisted to data model.

Converting the photo image file into an image buffer

To transmit photo images as JSON to a backend server (typical use case in enterprise apps), we want it to be in base64 encoded string format. To enable this conversion we first need to convert the photo file into an image buffer, as the encoding/decoding APIs expect a buffer.

It is strongly recommended to use an async pattern where possible, as all of this processing is happening inside the photo button click handler.

We access the photo file as a random access stream and copy it into a buffer using DataReader interface. Sample code is shown in Figure 10.

                StorageFile photo = await dialog.CaptureFileAsync(CameraCaptureUIMode.Photo);
                if (photo == null) return;
              
                byte[] photoBuf = null;

                using (IRandomAccessStream photostream = await photo.OpenReadAsync())
                {
                    photoBuf = new byte[photostream.Size];
                    using (DataReader dr = new DataReader(await photo.OpenReadAsync()))
                    {
                        await dr.LoadAsync((uint)photostream.Size);
                        dr.ReadBytes(photoBuf);
                    }
                }

Figure 10: Sample code to convert the photo file into a buffer ++

Encoding and decoding image buffers into/from base64 encoded strings

We can use the Convert class in System namespace to encode and decode base64 format strings. In the previous section we discussed how to convert the photo image into a byte buffer, which we can use in base64 conversion APIs. The ToBase64String method is documented here:

http://msdn.microsoft.com/EN-US/library/dhx0d524.aspx

The code snippet below shows how to convert the photoBuf buffer we created in the previous section into a base64 encoded string.

                Pic = Convert.ToBase64String(photoBuf, 0, photoBuf.Length);

The variable “Pic” is a public member variable of the patient view model class we discussed earlier.

To convert the string back into a buffer:

                var photoBuf = Convert.FromBase64String(pic);

Displaying the picture using Image Control and BitmapImage Binding

We can use the XAML Image control to display our photo. Please refer to the following document for more information:

http://msdn.microsoft.com/EN-US/library/windows/apps/windows.ui.xaml.controls.image.aspx

The Image control is very flexible, allowing different image formats and options. For example, we can display our photo by binding to our patient view model, as shown below:

<Image  Margin="0,40,20,0"            
                            DataContext="{Binding Patient}"
                            Source="{Binding Image}"></Image>

The Source property on the Image control can automatically convert either a BitmapImage instance or even a direct source path to an image file. To dynamically update our image, it is convenient if we bind it to a BitmapImage instance. You can find more details on this class here:

http://msdn.microsoft.com/EN-US/library/windows/apps/windows.ui.xaml.media.imaging.bitmapimage.aspx

To reiterate, it’s strongly recommended to use async methods where possible as these properties are bound to XAML UI elements.

BitmapImage gives us the option to directly use a URI or set our photo buffer as the input stream via the SetSourceAsync method. We use the SetSourceAsync method for generating the profile picture if available, or else a random avatar (from app assets) depending on patient gender. Please refer to the code snippet in figure 10 below.

    public class PatientsViewModel : BindableBase
    {
        public PatientsViewModel() 
        {
            this.PatientPicCmd= new DelegateCommand(DoPatientPicCmd);
        }

        private string pic = string.Empty;
        public string Pic 
        { 
            get { return pic; } 
            set { 
                this.SetProperty(ref pic, value);                
            } 
        }

        private BitmapImage image = null;
        public BitmapImage Image
        {
            get
            {
                if (image == null) GetImageAsync();
                return image;
            }
        }

        public async Task GetImageAsync()
        {
            image = new BitmapImage();
            if (pic.Length > 1)
            {
                var photoBuf = Convert.FromBase64String(pic);
                using (InMemoryRandomAccessStream mrs = new InMemoryRandomAccessStream())
                {
                    using (DataWriter dw = new DataWriter(mrs.GetOutputStreamAt(0)))
                    {
                        dw.WriteBytes(photoBuf);
                        await dw.StoreAsync();
                    }
                    await image.SetSourceAsync(mrs);
                }
            }
            else
            {
                Random rand = new Random();
                String url = "ms-appx:///Assets/" + gender.ToLower() + rand.Next(1, 4).ToString() + ".png";
                StorageFile file = await StorageFile.GetFileFromApplicationUriAsync(new Uri(url));                
                using (IRandomAccessStream fileStream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read))
                {
                    await image.SetSourceAsync(fileStream);
                }
            }
            OnPropertyChanged("Image");
        }
         …
         …

Figure 10: sample code for displaying a photo in an Image control using BitmapImage & SetSourceAsync ++

Pic string property stores the base64 encoded string that we converted from our camera photo file. This property can be persisted to a local database or transmitted to backend server. The “Image” property (of type BitmapImage), which we bind to our XAML image control, simply returns a BitmapImage depending on what is in the “Pic” variable. If it is empty, it returns a random avatar or else converts the “Pic” base64 string back into a byte buffer and returns a BitmapImage instance containing it.

Figure 11 shows the updated patients screen we referred to in Figure 1.

Figure 11: Updated patients UI page (captured from Windows* 8)

Users can click on any patient and update his/her profile picture, and the XAML binding will automatically update all references to it in Image controls.

Summary

We have discussed how iOS developers can port their camera-related code to Windows 8. We covered how to invoke the default camera UI control, process the result, use async programming patterns, and use the Image control to display the picture in Windows Store apps UI. We also covered how to encode and decode the picture to/from base64 format, which can then be easily transmitted as JSON data to RESTful backend services.

*Other names and brands may be claimed as the property of others.

++This sample source code is released under the Intel OBL Sample Source Code License (MS-LPL Compatible), Microsoft Limited Public License, and Visual Studio* 2012 License.

User Experience and Design

URL