Intel® INDE Media for Mobile Tutorials - Video Capturing for Cocos2d-x* Applications on Android*

This tutorial explains how to use Intel® INDE Media for Mobile to add video capturing capability to Cocos2d-x applications on Android.

Getting Started

This tutorial is made for Cocos2d-x 2.x version. Since Cocos2d-x 3.x version doesn't need to install Cygwin for building Android samples any more. In this tutorial, I will show you how to set up Cocos2d-x Android development on Windows 7. It is better to have a fast and stable network connection, because there are many software packages need to be downloaded during the configuration process.

We can go to Cocos2d-x official website to download the latest stable version of Cocos2d-x.

From the above picture, we can see that there are two versions available, one is called v3.3 and the other is called v2.2.6. In this tutorial, I will take Cocos2d-x 2.2.6.

Note: If you want to download any old version of Cocos2d-x, you can go to the link named "looking for an old version?". But I highly recommend you to try out the latest version of Cocos2d-x, it has many new features and bug fixes.

Ok, right click the download link and choose "Save link as..." to save the zip file to a proper location. In our case, the location is C:\code\INDE_MfM_Cocos2d-x\.

Note: Don't put Cocos2d-x folder to the root directory of drive C:\, since it will cause many privilege related problems.

Environment setup

Go to Eclipse official website and get Eclipse IDE for Java Developers.

Add the ADT (Android Development Tools) plugin to Eclipse:

1. Start Eclipse, then select Help > Install New Software.
2. Click Add, in the top-right corner.
3. In the Add Repository dialog that appears, enter "ADT Plugin" for the Name and the following URL for the Location: 

   https://dl-ssl.google.com/android/eclipse/

   Note: The Android Developer Tools update site requires a secure connection. Make sure the update site URL you enter starts with HTTPS.
4. Click OK.
5. In the Available Software dialog, select the checkbox next to Developer Tools and click Next.
6. In the next window, you'll see a list of the tools to be downloaded. Click Next.
7. Read and accept the license agreements, then click "Finish". If you get a security warning saying that the authenticity or validity of the software can't be established, click OK.
8. When the installation completes, restart Eclipse.

Get Python 2.7.9 from official website. Don’t forget to add C:\Python27 to your PATH environment variable. Check it with cmd command "where python".

Next, we should install cygwin. My installation path is C:\cygwin64\bin. When you finish downloading, you should install the following software packages through cygwin: 

autoconf, automake, binutils, gcc-core, gcc-g++, gdb, pcre, pcre-devel, gawk, make.

Now we should add cygwin's bin folder C:\cygwin64\bin at the end of the system's PATH.

Integrate INDE Media for Mobile

1. Import project \cocos2d-x-2.2.6\samples\Cpp\SimpleGame\proj.android\ to Eclipse.
2. Add line chmod 777 -R "$APP_ANDROID_ROOT"/assets to \SimpleGame\proj.android\build_natives.sh:

   # copy resources
   for file in "$APP_ROOT"/Resources/*
   do
   if [ -d "$file" ]; then
       cp -rf "$file""$APP_ANDROID_ROOT"/assets
   fi
   
   if [ -f "$file" ]; then
       cp "$file""$APP_ANDROID_ROOT"/assets
   fi
   chmod 777 -R "$APP_ANDROID_ROOT"/assets
   done

3. We need to link the cocos2dx Java source into our project. To do this, right click on your project in the Package Explorer in Eclipse, and select Properties. Select Java Build Path from the left-hand pane, and then select the Source:

4. Click on "Link Source..." and browse to COCOS2DX_ROOT\cocos2dx\platform\android\java\src\. For the folder name, put something other than "src", such as "cocos2dx-src", and then click Finish:

   After that, the project tree will change as follows:

5. Modify AndroidManifest.xml in \SimpleGame\proj.android\ folder: 

   <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/><uses-permission android:name="android.permission.RECORD_AUDIO"/>

6. Download and install Intel INDE by visiting http://intel.com/software/inde. After installing Intel INDE, choose to download and install the Media for Mobile. For additional assistance visit the Intel INDE forum.

   Go to the installation folder of Media for Mobile-> libs and copy two jar files (android-<version>.jar and domain-<version>.jar) to your \SimpleGame\proj.android\libs\ folder:

   To make sure Eclipse knows the new .jars is in the build path, right click on .jars in the package explorer and select Build Path > Add To Build Path:

7. In the folder \SimpleGame\Classes\ create a C++ header file MfM-C-Interface.h with the following code in it:

   #ifndef MFM_C_INTERFACE_H
   #define MFM_C_INTERFACE_H
   
   #include <jni.h>
   
   bool MfM_InitJVM(JavaVM * vm);
   
   void MfM_StartRecording(int width, int height, int frameRate, int bitRate, const char *videoName);
   
   void MfM_StopRecording();
   
   #endif // MFM_C_INTERFACE_H

   Then create another C++ file in the same directory. Name it MfM-C-Interface.cpp and put the following contents in it:

   #include "MfM-C-Interface.h"
   #include <android/log.h>
   #include <string>
   
   #define  LOG_TAG "MfM"
   #define  LOGD(...) __android_log_print(ANDROID_LOG_DEBUG, LOG_TAG, __VA_ARGS__)
   
   static JavaVM *s_javaVM = 0;
   static jclass s_classID = 0;
   static jobject instance = 0;
   static std::string s_videoDir;
   
   static jmethodID s_getInstanceMethodID = 0;
   static jmethodID s_initCapturingMethodID = 0;
   static jmethodID s_startCapturingMethodID = 0;
   static jmethodID s_stopCapturingMethodID = 0;
   static jmethodID s_getDirectoryDCIMMethodID = 0;
   
   bool MfM_InitJVM(JavaVM * vm)
   {
   	LOGD("----- MfM_InitJVM ------");
   
   	JNIEnv *env;
   	if (vm->GetEnv(reinterpret_cast<void **>(&env), JNI_VERSION_1_6) != JNI_OK) {
   		LOGD("Can't get the enviroument");
   		return false;
   	}
   
   	s_javaVM = vm;
   
   	// Search for our class
   	jclass clazz = env->FindClass("org/cocos2dx/lib/Capturing");
   	if (!clazz) {
   		LOGD("Can't find Capturing class");
   		return false;
   	}
   	// Keep a global reference to it
   	s_classID = (jclass)env->NewGlobalRef(clazz);
   
   	s_getInstanceMethodID = env->GetStaticMethodID(s_classID, "getInstance", "()Lorg/cocos2dx/lib/Capturing;");
   	if (!s_getInstanceMethodID) {
   		LOGD("Can't find getInstance() method");
   		return false;
   	}
   
       s_initCapturingMethodID = env->GetMethodID(s_classID, "initCapturing", "(IIII)V");
       if (!s_initCapturingMethodID) {
       	LOGD("Can't find initCapturing() method");
           return false;
       }
   
   	s_startCapturingMethodID = env->GetMethodID(s_classID, "startCapturing", "(Ljava/lang/String;)V");
   	if (!s_startCapturingMethodID) {
   		LOGD("Can't find startCapturing() method");
   		return false;
   	}
   
       s_stopCapturingMethodID = env->GetMethodID(s_classID, "stopCapturing", "()V");
       if (!s_stopCapturingMethodID) {
       	LOGD("Can't find stopCapturing() method");
           return false;
       }
   
       // Register our static method
       s_getDirectoryDCIMMethodID = env->GetStaticMethodID(s_classID, "getDirectoryDCIM", "()Ljava/lang/String;");
       if (!s_getDirectoryDCIMMethodID) {
       	LOGD("Can't find getDirectoryDCIM() static method");
           return false;
       }
   
       // Get DCIM dir
   	jstring value = (jstring)env->CallStaticObjectMethod(s_classID, s_getDirectoryDCIMMethodID);
   	const char *res = env->GetStringUTFChars(value, NULL);
   	s_videoDir = std::string(res);
   	env->ReleaseStringUTFChars(value, res);
   
   	return true;
   }
   
   void MfM_StartRecording(int width, int height, int frameRate, int bitRate, const char *videoName)
   {
   	JNIEnv *env;
   	if (s_javaVM->AttachCurrentThread(&env, NULL) < 0) {
   		LOGD("AttachCurrentThread failed");
   		return;
   	}
   
   	if (!instance)
   		instance = env->CallStaticObjectMethod(s_classID, s_getInstanceMethodID);
   
   	if (instance) {
   		// Setup format
   		env->CallVoidMethod(instance, s_initCapturingMethodID, width, height, frameRate, bitRate);
   
   		// Start capturing
   		std::string videoPath = s_videoDir + videoName;
   		jstring string = env->NewStringUTF(videoPath.c_str());
   		env->CallVoidMethod(instance, s_startCapturingMethodID, string);
   		env->DeleteLocalRef(string);
   	}
   }
   
   void MfM_StopRecording()
   {
   	if (instance) {
   		JNIEnv *env;
   		if (s_javaVM->AttachCurrentThread(&env, NULL) < 0) {
   			LOGD("AttachCurrentThread failed");
   			return;
   		}
   		env->CallVoidMethod(instance, s_stopCapturingMethodID);
   	}
   }

   We also need to tell the native build process to look for the new C++ file. In \SimpleGame\proj.android\jni\Android.mk, add the following line:

   LOCAL_SRC_FILES := hellocpp/main.cpp \
                      ../../Classes/AppDelegate.cpp \
                      ../../Classes/HelloWorldScene.cpp \
                      ../../Classes/GameOverScene.cpp \
                      ../../Classes/MfM-C-Interface.cpp

8. Finally, we add the necessary codes into the project to enable capturing. In \SimpleGame\proj.android\jni\hellocpp\main.cpp, add the following lines to JNI_OnLoad:

   #include "MfM-C-Interface.h"
   ...
   jint JNI_OnLoad(JavaVM *vm, void *reserved)
   {
       JniHelper::setJavaVM(vm);
   
       if (!MfM_InitJVM(vm)) {
       	return -1;
       }
   
       return JNI_VERSION_1_4;
   }

9. Add GUI to capture. We still need some way of telling the app to start/stop capturing footage. We add the following code block to \SimpleGame\Classes\HelloWorldScene.cpp:

   bool HelloWorld::init()
   {
   	bool bRet = false;
   	do
   	{
   		...
   
   		// Codes for INDE MfM Integration
   
   		// Set up start recording label
   		CCLabelTTF *startRecordingLabel = CCLabelTTF::create("Start Recording", "Artial", 12);
   		startRecordingLabel->setColor(ccc3(0, 0, 0));
   
   		CCMenuItemLabel *startRecordingMenuItem = CCMenuItemLabel::create(startRecordingLabel, this, menu_selector(HelloWorld::startRecording));
   
   		// Set up stop recording label
   		CCLabelTTF *stopRecordingLabel = CCLabelTTF::create("Stop Recording","Artial", 12);
   		stopRecordingLabel->setColor(ccc3(0, 0, 0));
   
   		CCMenuItemLabel *stopRecordingMenuItem = CCMenuItemLabel::create(stopRecordingLabel, this, menu_selector(HelloWorld::stopRecording));
   
   		// Make the menu
   		CCMenu *menu = CCMenu::create( startRecordingMenuItem, stopRecordingMenuItem, NULL);
   		menu->alignItemsVertically();
   
   		this->addChild(menu);
   		menu->setPosition(ccp(origin.x + visibleSize.width - 60, origin.y + visibleSize.height - 50));
   
   		// End Codes for INDE MfM Integration
   
   		...
   
   	} while (0);
   
   	return bRet;
   }

   This code adds Start Recording and Stop Recording buttons to the game screen. The last thing we need to do is add the calls to the correct functions for starting and stopping a capturing. Inside the HelloWorld class, we put the following lines:

   void HelloWorld::startRecording(CCObject *pSender)
   {
       MfM_StartRecording(1280, 720, 30, 3000, "Cocos2dxCapturing.mp4");
   }
   
   void HelloWorld::stopRecording(CCObject *pSender)
   {
       MfM_StopRecording();
   }

   Finally, we added the corresponding function headers to \SimpleGame\Classes\HelloWorld.h:

   	void startRecording(CCObject *pSender);
   	void stopRecording(CCObject *pSender);

   And don't forget to include MfM-C-Interface.h to HelloWorld.cpp.

Java Side

1. In the COCOS2DX_ROOT\cocos2dx\platform\android\java\src\org\cocos2dx\lib\ folder create a Java* file Capturing.java with the following code in it:

   package org.cocos2dx.lib;
   
   import com.intel.inde.mp.IProgressListener;
   import com.intel.inde.mp.domain.Resolution;
   import com.intel.inde.mp.android.graphics.FullFrameTexture;
   import com.intel.inde.mp.android.graphics.FrameBuffer;
   import com.intel.inde.mp.android.graphics.EglUtil;
   
   import android.opengl.GLES20;
   import android.os.Environment;
   import android.util.Log;
   import android.content.Context;
   
   import java.io.IOException;
   import java.io.File;
   
   public class Capturing
   {
   	private static FullFrameTexture texture;
   	private FrameBuffer frameBuffer;
   	private VideoCapture videoCapture;
   	private int videoWidth = 0;
   	private int videoHeight = 0;
   
   	private static Capturing instance = null;
   
   	private IProgressListener progressListener = new IProgressListener() {
           @Override
           public void onMediaStart() {
           }
   
           @Override
           public void onMediaProgress(float progress) {
           }
   
           @Override
           public void onMediaDone() {
           }
   
           @Override
           public void onMediaPause() {
           }
   
           @Override
           public void onMediaStop() {
           }
   
           @Override
           public void onError(Exception exception) {
           }
       };
   
       public Capturing(Context context, int width, int height)
       {
   		videoCapture = new VideoCapture(context, progressListener);
   
   	    frameBuffer = new FrameBuffer(EglUtil.getInstance());
   		frameBuffer.setResolution(new Resolution(width, height));
   		texture = new FullFrameTexture();
   
   		instance = this;
       }
   
       public static Capturing getInstance()
       {
       	return instance;
       }
   
       public static String getDirectoryDCIM()
       {
           return Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DCIM) + File.separator;
       }
   
       public void initCapturing(int width, int height, int frameRate, int bitRate)
       {
           VideoCapture.init(width, height, frameRate, bitRate);
           videoWidth = width;
       	videoHeight = height;
       }
   
       public void startCapturing(String videoPath)
       {
           if (videoCapture == null) {
               return;
           }
           synchronized (videoCapture) {
               try {
                   videoCapture.start(videoPath);
               } catch (IOException e) {
               }
           }
       }
   
   	public void beginCaptureFrame()
       {
       	frameBuffer.bind();
       }
   
   	public void captureFrame(int textureID)
       {
           if (videoCapture == null) {
               return;
           }
           synchronized (videoCapture) {
               videoCapture.beginCaptureFrame();
               GLES20.glViewport(0, 0, videoWidth, videoHeight);
               texture.draw(textureID);
               videoCapture.endCaptureFrame();
           }
       }
   
   	public void endCaptureFrame()
       {
       	frameBuffer.unbind();
   		int textureID = frameBuffer.getTextureId();
       	captureFrame(textureID);
   		texture.draw(textureID);
       }
   
       public void stopCapturing()
       {
           if (videoCapture == null) {
               return;
           }
           synchronized (videoCapture) {
               if (videoCapture.isStarted()) {
                   videoCapture.stop();
               }
           }
       }
   
       public boolean isRunning()
       {
       	return videoCapture.isStarted();
       }
   
   }

2. Then create another Java file in the same directory. Name it VideoCapture.java and put the following contents in it: 

   package org.cocos2dx.lib;
   
   import android.content.Context;
   import com.intel.inde.mp.*;
   import com.intel.inde.mp.android.AndroidMediaObjectFactory;
   import com.intel.inde.mp.android.AudioFormatAndroid;
   import com.intel.inde.mp.android.VideoFormatAndroid;
   
   import java.io.IOException;
   
   public class VideoCapture
   {
       private static final String TAG = "VideoCapture";
   
       private static final String Codec = "video/avc";
       private static int IFrameInterval = 1;
   
       private static final Object syncObject = new Object();
       private static volatile VideoCapture videoCapture;
   
       private static VideoFormat videoFormat;
       private static int videoWidth;
       private static int videoHeight;
       private GLCapture capturer;
   
       private boolean isConfigured;
       private boolean isStarted;
       private long framesCaptured;
   	private Context context;
   	private IProgressListener progressListener;
   
       public VideoCapture(Context context, IProgressListener progressListener)
       {
   		this.context = context;
           this.progressListener = progressListener;
       }
   
       public static void init(int width, int height, int frameRate, int bitRate)
       {
       	videoWidth = width;
       	videoHeight = height;
   
       	videoFormat = new VideoFormatAndroid(Codec, videoWidth, videoHeight);
       	videoFormat.setVideoFrameRate(frameRate);
           videoFormat.setVideoBitRateInKBytes(bitRate);
           videoFormat.setVideoIFrameInterval(IFrameInterval);
       }
   
       public void start(String videoPath) throws IOException
       {
           if (isStarted())
               throw new IllegalStateException(TAG + " already started!");
   
           capturer = new GLCapture(new AndroidMediaObjectFactory(context), progressListener);
           capturer.setTargetFile(videoPath);
           capturer.setTargetVideoFormat(videoFormat);
   
           AudioFormat audioFormat = new AudioFormatAndroid("audio/mp4a-latm", 44100, 2);
           capturer.setTargetAudioFormat(audioFormat);
   
           capturer.start();
   
           isStarted = true;
           isConfigured = false;
           framesCaptured = 0;
       }
   
       public void stop()
       {
           if (!isStarted())
               throw new IllegalStateException(TAG + " not started or already stopped!");
   
           try {
               capturer.stop();
               isStarted = false;
           } catch (Exception ex) {
           }
   
           capturer = null;
           isConfigured = false;
       }
   
       private void configure()
       {
           if (isConfigured())
               return;
   
           try {
               capturer.setSurfaceSize(videoWidth, videoHeight);
               isConfigured = true;
           } catch (Exception ex) {
           }
       }
   
       public void beginCaptureFrame()
       {
           if (!isStarted())
               return;
   
           configure();
           if (!isConfigured())
           	return;
   
           capturer.beginCaptureFrame();
       }
   
       public void endCaptureFrame()
       {
           if (!isStarted() || !isConfigured())
               return;
   
           capturer.endCaptureFrame();
           framesCaptured++;
       }
   
       public boolean isStarted()
       {
           return isStarted;
       }
   
       public boolean isConfigured()
       {
           return isConfigured;
       }
   
   }

3. Make changes to the COCOS2DX_ROOT\cocos2dx\platform\android\java\src\org\cocos2dx\lib\Cocos2dxRenderer.java source file:

   ...
   
   public class Cocos2dxRenderer implements GLSurfaceView.Renderer {
   
   	...
   
   	private Context mContext;
   	private Capturing mCapturing;
   
   	public Cocos2dxRenderer(Context context) {
   		mContext = context;
   	}
   
   	@Override
   	public void onSurfaceCreated(final GL10 pGL10, final EGLConfig pEGLConfig) {
   		Cocos2dxRenderer.nativeInit(this.mScreenWidth, this.mScreenHeight);
   		this.mLastTickInNanoSeconds = System.nanoTime();
   		mCapturing = new Capturing(mContext, mScreenWidth, mScreenHeight);
   	}
   
   	...
   
   }

4. Add beginCaptureFrame() and endCaptureFrame() around the Cocos2dxRenderer.nativeRender(); call in the same Cocos2dxRenderer.java file in the onDrawFrame() function:

   	public void onDrawFrame(final GL10 gl) {
   
   		if (mCapturing.isRunning())
   			mCapturing.beginCaptureFrame();
   
   		Cocos2dxRenderer.nativeRender();
   
   		if (mCapturing.isRunning())
   			mCapturing.endCaptureFrame();
   
   	}

5. Finally, pass the application context to the Cocos2dxRenderer's constructor inside Cocos2dxActivity's init method:

   public void init() {
   
   	...
   
   	this.mGLSurfaceView.setCocos2dxRenderer(new Cocos2dxRenderer(this));
   
   	...
   
   }

It’s all you need to know to be able to add video capturing capability to Cocos2d-x applications. Now build and run your test application for Android platform. You can find recorded videos in /mnt/sdcard/DCIM/ folder of your Android device.