Monthly Archives: March 2018

Sharing an image via ActivityViewController with iOS and Unity3D

Welcome to my tutorial on how to share a screen image on an iOS device by extending our Unity Plugin.

We will add a share function to our plugin code that will make use of the standard iOS ActivityViewController. Using this View Controller is quite complex and powerful, and will require a setup that is dependent on the device being used.

Once again this tutorial assumes a reasonable familiarity with unity, C sharp, objective-c, iOS programming and XCode.

Let’s get started by loading up our previous project in Unity and adding a share button to the scene.

In the Hierarchy view, click ‘Create’ and select UI->Button. This will create a Canvas and add a button directly to it.

Select the button, adjust it’s anchor to be top-center and set the Y pos to -15, which should put the top of the button against the top of the screen.

Expand the button in the hierarchy and click on Text and change the text string to ‘Share’.

Now double click our PluginTest script to open it up in Visual Studio so we can add the C# share function.

First, let’s add the reference to the external IOS method that we’ll use. In the section with our other extern declarations, add the following two lines:

[DllImport("__Internal")]
 private static extern void IOSshareScreenImage(byte[] imagePNG, long imageLen, string caption, intCallback callback);

We’re going to need a public reference to our UI button. So add the line

 public Button shareButton;

We’ll also need to add the appropriate Using statement, so you can right click on Button, select Quick Fix and then ‘using UnityEngine.UI;’.

Comment out the line in our Start method that randomly shows the alert dialog. We’ll use the alert dialog to let the user know when the share method is finished along with the completion result later.

Let’s add the method that our Button will hook into. This method will call into another method that will actually create the screenshot and share it. To make our screenshot clean, we’ll hide the button before we take the screenshot and then restore it when the screenshot has been shared.

//called when the user taps the 'share' button
public void ShareScreenTapped()
{
  if (shareButton != null)
    shareButton.gameObject.SetActive(false);
  ShareScreenShot(Application.productName + " screenshot", (int result) =>;
    {
      Debug.Log("share completed with: " + result);
      CreateIOSAlert(new string[] { "Share Complete", "Share completed with result " + result, "Ok" });
      if (shareButton != null)
        shareButton.gameObject.SetActive(true);
    });
}

We’re using an anonymous function that will be called when the share function completes. This lets us re-enable the button and pop-up an alert view with the result from the share function.

This next section is quite complex. It involves saving off a reference to the passed function, a callback function that will receive the success/fail status from the iOS ActivityShare function, a method to take the screenshot and then a co-routine to wait for the end of the frame before grabbing a copy of the frame buffer and passing it to our iOS method as a PNG.

First, let’s create two static variables. One to hold the passed function reference:

static System.Action ShareCompleteAction;

And one to hold a bool state that will be true while we’re sharing a screenshot:

static bool isSharingScreenShot;

This is to prevent the method being called while we’re waiting for a previous call to complete.
Now we’ll add the callback function that will be called from iOS. In order to do that, we need to mark it with MonoPInvokeCallback so the compiler knows to marshal the call correctly.

[AOT.MonoPInvokeCallback(typeof(intCallback))]
static void shareCallBack(int result)
{
  Debug.Log("Unity: share completed with result: " + result);
  if (ShareCompleteAction != null)
    ShareCompleteAction(result);
  isSharingScreenShot = false;
}

This is similar to the callback function used by the AlertView, and in fact uses the same intCallback type. When this function is called, it will trigger the stored reference to the function passed into ShareScreenShot, if it’s not null, and clear the isSharingScreenShot flag as we’re finished.

We’re now ready to add the ShareScreenShot method. This method will make sure we’re not already sharing a screenshot and then start a co-routine that will create a texture containing the current frame buffer. As recommended by Unity, we’ll wait for the end of the next frame to ensure the frame buffer is fully rendered. Add the following lines:

public void ShareScreenShot(string caption, System.Action shareComplete)
{
  if (isSharingScreenShot)
  {
    Debug.LogError("Already sharing screenshot - aborting");
    return;
  }
  isSharingScreenShot = true;
  ShareCompleteAction = shareComplete;

  //grab the screenshot & send it to iOS
  StartCoroutine(waitForEndOfFrame(caption));
}

The co-routine will wait for the end of the next frame and then send the texture we create to our iOS method. Add the following lines to complete our C# modifications:

IEnumerator waitForEndOfFrame(string caption)
{
  yield return new WaitForEndOfFrame();
  Texture2D image = ScreenCapture.CaptureScreenshotAsTexture();
  Debug.Log("Image size: " + image.width + " x " + image.height);
  byte[] imagePNG = image.EncodeToPNG();
  Debug.Log("png size: " + imagePNG.Length);
  if (Application.platform == RuntimePlatform.IPhonePlayer)
    IOSshareScreenImage(imagePNG, imagePNG.Length, caption, shareCallBack);
  Object.Destroy(image);
}

After waiting until the end of frame, we grab the frame buffer into a 2D texture. From this, we construct a PNG rendition of the image, and then after ensuring we’re on an iOS platform, call into the iOS method, passing the PNG, it’s length, the caption and a pointer to our shareCallBack function.

With the C# modifications completed, we need to hook up our button and also set the reference to it for our script. I had to adjust my Unity layout so you could see how I connected the button to the ShareScreenTapped method.

Now it’s time to update our iOS code. Double click the file ‘MyPlugin’ in the Plugins/iOS folder. This will launch XCode and open our file.

First, add a method to the MyPlugin class that will package the PNG image and caption into an NSArray and pass them to an ActivityViewController.
Before we do that, we need to add a few extra variables to our class. In the @interface declaration at the top of the file add these two lines:

INT_CALLBACK shareCallBack;
UIPopoverController *popover;

The shareCallBack variable will hold the pointer to our C# callback function, while the popover variable will point to the popover controller we’ll use if the code is running on an iPad device.

With that done, we can add the method’s we’ll need to the main class by adding the following lines before the @end statement for the MyPlugin @implementation:

-(void) shareScreenImage:(const unsigned char*) imagePNG_in length:(long)length caption:(const char*) caption_in callback:(INT_CALLBACK) callback
{
  NSMutableArray *shareableItems = [NSMutableArray arrayWithCapacity:2];
  //This array will hold the caption and image so we can send it to the share activity.
  NSString *caption;
  UIImage *image;
  if (caption_in!=nil)
  {
      caption = [MyPlugin createNSString:caption_in];
      [shareableItems addObject:caption];
  }
  if (imagePNG_in!=nil)
  {
      NSData* pngData = [NSData dataWithBytes:imagePNG_in length:length];
      image = [UIImage imageWithData:pngData];
      [shareableItems addObject:image];
      pngData = nil;
  }

This will convert the caption, if it’s set, to an NSString and the PNG data to a UIImage. Note we have to first convert the supplied byte array into a NSData object, and we need the length of the array to achieve this.

shareCallBack = callback;
UIActivityViewController *activityViewController = [[UIActivityViewController alloc] initWithActivityItems:shareableItems applicationActivities:nil];
activityViewController.completionWithItemsHandler = ^(NSString *activityType, BOOL completed, NSArray *returnedItems, NSError *activityError) {
    NSLog(@"Activity %@ completed: %d",activityType,completed);
    if (activityError!=nil)
        NSLog(@"Error: %@",[activityError localizedDescription]);
    if (shareCallBack!=nil)
        shareCallBack(completed);
};

Here we create a UIActivityViewController with the shareableItems array we setup and populated earlier. We also construct a completionWithItemsHandler. This function will be called when the user completes the share function, either by selecting a method, or by cancelling the request.

The completion function will pass the completed bool to our callback method, assuming it’s not nil.

if (UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPhone)
    [UnityGetGLViewController() presentViewController:activityViewController animated:YES completion:^{
        NSLog(@"share presented");
    }];
else
{
    popover = [[UIPopoverController alloc] initWithContentViewController:activityViewController];
    UIView *mainView = UnityGetGLView();
    popover.delegate = nil;
    [popover presentPopoverFromRect:CGRectMake(mainView.frame.size.width/2, mainView.frame.size.height-10, 0, 0) inView:mainView permittedArrowDirections:UIPopoverArrowDirectionAny animated:YES];
}
}

In order to present the activity view controller, we need to use two different methods, one for iPhone devices and another using a popover controller for iPad devices. We’ll make the choice by checking the result of the macro UI_USER_INTERFACE_IDIOM.

On iPhone devices, we can present the ViewController using the main Unity ViewController directly, while on iPad devices, we first create a popover controller and then use the main Unity View as the parent view to present the popover.

In both cases, the ActivityViewController finish method is used to notify Unity that the share is complete via our supplied callback function.

Finally, we need to add a C style method that our C# code will call into, which will call our new share method on the plugin. Add the following lines to the extern C block:

void IOSshareScreenImage(const unsigned char* imagePNG, long imageLen, const char* caption, INT_CALLBACK callback)
{
    [[MyPlugin sharedInstance] shareScreenImage:imagePNG length:imageLen caption:caption callback:callback];
}

This completes the iOS code modifications, however we need to add a key to the application’s plist file that let’s iOS know we may want to save screenshots into the user’s camera roll.

Click on the Info.plist file in the file hierarchy in Xcode, and then click the + icon to the right of the first line. In the text box that pops up, type the following:

Privacy - Photo Library Additions Usage Description

And then in the string box to the right, add a description that a user might see when they try to save a screenshot. I used:

Allow access to save screenshots

With all these modifications in place, we’re ready to build and test the code. Switch back to Unity, give it a few seconds to rebuild the C# code and then press CMD-B to start the project building.

Once XCode has finished building the project the code will execute on our simulator. Tap the ‘share’ button at the top of the screen and you’ll see the activity controller appear. You can then select what iOS will do with your image.

Go back to XCode and select an iPad as a target device. Now when you run the code and tap the share button, you’ll see a popover appear with the share activities in it. Unity continues to run behind this popover as you’d expect.

Both types of device will display an alert when the share is completed with the bool completion status. A 0 indicates the share failed, while a 1 indicates success.

If you run on a real device, then you’ll get more share options, including iMessage and Facebook if you have the app installed. If you use iMessage, along with the screenshot image you’ll see the caption text we’ve passed through. Notice that the screenshot doesn’t have our ‘share’ button on it, as we hid that before we grabbed the frame buffer.

We’ve now added a share function to Unity that allows the user to send a screenshot to various activities on their device without leaving your App. Use this to let users send highscore images, or new level images to their friends.

I hope you found this tutorial useful and are able to use it to add sharing to your apps. The code for this plugin can be found at https://github.com/cwgtech/iOSUnityShareScreenShot and the video at https://youtu.be/NwphcgWQMhQ.

As always, please feel free to reach out with any comments or questions.

Using a child activity to wait for onActivityResult with Unity3D

Welcome to my tutorial on how to extend our Unity plugin to get a callback from onActivityResult without overriding the standard Unity Player Activity.

We will add a child activity to our plugin that will be launched when required, and will wait for a call to startActivityForResult, and pass that back to our C# callback function.  Normally, we’d do this by extending the UnityPlayerActivity class, but that means we won’t play nice with any other plugins or extensions that want to do the same thing, and we must make sure our Android project imports the correct version of the UnityPlayer each time we upgrade.

Once again this tutorial assumes a reasonable familiarity with Unity, Java, Android programming and Android Studio.

Start by loading up our previous project in Unity and the MyPlugin project in Android Studio.  All of the changes we’re going to make this time will be entirely to the Java code.

Right click on the Unity tab in the Project View and select New/Activity/Empty Activity.

Call the Activity “OnResultCallback”, deselect ‘backwards compatibility’ and make sure the package name matches the package name you’ve been using.  For me, that is ‘com.cwgtech.unity’. Click Finish.

If you forget to uncheck the backwards compatibility box, you’re new activity will extend AppCompatActivity.  You need to change that to Activity.

Add the following four lines:

public static final String LOGTAG = MyPlugin.LOGTAG + “_OnResult”;
public static MyPlugin.ShareImageCallback shareImageCallback;
String caption;
Uri imageUri;

You’ll get an error on MyPlugin.LOGTAG, so you’ll need to switch back to the MyPlugin class and change the LOGTAG definition from private to protected.  We’re going to use this modified LOGTAG to identify the Log entries from this child activity, while the static callback variable will hold a pointer to the C# callback our main plugin receives.

Switch back to our new activity, and add the following method before the onCreate method:

void myFinish(int result)
{
    if (shareImageCallback!=null)
        shareImageCallback.onShareComplete(result);
    shareImageCallback = null;
    finish();
}

We’ll use this to exit our activity, calling the callback method if it exists, and then clearing it after use.  Now modify the default onCreate method. Remove the line

setContentView(R.layout.activity_on_result_callback);

This new activity will not have a content view, so we’ll not need to set it.

Add the following lines:

Log.i(LOGTAG, "onCreateBundle");
Intent intent = getIntent();
if (intent != null) {
    caption = intent.getStringExtra(Intent.EXTRA_TEXT);
    imageUri = (Uri)intent.getExtras().get(Intent.EXTRA_STREAM);
    Log.i(LOGTAG, "Uri: " + imageUri);
}
if (intent==null || imageUri==null)
{
    myFinish(1);
    return;
}

This will get the intent passed to our activity and grab the caption and imageUri that were included in the intent.  If there is no intent or image, then just exit the activity, as we’ve nothing to do and we’ve been called incorrectly.

Now to call the share intent and wait for a result.  Add the following:

try
{
    Intent shareIntent = new Intent(Intent.ACTION_SEND);
    shareIntent.setDataAndType(intent.getData(),intent.getType());
    shareIntent.putExtra(Intent.EXTRA_STREAM,imageUri);
    if (caption!=null)
        shareIntent.putExtra(Intent.EXTRA_TEXT,caption);
    startActivityForResult(Intent.createChooser(shareIntent,"Share with..."),1);
}
catch (Exception e)
{
    e.printStackTrace();
    Log.i(LOGTAG,"error: " + e.getLocalizedMessage());
    myFinish(2);
}

We copy forward the data from the incoming intent to a new intent, which we then pass on to the chooser and wait for a result.  Wrap the whole thing in a try/catch so any errors will be flagged and the app will not crash.

Add a new method that will override the default onActivityResult method and pass the resultCode back to our callback.

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    Log.i(LOGTAG,"onActivityResult: " + requestCode + ", " + resultCode + ", " + data);
    myFinish(resultCode);
}

The result code will be -1 or 0 depending on the share activity, so we’ll pass that back to our C# method.  Note that earlier we sent a 1 or 2 depending on the error condition, so our C# code could check for that and give the user more information.

That’s the Java code for our child activity completed, but we’ve a few things to clean up in the manifest and we can also remove the layout that was auto-created.

Expand the res folder in the project hierarchy and right click on the ‘layout’ folder.  Select delete and confirm the deletion. This will remove the folder and the un-needed layout file.

Now expand the manifest folder and double click the AndroidManifest.xml file.  After the name=”.OnResultCallback” but before the closing > add the following line:

android:theme="@android:style/Theme.Translucent.NoTitleBar"

This will cause our child activity to effectively have no display.  If we use the NoDisplay theme, then we run into a problem on Android 6 and higher.  A NoDisplay theme expects an activity to call finish before it’s resumed, and in our case that means we don’t get the OnResultCallback as our activity has been terminated.

Switch back to the MyPlugin java source where we will modify the plugin to use our new child activity.

Go to the section in shareImage where we prepare the shareIntent.  We’re going to replace the call to startActivity with the following three lines:

shareIntent.setClass(mainActivity,OnResultCallback.class);
OnResultCallback.shareImageCallBack = callback;
mainActivity.startActivity(shareIntent);

Make sure you remove the line:

mainActivity.startActivity(Intent.createChooser(shareIntent,"Share with..."));

And you can also remove any references to the ‘result’ variable, as we will no longer use it, including the line:

callback.onShareComplete(result);

As our new child activity will call the callback hook when the OnActivityResult is triggered.

That’s all the modifications completed, so build the plugin by clicking on the green play button, assuming you’ve still got the copyPlugin task indicated in the dropdown.  When gradle completes, you can switch back to Unity and build the project using the new plugin.

I’m going to run the apk on the emulator as before, but now when I tap the share button, I’ll still get the share dialog, however the result pop-up alert will not occur until I’ve finished interacting with the share dialog.

And that’s it done.  We’ve now got a child activity that will send our image to the share system, and wait for it to finish and return a value, which we then forward to our Unity App.  You can use this technique for any intent you need a result from. You’ll need to add custom code that either uses the requestCode passed when the activity is started to decide how to handle the passed intent, or create other child activities that just handle your specific case, whether that is a photo-picker or a QR code scan request.

You can download the source code for this plugin from https://github.com/cwgtech/AndroidActivityResult, and watch the video of this tutorial at https://youtu.be/HrhYWBqxkn8

Please feel free to post any comments or questions.