In the wake of last year’s cancelled Google IO, it’s hard to know what sort of goodies Google had to postpone. In lieu of letting their new tech go unannounced, Google scheduled a Google Assistant Developer Day in October 2020. You can watch the keynotes themselves here: https://developersonair.withgoogle.com/events/assistant-devday but I wanted to drill down into some clarification Google offered on some previously hazy topics:
App Actions & Android Slices
https://developers.google.com/assistant/app/overview App Actions and Android Slices have been talked about since Google IO 2018, they were unveiled as complimenting features, though not necessarily depending on one another.
App Actions are a way for Android developers to hook their apps into Google Assistant, expanding Assistant’s capabilities and providing another entry point into those apps. These Actions differ from a typical Google Assistant Action in that they’re enabled by adding a special
actions.xml file to the Android app, meaning they’re only accessible on an Android device with that app installed. Whereas a typical Google Assistant Action is built as a web service, and is accessible from any Assistant-enabled device.
App Actions can function in one of two ways, performing a task by deep linking into an app, or answering something by providing information directly to the user from within the Assistant itself in the form of a Slice.
Say you had a fitness app that kept track of how much you ran. Interacting with that app’s App Action would go something like, “Hey Google, tell Paul’s Fitness App that I ran 5 miles today”. The fitness app registered to that Assistant built-in-intent will now receive an Android intent with a URI that matches what’s been declared in the
actions.xml file. From there it could navigate to the correct screen within the app just like it would with any other deeplink.
Foreground invocation is also supported, allowing specific intents to only be available when a specific activity is visible, then allowing the user to interact directly with that screen and preserving the other existing state of the app. To build on the fitness example, a user could be looking at a workout and ask the assistant, “Ask Paul’s fitness app to change this workout to a bike ride”. The app could then handle that request using the context from the current screen.
Providing information from an app to the user without them even leaving Google Assistant is arguably the niftiest way to integrate with App Actions. This is done through Slices. Android Slices allow apps to expose specific pieces of information through small, templated views, sort of like less-flexible widgets.
Going back to the fitness app example again, an interaction could be: “Ask Paul’s Fitness App how much I ran this week”. The fitness app could then display a Slice within the Assistant that shows how many miles were run, as well as any other relevant information or an option to then deep link into the app.
Where Did We End 2020?
While developing and testing App Actions and Slices isn’t new this past October, the ability to do so easily, without jumping through additional hoops, is. Previously there were whitelists developers had to be added to, approvals that needed to be done, and flakey documentation. Fast forward to the end of October 2020, and a number of changes have come down the pike, including:
Expanded Built-In-Intents - https://developers.google.com/assistant/app/reference/built-in-intents/
App Actions’ Built-In-Intents (BIIs) have expanded scope greatly to include 10 specific domains ranging from finance to travel, including a number of “common” intents to either create or search for entities within an app.
Custom Intents - https://developers.google.com/assistant/app/custom-intents
In case the built-in-intents aren’t sufficient for your app’s use case, App Actions also now support using custom intents that have their own developer-defined query patterns. That means something like, “Hey Google, open Paul’s Fitness App and send a gold star to Lauren” is now possible, even though Assistant doesn’t have a built-in-intent for “sending a gold star”.
Public App Actions Review Process
All developers can now release APKs with an
actions.xml file, though there are two implementation requirements that must be met before your App Action can be approved:
- Implement the GET_THING BII if your app has any sort of search feature. This actually replaces the old
android.intent.ACTION_SEARCHintent, Google says they recommend implementing GET_THING instead of
- Report App Action interaction results via Firebase
Once those steps are complete, you can begin the App Actions review process. It is worth noting that the App Actions review process runs separately from the review process for your Android app, so including App Actions shouldn’t jeopardize your app release. https://developers.google.com/assistant/app/get-started#request-review
Slices Have A Purpose!
Now that App Actions are here and public, there’s incentive to create Android Slices because there’s a clear path to having those slices seen by actual customers.
Alright, so now that it’s all here, how do you go about adding it to your app and spicing up Google Assistant for all your users?
The cop-out answer is to follow Google’s codelab here: https://codelabs.developers.google.com/codelabs/appactions It’s been very recently updated and does a great job of describing how to get started on all of this, including the requirements needed for testing your App Action locally. The completed codelab project is also available here: https://github.com/actions-on-google/appactions-fitness-kotlin The sample includes examples of both deep links and Slices.
Couple things to note in the codelab:
- It mentions that you can’t actually upload an apk with an
actions.xmlfile to Google Play, that’s no longer true as long as you go to “Advanced Settings” and agree to the “Actions on Google” terms of service
- The “Creating Preview” dialog sometimes ends up behind the App Actions Test Tool for Android Studio, if you press “Update Preview” and don’t see any progress, that might be why
- Invocation through adb/test tool works immediately, but implicit invocation through the assistant seems to need some time between creating the preview and it actually working on device
- A physical device is required for testing
Hop To It!
App Actions and Android Slices are another way to increase user engagement with your app, as well as simplify tasks users are already completing. If your app already supports deep linking, then getting basic App Actions integration should be straightforward, and is worth considering. The scale of adoption on these APIs is still unclear as they’re so new, but one thing’s for certain: if a user asks the Assistant to do something, you don’t want your app to be the one that isn’t showing up as eligible to fulfill that request.