Software : How Cortana will be your digital assistant at work

How Cortana will be your digital assistant at work

Posted:

Introduction and Power BI

Microsoft is banking on its digital assistant, Cortana, to help Windows 10 and the Edge browser stand out – but Cortana is also ready to show up at work.

At the Ignite conference Microsoft demoed Cortana answering questions about how to use Windows 10, finding PowerPoint presentations on specific topics and drilling into the statistics of conference attendees by showing dashboards from the Power BI service.

Power BI is a good match for Cortana because it has a model of the data you're interested in that's already set up for asking questions using everyday language. The search field in Power BI is actually a feature called Power BI Q&A that extracts information like project names, customer addresses, dates and other significant fields from your data. This lets you ask questions like 'which projects are due to be finished by July?' or 'which countries are my best customers in?' or 'how many PCs do my customers have by industry?' – which sound very like the questions you can already ask Cortana, just with a knowledge base about your business rather than more general web searches.

Power BI also creates visual representations automatically, like tables and maps, and the maps let you zoom in to see more detail. Again, those kinds of small chunks of information fit into the Cortana interface.

And Power BI gives you a model for lots of kinds of data – from data you load in from Excel spreadsheets to data it extracts from SharePoint, Microsoft Dynamics, SQL Server or third-party services like Salesforce and Twilio. That would turn Cortana into an assistant for a wide range of business tools, especially as the connection uses the Azure Active Directory single sign on system, so you only get access to the information your account has the rights to see.

Power BI integration with Cortana 1

Cortana means business

Using Cortana to drive Power BI was just a demo, Stella Chernyak of the Windows team told TechRadar, but connecting to different applications is a feature that will be included in the business versions of Windows 10. "Cortana has an extensibility framework so other third-party providers can connect to Cortana to let me use natural voice commands to extract business information from their systems," she said.

"It's one thing to use Cortana personally and another when you can join Azure Active Directory to get access and it's so relevant to you. I'm meeting a customer; I can ask Cortana 'what is their profile or what did we talk about last time'," she suggested.

Plus Cortana can track your business interests and make suggestions the way she already does for your interests and travel plans. "Cortana is proactive," Chernyak pointed out. "Cortana can use the data – with your consent – to do meeting preparation for you, to make intelligent predictions about your line of business and leverage knowledge across employees. Think of getting new employees up to speed being as proactive as Cortana suggesting things."

Privacy and training Cortana

Cortana can also work directly with universal apps in Windows 10 – a universal app can register to handle certain commands that users speak to Cortana, and they can specify what Cortana will say in return and what she will do in response to the command.

Using Azure Active Directory as the way to connect Cortana to business systems is important not just to simplify the way you make those connections, but also to secure them and make sure only the right employees get access, a Microsoft spokesperson told us.

Power BI integration with Cortana 2

If businesses are going to be comfortable with Cortana having access to information like sales forecasts and customer information, they need to have control over who sees that information, and Cortana needs to respect their need for data protection and privacy and compliance. All of that is already built in to Azure Active Directory, along with authentication for the kinds of third-party services like Salesforce that it will be useful to have Cortana talk to, through Power BI or directly through the extensibility framework.

As you'd expect, the Cortana team is looking at what other Microsoft services will make sense to connect to, like the Office graph and Office 365 content. Cortana can already find documents you have access to on the topics you're asking about – it might make sense to have the documents that Delve suggests are relevant to you also show up as suggestions from Cortana. The new Cortana interface with small cards for multiple topics of interest could be a good way to show those without getting in the way when you're doing a search.

Training Cortana

Adding business information to Cortana will also help improve her voice recognition. Cortana does very well with common web searches, a Microsoft spokesperson told us, and the number of speech searches Cortana users perform is going up (voice searches are most common in China, currently).

But to make voice recognition more accurate across a wider range of questions, they need examples of people using less common phrases to train the system up. The Project Oxford APIs may help with that too, because they use the same voice recognition APIs as Cortana.

So not only will Cortana connect to more business systems for you to ask questions of, but more apps may start offering the same voice recognition and natural queries we'll get used to with Cortana.

Making Cortana smarter: how machine learning is becoming more dynamic

On Android? You can finally make collages in Instagram

Posted:

Android users, prepare to share. Instagram's app for building collages, Layout, has made it's arrival to the Google Play store.

The app, which debuted in March for iOS, combines multiple pictures from your phone and turns them into an Instagram-friendly, square and shareable shape. Before Layout, Instagramers had long awaited the ability to stitch together their favorite pictures and share them with their friends.

Instagram collage app Layout

Unlike many collage apps, which start by asking to pick a grid shape, Layout has you choose the photos you want to share first. After you pick the ones you like, its algorithm offers suggestions for the most pleasing ways to arrange those (sort of blurry) shots from last night's concert. From there, you can fix borders, exchange images; you get the picture.

Instagram also introduced a new tool on its editing panel called "Structure", which will help bring out the finer details and textures in your images. Inverse of the Layout release, Structure is only available on Android phones now but will be coming soon to iPhone.

Just like Instagram, Layout is a free app and allows you to share through Instagram to Facebook, Twitter, and other social media.

Yahuui Blog

Wednesday, May 20, 2015