Run Azure Stream Analytics job from UWP app

 

Working with the newest technologies is great and frustrating. I can create solutions I’ve never made before while complaining about the outdated/incomplete documentation.

I had one of those feelings while working with Azure Stream Analytics (ASA). My solution worked but there was one ‘elementary and simple’ thing I wanted: Start the ASA-jobs within my C#-code. That shouldn’t be hard and there’s some documentation. But no, I needed to combine several opposed solutions to a new one to make it possible.

In this post I shortly explain how you can start ASA-jobs within your C# UWP application:

  • I explain which components you have in the authentication process and which parameters you need.
  • Example code is provided. You only need to enter your parameter values.

Prerequisite

 

The following diagram shows how the authentication works, corresponding to the steps below.

AuthAzureADappASA

 

Step 1: Create App for accessing Azure Resources

You will create an application in your Azure Active Directory and assign a role to this application.

This is described well in the following post: https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/.

Follow these steps. You might change some settings depending on your exact needs. A few choices I made:

  • Paragraph ‘Configure multi-tenant application’ I did not enable multi-tenancy. Because it was not the case for my solution.
  • Paragraph ‘Assign application to role’ I selected the role Contributor to make sure my application can also start a ASA-job.

Important id’s/keys you created in this steps and need later on:

  • Client ID of the application (See paragraph ‘Get client id and authentication key’)
  • Key of the application (See paragraph ‘Get client id and authentication key’)
  • TenantID (See paragraph ‘Get tenant id’)

Step 2: Get all the information you need

When running ASA jobs you’ll need some ID’s, keys and names. Below, you see a list of properties you need. The first 3 you created in the previous step. The other values are already available because you have an ASA job, which belongs to a ResourceGroup which is part of an Azure Subscription.

  • Client Id from the application
  • Key of the application
  • Tenant id of the application
  • Id of the subscription where the ASA is part off
  • Name of the Resourcegroup where the ASA belongs to
  • Name of the ASA job

You will use a few references. See the list below:

  • System.Threading.Tasks
  • Windows.UI.Xaml
  • Microsoft.Azure.Management.StreamAnalytics
  • Microsoft.Azure.Management.StreamAnalytics.Models
  • Microsoft.Azure
  • Microsoft.IdentityModel.Clients.ActiveDirectory

Step 3: Request AccessToken

Now you request your new application for an AccessToken. To get this you sent the clientId and key of the application to your Active Directory.

The response message containts an AccessToken for this application. This AccessToken and your SubscriptionId (from the Subscription which contains your ASA-job) is your Token Cloud Credential.

Step 4: Start Azure Stream Analytics job

You use this Token Cloud Credential to create a new Stream Analytics Client. This client is used to run your ASA job. You can set job parameters like the JobStartTime. Besides the parameters you’ll need the name of the ResourceGroup where your ASA job belongs to and – of course – the name of the ASA job.

When the call is made, it will take about 2 minutes before the job is started. During these 2 minutes:

  • Steps 4-A are processed at the back. This will check if the AccessToken you send has the valid permissions.
  • The ASA-job is started (which takes most of the time)

Code:

using System.Threading.Tasks;
using Windows.UI.Xaml;
using Microsoft.Azure.Management.StreamAnalytics; //Get this from NuGet Package Manager
using Microsoft.Azure.Management.StreamAnalytics.Models; //Automatically loaded with Microsoft.Azure.Management.StreamAnalytics
using Microsoft.Azure; //Automatically loaded with Microsoft.Azure.Management.StreamAnalytics
using Microsoft.IdentityModel.Clients.ActiveDirectory; //Get this from NuGet Package Manager

namespace RunAnAsaJob
{
    public sealed partial class MainPage
    {
        //Parameters needed for managing ASA jobs
        static string appClientId = "<Client Id from the application>";
        static string appKey = "<Key from the application>";
        static string appTenant = "<Tenant id from the application>";
        static string subscriptionId = "<Id from the subscription where the ASA is part off>";
        static string resourceGroupName = "<Name of the Resourcegroup where the ASA belongs to>";
        static string streamAnalyticsJobName = "<Name of the ASA job>";

        public MainPage()
        {
            //Start the ASA jobs
            StartStreamAnalyticsJobs();

            //Stop the ASA jobs
            StopStreamAnalyticsJobs();
        }

        private static async Task<TokenCloudCredentials> GetManageAzureResourcesCredentials()
        {
            //Connect to app, registered in Classic Azure Portal. Check the necessary permissions of this app for the ASA jobs
            var clientCredential = new ClientCredential(appClientId, appKey);
            var authContext = new AuthenticationContext("https://login.microsoftonline.com/" + appTenant + "/oauth2/token");
            authContext.TokenCache.Clear();//Once AcquireTokenAsync fails, it caches it and won't work for a while. Without using cache a new request is always made.
            var result = await authContext.AcquireTokenAsync("https://management.core.windows.net/", clientCredential);
            var creds = new TokenCloudCredentials(subscriptionId, result.AccessToken);
            return creds;
        }
        private static async void StartStreamAnalyticsJobs()
        {
            var asaClient = new StreamAnalyticsManagementClient(await GetManageAzureResourcesCredentials());
            JobStartParameters jobStartParameters = new JobStartParameters
            {
                OutputStartMode = OutputStartMode.JobStartTime
            };
            LongRunningOperationResponse jobStartResponse = asaClient.StreamingJobs.Start(resourceGroupName, streamAnalyticsJobName, jobStartParameters);
        }
        private static async void StopStreamAnalyticsJobs()
        {
            var asaClient = new StreamAnalyticsManagementClient(await GetManageAzureResourcesCredentials());
            LongRunningOperationResponse jobStopResponse = asaClient.StreamingJobs.Stop(resourceGroupName, streamAnalyticsJobName);

            Application.Current.Exit();
        }
    }
}

Useful links:

https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/

https://lookonmyworks.co.uk/2015/07/19/creating-a-stream-analytics-query-job-programatically/

http://qaoverflow.com/question/authorization-failure-when-creating-a-stream-analytics-job/

http://stackoverflow.com/questions/31471828/authorization-failure-when-creating-a-stream-analytics-job

https://azure.microsoft.com/nl-nl/documentation/articles/stream-analytics-dotnet-management-sdk/

https://azure.microsoft.com/en-us/documentation/articles/active-directory-integrating-applications/#BKMK_Native

Hylke Peek is a Data Solution consultant and is currently working for The Backbone. His focus is on BI, Analytics, IoT and Cloud. He is also a speaker, teacher and blogger. Mainly he is working with the Microsoft stack (MCSA Data Platform and MCT). Always trying out the new stuff!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Next ArticleBuilding the Poor Man's IoT Presentation (+ inspiration for new posts)