Tuesday, July 19, 2016

Microsoft Integration MVP 2016– 7th Time in a row!

A few weeks ago I received an e-mail from Microsoft with exciting news that my MVP status has been renewed again. I am now an Azure MVP!

[MVP_Horizontal_FullColor%255B4%255D.png]

For me this is the seventh time to receive this award. The sixth year in the program has been again an awesome experience, which gave me the opportunity to do great things and meet inspiring, very skilled people. I have had some interesting speaking engagement, which were fun to do and were very fulfilling. I learned a lot through speaking thanks to the community and fellow MVP's. I was able to share my experiences through these speaking gigs and other channels like this blog, MSDN Gallery, and above all the TechNet Wiki.

I would like to thank:
  • My MVP old lead William Jansen, and new MVP lead Tina Stenderup-Larsen.
  • The BizTalk Product Team, Jim Harrer, Ed Price, Mandi Ohlinger, Tord G. NordahlDan RosanovaJon FanceyPaolo Salvatori, and all other Microsoft employees involved.
  • People at my former employers: Rene BrauwersEldert Grootenboer, fellow MVP Edward Bakker and many others. 
  • At my current company DutchWorkz : Rutger van Hagen and colleguaes.
  • Fellow Microsoft Integration MVP's: Richard SeroterKent WeareMikael HåkanssonJohan Hedberg, Saravana KumarNino Crudele, Sandro PereiraStephen W. Thomas, Mick Badran (Azure), Micheal StephensonTomasso GroenendijkNicholas HauensteinSalvatore Pellitteri,Sam VanHoutteGlenn ColpeartBill ChesnutHoward S. Edidin, Martin Abbott, Leonid Ganeline, and Ashwin Prabhu, who I got learn even better and supported me in this program.
  • The BizTalk community: Mikael SandLex HegtColin Meade, Naushad Alam, Johann Cooper, Mark Brimble, Mitch VanHelden, Sven Van den Brande, Jérémy Ronk,  Maxime Labelle, Jean-Paul Smit, Dean Robertson and the collegueaes at Mexia, and many others that make the BizTalk community strong! 
  • Andrew Slivker from Sentinet.
  • Finally my wife Lian and children StanEllis and Cato for their support.
I’m looking forward to another great year in the program.

Cheers,

Steef-Jan

Wednesday, May 11, 2016

A solid vision for integration - Day 1 Integrate 2016

Microsoft is bringing the technologies around integration together in their vision for integration. API Management, Logic Apps, Connectors, Service Bus, and BizTalk are shaped into a hybrid type of platform. BizTalk and the logic apps form the heart in their vision to provide a seamless hybrid experience. Jim Harrer, the new PM for integration, shared the vision, services, and roadmap with us today at Integrate 2016. This event is the only integration focused event worldwide in the London Excel.


Below the picture that tells it all. And around BizTalk will see a release of the CTP2 version soon, including a first version of a logic app adapter, which will broaden the connectivity spectrum of BizTalk.

The vision shows Microsoft commitment in the integration space. Many thought that the BizTalk product was on life support and that there was no integration story. Well there is one and the roadmap shows a lot of promise. And what Microsoft has planned for us can be seen in the next picture.










The Visual Studio Integration is one of the much anticipated features that many integration professionals cheered. And it shows a good step forward. The product team shows what investments the Product team is making, which will enhance our confidence in integration with Microsoft technologies.
Integrate 2016 started with a blast and demonstrates that this is event is much needed to reestablish Microsoft as frontrunner in the integration space and that their technology and services are very relevant.

Thanks BizTalk360, their team, MVP’s, the attendees and Microsoft for making this event happening.

Cheers,

Steef-Jan

Thursday, March 03, 2016

BizTalk Server Extensibility (e)Book

A new BizTalk Server eBook is available for the 2013 R2 version with the focus on extensibility.

Together with Johann Cooper and Eldert Grootenboer I have written on various aspects of extensibility with BizTalk Server. The .NET framework adds tremendous value in extending BizTalk Server solutions to create more versatile, robust solutions. We discuss the extensibility for BizTalk Server from a holistic view around the following topics:

- Ports
- Orchestrations
- BRE
- BAM
- Deployment
- Operations
- Tooling


The eBook is available through the BizTalk360 website.

The journey started after my first webcast BizTalk Extensibility for the Integration Monday a year ago in February 2015. At first I did not have any plans to pursue anything after that talk. However, a few months later I got in touch with Johann Cooper, who I interviewed for my blog and met in person in 2014 during my visit to Australia.

We discussed opportunities of writing a book for a well-known publisher. Unfortunately a deal never materialized and we started to think about other options. Our initial thought was to create a white paper on BizTalk Server Extensibility as this didn’t have the attention we though it should have. We felt the .NET framework empowers BizTalk Server in a very valuable way, if applied with thought.

Johann and I started writing the paper at the end of September 2015. And soon it turned out that we had many topics around extensibility and predicted that the amount of material would validate making an eBook out of it rather than a paper.

Our prediction was right. After a few months we had close to 100 pages, yet some topics still required attention. Therefore, we involved Eldert as an extra co-author to end of the book in time with his expertise on some of the topics like BAM and the BTDF. His help and our motivation helped us to finish what we started in September.

We were not alone in this, as Sandro Pereira, Mark Brimble and James Corbould put in numerous hours to review our eBook. These guys are superb reviewers and helped us a great deal to evolve the eBook to a very valuable resource for you as readers to benefit from. Therefore, Sandro, Mark and James, thank you so much for your efforts and support!

I also like to extend my gratitude to BizTalk360 and Saravana Kumar, who has publish the eBook through his site. Saravana is a great support towards the BizTalk community, thank you my dear friend.

Working on this eBook demanded a lot of time for us and it was worth every second of it. I enjoyed working on it and collaborating with both co-authors and like to thank both Johann and Eldert for realizing this eBook. A year ago I never thought this end result was in the making.

Enjoy reading it,

Steef-Jan

Tuesday, February 09, 2016

BizTalk360 Version 8.0 Review

A new version of BizTalk360 has been released, version 8 which comes with a boatload of features and a complete overhaul of its user interface to provide an exceptional user experience. The product adopted a concept of widgets in the dashboard similar to the new Microsoft Azure portal. A BizTalk 360 user can now create and customize dashboards for different use cases.

The User Interface, Dashboards and Widgets.


In this post I will review a few of the new features and let’s kick off with the UI, which has improved dramatically. Intially BizTalk360 was a Silverlight application before it migrated to HTML 5. And in version 8.0 it has adjusted to what you experience on the Azure Portal and what you see with UI in devices. Below you see a customized dashboard with several widgets.
















Figure 1. An example of my customized monitoring dashboard.

The end user has also the ability to create his own custom widgets via the settings page.


























Picture 2. An example of creating a custom widget.

Furthermore the separation of monitoring and analytics is based upon audience targeting. An analyst or support engineer can interpret information of messaging and processes within a BizTalk Group from a different perspective as shown below demonstrating a customized dashboard.











Figure 3. An example of a customized analytics dashboard..

BizTalk360 API’s


Another incredible addition are the API’s, which offer a tremendous amount of value to enterprise customers. It brings extreme extensibility for its end users and enables the for instance to pull information from several environments. The API’s (REST web services) are the core of the BizTalk360 capabilities and the BizTalk360 front-end uses them. And as a BizTalk360 end user you can leverage all of these API's to have access to your BizTalk environment. In the setting menu under API Documentation you can find the descriptions of each API and operation. For instance if you select BizTalkApplicationService you can observe a list of operations belonging to it. Subsequently select one of the operations, Services.REST/BizTalkApplicationService.svc/GetOrchestrations, specify the environmentId and applicationName and hit “Try it out!”.














Figure 4. BizTalk360 API’s document page..

You can call the API’s from different machines, by setting the appropriate authorization in the user access policy. Calling the API operation from SoapUI will result in the same response as from BizTalk360 API Documentation.















Figure 5. Calling BizTalk360 API from SoapUI.

The look-and-feel are similar when you publish an (Web) API configured with swagger documentation describing it.

Business Rules in a browser


Yet another new feature added to BizTalk360 is the rule engine capability of BizTalk Server i.e. the business rule and composer are accessible from a browser. You can view what type rules are deployed.














Figure 6. Deployed rules in a BizTalk Group.

And one of the capabilities is that you as end user can compose new rules, edit or test them. Below you see an example of testing a deployed rule, which you can do in a browser!

















Figure 7. Test of a deployed business rule.

The version of the business rules composer in BizTalk360 only supports pre-created vocabularies. Therefore, to ensure a smooth user experience, you need to create all the vocabularies you will require to develop or edit your rules in the BizTalk Business Rules Composer. In a nutshell follow the instructions on the BizTalk360 portal with regards to rules.

Live feeds


In an enormous environment various operation people perform various actions in a production or UAT environment, and it can be valuable means of real-time tracking of what happens. This new version of BizTalk360 offers live feeds and you view on the right side after click the icon (indicated in red in the picture below).


Figure 8. Live feed in BizTalk Group.

Alarms and notifications


In BizTalk360 you can set alarms, which can result in an email being sent to you.















Figure 9. An example of email indicating a receive location is down.

Or you can leverage one of the out of the box notification channels like Slack and configure an alarm using a notification channel.




















Figure 10. Enable the SlackNotification on a custom alarm.

Once you have set up Slack following the instructions on BizTalk360 UI and the alarm you will receive notifications in Slack.

















Figure 11. Notification in Slack.

Final words


The BizTalk360 team has worked intensely the last 12 months to bring this to the table. Not just the flexibility with widgets, yet also with the separation of monitoring and analytics, the 360 plus API’s on top of your BizTalk Group, live feeds, notifications and the rules engine. This product has definitely evolved to the next level. Expect the release of 8.0 to change the way BizTalk is being monitored!

Cheers,

Steef-Jan

Sunday, December 27, 2015

Year 2015

Time flies when your having fun. I did have fun this year working with Microsoft Azure and Integration at a utility company. Did an IoT project, bridged on premise applications to the outside world (Service Bus Relays and Queues) and some old skool integration. What else did I do? Well you might have guessed: speaking, travelling, and have an occasional beer. Pictures tell more than a thousand words.


























Thanks everyone for reading my blog and I will continue to share my knowledge through speaking and writing in 2016.

Cheers,

Steef-Jan

Monday, October 12, 2015

Upcoming Events

In two weeks time I will head out to North America. My first stop will be Calgary to visit my friend Kent Weare. My MVP buddies Saravana Kumar and Michael Stephenson will join me as we will be speaking during the Azure Hybrid Integration the 30th of October at Microsoft Calgary.

The Azure Hybrid Integration will be a full day with Kent, Saravana, Michael, myself and Darren King from Microsoft speaking on the following topics SaaS connectivity, IoT, Hybrid SQL Server, BizTalk administration & operations and Two Speed IT using Microsoft Azure.

The free event takes place on October 30th, 2015 at the Calgary Microsoft office. You can find more details here.

After this event we will head out to Redmond for the yearly MVP Summit. This is a multi-day event that is hosted in Bellevue and at Microsoft headquarters in Redmond, Washington. All the MVP in the world that are able to come will be there to connect with fellow MVP’s and the various product groups.

I look forward to both events as it will enable me to engage fellow MVP’s, the integration community in Canada and to learn more about the future road maps of integration, Microsoft Azure and direction of Microsoft itself. The IT world around me is changing rapidly and for personally I am noticing it as my projects change from an on premise integration- to hybrid solution focus.

For those people that register for the Calgary event,  I am looking forward to meet you and to have some interesting discussions.

Cheers,

Steef-Jan

Saturday, August 22, 2015

Azure WebJobs: ServiceBusTrigger

Challenge is to build a WebJob that listens or monitors a queue in the Microsoft Azure Service Bus within a certain namespace and pick of each message that is send there by a message producer and process it. The WebJob acts as a message consumer of the messages on the queue. Below a high level diagram of a scenario that will be explained in this post and how I faced the challenge.

image

You can build WebJobs inside Visual Studio by installing the WebJobs SDK. Once you have installed the SDK you have template available to build a WebJob in C# or Visual Basic.

clip_image004[6]

You can select this template specify a name for the WebJob and click Ok. You will see that a program class will be created.

clip_image006

And a Functions.cs.

clip_image008[6]
By default a method will be created for you to monitor or listen to an Azure Storage Queue, not Service Bus Queue! To have method that will be triggered/executed when a message is written to an Azure Service Bus Queue you will need to have ServiceBusTrigger. This not available in the project and you will need to add the Microsoft.Azure.WebJobs.ServiceBus NuGet package.

Now a can change to ServiceBusTrigger in method ProcessQueueMessage and specify the queue I want to listen to.

clip_image010[5]

Next change is changing the type of the message from string to BrokeredMessage type. This type is not available in your class unless you add using statement for Microsoft.ServiceBus.Messaging. The package is already in the project, because it is part of the imported NuGet package. The TextWriter object can be used to write log statements that can be viewed in the AzureWebJob Dashboard.

When a message arrives on inboundqueue it will be picked up by WebJob and enter the ProcessQueueMessage method in runtime. Here I can extract the message body and send it for instance to Redis cache as key value pair (reason of picking this example is based on a request from someone on twitter to share how to do that). To send it to Redis Cache I need to import another NuGet Package i.e. StackExchange.Redis (client library). Now the complete code for the functions class looks like below:

clip_image012

Before the WebJob can be deployed to a WebApp a few configuration settings have to be done in the app.config. The connection strings for the AzureWebJobsDashboard and AzureWebJobsStorage need to be provided in the connectionStrings Section. These are required to view the log in Azure i.e. AzureWebJobsDashboard. The connection string that needs to be specified is the connection string to an Azure Storage account. Format is as follows:

DefaultEndpointsProtocol=https;AccountName=[Storage Account Name];AccountKey=[Access Key]

The other connection string that has to be provided is for the AzureWebJobsServiceBus. Format is:

Endpoint=sb://namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[Access Key]

Finally in the appSettings section the connection string for the Redis needs to specified. Connection string has the format of:

namespace.redis.cache.windows.net,abortConnect=false,ssl=true,password=[password]

Once configuration is done the app.config will look like:

clip_image014

Before the deployment (Publish to Azure) of the WebJob can be done a configuration setting in WebApp has to be done to enable AzureWebJobDashboard.

clip_image015
This is an important step. In case you forget this than observing the WebJob logs will result in the following error:

clip_image017[6]

Now the WebJob can be deployed via Visual Studio to a WebApp. Right click on the project and choose Publish as Azure WebJob...

clip_image018

You will see a Publish Web dialog and here you import the publishing setting from WebApp. These settings can be downloaded from Azure Portal.

clip_image020

Next you can click Ok and you will go to next section of the dialog i.e. Connection.

clip_image022

Click Validate Connection to see if connection info is correct. When valid you can click Publish. Now the WebJob will be published to WebApp. In the output window of Visual Studio you will see that deployment went successfull.

clip_image024

In the Azure Portal you can see the WebJob in the WebApp.

clip_image025

When you click on the logs url you will be redirected to the Microsoft Azure WebJobs portal.

clip_image027

Nothing much has happen so far, only that the Job has started. In case I send a message to the queue using for instance ServiceBus Explorer, I will see some action. Send a message via the ServiceBus Explorer to the queue.

clip_image029

Refresh the AzureWebJob Portal and a new entry is available.

clip_image031

Once you click on the Functions.ProcessQueueMessage you examine the logs.

clip_image033

To explore what is in my REDIS cache I need to navigate to the service in the Azure Portal and open a console. Enter GET and messagid.

clip_image035

image

As you can see the message is now in the Cache.

It took me sometime to get the ServiceBusTrigger working. After some digging around I was able to get the ServiceBusTrigger working and see its behaviour through the Azure WebJob Portal. The trigger is not limited to queues as it will also work for Service Bus Topic/Subscription. The signature of the method would look like:

clip_image037

Resources to explore with regards to this blog post are:

· http://stackoverflow.com/questions/15441853/with-azure-brokeredmessage-get-the-body-without-knowing-the-type
· https://azure.microsoft.com/en-us/documentation/articles/websites-dotnet-webjobs-sdk-get-started/
· http://blogs.blackmarble.co.uk/blogs/sspencer/post/2014/09/22/5-Tips-for-using-Azure-Web-Jobs.aspx
· http://stackoverflow.com/questions/28077330/why-do-i-need-to-configure-connection-strings-for-webjobs-in-azure-management-po
· https://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/


Cheers,

Steef-Jan