Monday, February 5, 2018

D365 Developer Extensions - Beta

It’s long overdue but the next iteration of my Visual Studio extension CRM Developer Extensions (now called D365 Developer Extensions) is ready for some testers. It was nearly a ground up rebuild to hopefully make it a bit easier to maintain and have people contribute to.

What's it's got currently:
  • Web resource deployer*
  • Plug-in deployer & registration*
    * Compatible with Scott Durrow's Spkl deployment framework
  • Solution packager UI
  • Plug-in trace log viewer
  • Numerous project & item templates
  • Intellisense for organization entity & attribute names
For Visual Studio 2015 & 2017

You can check out the documentation here to get an overview of the functionality.

I’m hoping that most of the kinks are worked out but there are bound to be some yet – thus the Beta release.

Grab the latest release here and manually install. Once a few people have kicked the tires I’ll get it up on the Visual Studio Marketplace.

Post any bugs, ideas, or thoughts in the Issues area of GitHub.

Monday, December 18, 2017

Creating a Custom Virtual Entity Data Provider

One of the cool new features of Dynamics 365 Customer Engagement (or whatever we’re calling it now) v9.0 is Virtual Entities. A quick overview is you weren’t aware, Virtual Entities let you surface external data into the system as if it were actually present in the organization database (naturally with some caveats). This means it could be in forms, views, reports, available from an API, etc. Read-only at the moment, with the promise of create, update, and delete sometime in the future. With that said, this is the foundation for what is to come.

If you aren’t looking to code anything and happen to have some data exposed via an OData v4 endpoint, has a Guid for a primary key, and is not secured or only secured by a static token that can be included in the query string, then you are in good shape. You might actually be able to configure via the Virtual Entity Data Sources UI under Administration.

Of course the problem is that if you do happen to already have a web service for your data, it probably doesn’t meet all those criteria. Enter the Custom Virtual Entity Data Provider. At the time of writing, the first thing you see the linked page is a message saying that it would be better to re-build your existing data provider than to take this approach. Sounds encouraging :) Lets move on.

Again, at the time of writing the documentation: Sample: Generic virtual entity data provider plug-in is in fact incomplete. Mainly it’s missing some key steps in the set up and the information on debugging is TBD. But if you do look at the code, it might seem a little familiar. In the example we’re writing a plug-in for the RetrieveMultiple message and putting our own content in the Output “BusinessEntityCollection”. It doesn’t show Retrieve but again all the really happens is you put your own data in the Output “BusinessEntity”. Nothing we couldn’t do in CRM 2011 but remember this is the foundation of things to come.

The other item to note on the sample is the use of a QueryExpressionVisitor. Stop and think for a moment, if people are going to use Advanced Find to look for your external data, how does the system translate that into something your data source can use? Well it doesn’t, you need to handle that yourself. Microsoft has given us the Microsoft.Sdk.Data namespace to assist with converting a QueryExpression into some other format. Considering how many operators Advanced Find has between all the different data types, it seems like monumental task to try and support everything. So later when we’re defining the attributes for our external data, don’t forget you can mark them as unsearchable so they can’t be used in Advanced Find.

The code for this is located in GitHub so your can take a look at it. I’m not going to spend too much time on that because the real value comes from walking through the setup I think. Also note in my example, the web service I hooked up to didn’t provide Guids for identifiers but instead had some random characters. Since there was only about 9.7k records for the purposes of making this example work with something I found a bit funny, I created some Guids and joined them up with the provided random character ids and stored them in the plug-in. Obviously, the Guid is important of course so when you try and open a record from a view, the system needs to know which record to retrieve.

Chuck Norris jokes

So I chose to bring Chuck Norris jokes in as external data. As it turns out there is an existing web service for that: https://api.chucknorris.io/ I’ll just explain how the code is working and if you want to grab it later and look, feel free. FYI – there might not be the most “politically" correct” content on this site so be warned.

RetrieveMultiple plug-in

The API lets you search on a keyword. To keep my example minimal I extracted the first value from the first condition in the QueryExpression. Once I had that I made a search request to the API with that key word and got some results back. From there I parsed that data and created new Entity records for each result matching the schema I defined. I used a predefined list to match the API id to a Guid – naturally you’d use the Guid returned from your API. Once I had the Entities in an EntityCollection I put them in the Output “BusinessEntityCollection” and that was it. In case someone didn’t enter any search conditions, instead of returning everything I opted to use the API’s “random” feature and just returned a single result.

Retrieve plug-in

The plug-in Target Id will be the Guid you set in RetrieveMultiple. So here I grab that and use it to look up a specific joke’s random character id. As it turns out there was not a service method to retrieve a single record, it expects you’re just going to build a url and open it in the browser I guess. With that being the case I grabbed the HTML content and parsed out the values I needed for my Entity. Once I had that I put it into the Output “BusinessEntity and was done.

Lots of pictures

Here’s the walkthrough on the setup. I’m using v9.0.0.7 of the Plug-in Registration Tool for this which you can get from NuGet.

First off, register your assembly – don’t worry about adding any steps

image

Choose the new Register New Data Provider option

image

We’re creating a new solution here so add the details

image

Data Source Entity “Create New Data Source”

image

Choose your plug-in assembly & set Retrieve/RetrieveMultiple handlers you’ve created

image

Register!

I had a few issues with crashes so watch out.

In D365

Under Administration –> Virtual Entity Data Sources – New

Choose the data provider you created

image
image

Now you should have a new Virtual Entity in your solution

image

Add the new Data Source and assembly to your solution (optional)

At this point you’ll already see a Virtual Entity for the Data Source
image

Create a new Virtual Entity – this is the one for the actual data

Set the Data Source to the one you just created

image

Save

Create any other data fields, views, etc. – the external names aren’t that important I don’t think since the plug-in is doing the mapping from the real source

image

Publish All Customizations

And that’s it

Advanced Find or View

Get random joke – no search criteria
image
Get a joke containing the search word – this is all the plug-in handles in conjunction with the Chuck Norris joke web service
image
image

Final thoughts

Obviously the data is only going to be returned as fast as the web service can provide it If it’s slow, user experience is going to suffer. Getting your provider in Azure would probably be helpful.
I didn’t look at how paging might work, but that’s something that will need to be figured out at some point.

You also noticed that we didn’t explicitly register any steps on our plug-ins since the setup did that for us. I’m still wondering how you’d go about debugging with the Profiler with no steps (remember the TBD earlier?).

Get the code on GitHub:
https://github.com/jlattimer/D365CustomDataProvider




Monday, August 21, 2017

Easy Plug-in Logging Using IL-Weaving

So what is IL-weaving or Aspect Oriented Programming (AOP)? At a high level, it’s the process of "magically" injecting additional code into what you’ve written at some point during the assembly’s life cycle. But why the need? Using an automated and repeatable process to examine the code that’s written and add to it can be beneficial because it’s less code the developer has to write in the first place (so less potential mistakes) and can also address things the developer forgot (like disposing of objects). For the purposes of this post I’m not going to get into the specifics on how any of the techniques work under the hood, but if you’re interested in more detail on these processes you can read more about approaches in .NET, run-time weaving and compile-time weaving.

The example I’m going to run through stems from something that happened a few months ago and how injecting code would have made dealing with the situation easier.

Scenario, things are processing slower than the client would like. There are plug-ins, lots of plug-ins. Those plug-ins fire other plug-ins which fire other plug-ins. Also everything is running synchronously. When reviewing the code the plug-ins are quite complex. It’s fairly clear there are extra executions happening that shouldn’t be but based on the amount, complexity, and state of the code it’s hard to tell where to begin looking. My colleagues I was working with decide to add some timers to the methods that are most likely the ones that are the root of the problem to see where the bottlenecks are. The result was adding Stopwatch code to the beginning and end of methods all over the place. Not terribly difficult, just time consuming and messy when done in mass.

Well after the fact I ran across this notion of AOP and saw an example of dynamically injecting this Stopwatch code without having to actually write it. To demonstrate I’m using the open source library Fody (specially MethodTimer) which uses compile-time weaving to inject the Stopwatch code.

Walking through the example:

ManualTimer method: This might be what your code would look like when you add Stopwatches to each method’s execution today. While I'm sure you could probably simplify this some, there are still many extra lines of code.

IlWeavngTimer method: This produces the same result only by adding an attribute to the method. There is some code to support this. Toward the end there are 2 internal classes, one for the attribute and one for the actual logger. In the Execute method of the plug-in I’ve created a static instance of the logger and gave it the CRM Tracing Service to write to.

IlWeavngTimerCompiled method: If you decompile the assembly, this is what actually ended up being compiled. As you can see it added the Stopwatch code around my existing code.

The process of adding code is done during the build process and doesn’t take any dependencies on any third-party libraries – so it’s safe for CRM/D365 Online. The 2 internal support classes are also removed in the compiled assembly – assuming you’ve designated them internal.

Now for every method you want to time execution on you can simply add the [Time] attribute (including Execute) and it will write the execution time to the plug-in trace log.


You can see the full solution here:

https://github.com/jlattimer/D365FodyLogging

Wednesday, August 9, 2017

Dynamics CRM/365 Yeoman Generator

As a result of a recent Twitter exchange I spent a few hours this past weekend and came up with a Yeoman generator for CRM project templates. Now you can create CRM projects from the command line – what fun! But really the potential benefit is being able to work using Visual Studio Code. Maybe not something you’d do for your day to day development but when you’re just trying to do a quick POC of something, firing up VS Code and generating a template here is probably going to be a lot faster then opening up the full version of Visual Studio. As long as you’ve got the C# extension installed you’ll get intellisense and the project templates are configured so you can start running and debugging code right away.

If you aren’t familiar with Yeoman it’s really just a framework for scaffolding out templates for various types of projects and files. It allows you to pull in dependent packages from sources like NuGet or npm as well as to manipulate the items you end up creating.

A few of the features of what I’ve built

  • Choose from a number of different templates:
    • Console 2016+ Project (CrmServiceClient)
    • Console 2011-2015 Project (CrmConnection)
    • Plug-in Project
    • Plug-in Class
    • Custom Workflow Project
    • Custom Workflow Class
  • Pick from the latest version of each major/minor version of the SDK Core Assemblies from NuGet
  • Console projects will get the latest XrmTooling.CoreAssembly or CrmSdk.Extensions version from NuGet corresponding to the SDK Core version that was selected 
  • Console projects will get an appsettings.json file with ready to update connection strings
  • All projects can be used with VS Code or Visual Studio
  • Plug-in and workflow projects will have a new strong name key file generated and assigned to the project
  • Optionally create a Visual Studio solution file

You can check out the code on GitHub:
https://github.com/jlattimer/D365Yeoman

Or on npm:
https://www.npmjs.com/package/generator-d365


If you’re looking to install:
npm install -g yo

npm install -g generator-d365

And then to run:
yo d365

Have fun!

Tuesday, May 30, 2017

CRM Developer Extensions - Where's VS 2017 Support?

In the process of a rewrite for Visual Studio 2017.

Why a rewrite? There were some changes to be made to make the original extension work in VS 2017, it wasn't going to be a major effort but I thought this would probably be a good time to re-evaluate what was done so far and make some improvements. When I started out originally I really didn't have any plan, I just started coding. It didn't help either that I had no previous experience developing extensions for Visual Studio so I was learning as I went. Ultimately though in the end, it was a mess and it was amazing anyone at all contributed (big thanks to those that did).

As a quick refresher for those that maybe aren’t familiar with this project. In the 2011/2013 days Microsoft shipped the Developer Toolkit for Visual Studio with the SDK which had templates, deployment tools, etc. to make a CRM developer’s life easier. When 2015 rolled around it took a hiatus. Preferring not to not waste time on the tedious tasks I built my own set of extensions (VS Marketplace / GitHub) with templates, deployment tools, and other things. It seemed to fill the gap and may people got value out of using it – nearly 26k downloads thus far. Now we’ve come full circle and with Dynamics 365, Microsoft has released a new version. It’s got some cool features but ultimately isn’t getting the love it needs. Microsoft probably needs to hire someone that’s beyond outgoing and is an authority on Azure to work on it – there’s a not funny joke in there BTW :)
So what's the plan this time around? Here's what I had in mind so far.

  • Visual Studio 2017 compatibility – still thinking about supporting back to 2012 as the new VSIX v3 is supposed to support that
  • Still avoiding proprietary project types – this is one of the things I dislike most about the Microsoft offering, some of the projects won’t open unless the Developer Toolkit is installed which makes it headache in multi-developer scenarios where every person might not have the same tools installed
  • Removing the custom login and replacing with the SDK Xrm.Tooling login control – one of the things people have the most problems with and to this day I can’t explain why many still end up with an empty OrganizationService after connecting –also this removes the need to handle credentials and puts the onus on Microsoft to make sure it connects for everyone
  • Solution level mappings – should prevent having to re-connect when switching between projects in a solution, also out with the XML and in with JSON
  • Better code structure – since I didn’t know what I was doing last time out there were many things replicated and/or poorly implemented making it difficult for anyone else to pick it up and make changes – adding some level of abstraction to the core interactions that extensions rely on the work within the solutions system so it’s easier to consume without prior extension knowledge
  • Better NuGet support – to try and guide people down the right path I was choosing which versions of the various SDK assemblies got in installed when creating a new project, it worked well but relied on a lot of hardcoded stuff to make work and required updating when new major versions were released, putting stress on me to get an update out right away – this time new projects will be presented with a dynamic list of the required packages straight from NuGet, any version can be selected and will always be up to date
  • Web resource deployer – probably not too many fundamental changes from the previous version but still hoping to incorporate previous feedback I’ve received
  • Plug-in deployer – the current version is pretty basic and only handles moving the actual assembly and doesn’t address anything associated with the registration – what I’d like this time around is to address those gaps (effectively re-building parts of the Plug-in Registration Tool) and/or incorporating what Scott Durrow is doing with his SparkleXrm task runner, specifically around putting the registration details directly into the plug-in code and using it during deployment to handle setting up all the steps, images, configurations, etc. – as a person that often has to look at other people’s code I can tell you that having the registration details readily available in the code without having to open and click through another tool to figure out when it’s being executed is darn useful
  • Solution packager – probably not too many changes from the current implementation, maybe adding a deploy solution option
  • Report deployer – may get dropped, feels like it’s usefulness is lessened if you can only create reports in VS 2012
  • Early bound class creation – would love to have this functionality, hoping maybe to maybe see if Daryl LaBar will allow a port of his XrmToolbox implementation
  • Templates – would definitely like to explore other options, specifically around unit testing, maybe including some of the community testing frameworks – would really like to see someone develop a framework for JavaScript unit testing that incorporates CRM – maybe even VB templates for that 1 guy who gave the project 1 star on the VS Marketplace because it didn’t have them – also had thoughts about a tool that would capture data from a live CRM instance to incorporate into unit tests
  • Localization – would like to get all text into resource files so it can eventually support multiple languages
  • Documentation – provide better documentation how the project is put together and where to find things in hopes it makes it easier for others to contribute
  • Unit tests – actually including some - again hoping this will work toward being able to merge changes with more confidence and get quicker turn around

That brings us to what people could to do to help. But first let me share my opinion on something. It’s easy to sit back and call out Microsoft for not updating their tools to VS 2017 fast enough or to complain about this, that, or the other thing not working. As hard as I tried I’m not able to aid that cause (and I did try), but what I can do is solve the problem from a different angle which is delivering this alternative. So if you’ve made it this far and weren’t turned off by what I just said and are willing to help, this is what I’m looking for:

  • Ideas – I’d really like to hear about what other people are doing or what kinds of things would make CRM development easier or better. As this progresses, feedback on functionality is welcomed.
  • Code – Ideally I’m hoping a few people will actually want to get in and submit some pull requests. Willingness to learn a little about Visual Studio extensions is helpful but probably not required as most of that I’m willing to handle and/or expose in some manner. If there’s something you’re interested in and it isn’t in place, let me know and I’ll get it there. If you aren’t able or comfortable in submitting directly and have something like project or item templates or functionality built in WPF (user controls) that doesn’t rely too heavily on interaction with Visual Studio you can always send over something partially finished and I can try to get it added that way. Even going through and refactoring code or moving text to the resource files would be helpful.
  • Unit tests – probably a long shot but if you like writing them there’s plenty of opportunity.
  • Testing – I’ll try and get some early builds out that can used prior to final release to the VS Marketplace that can be installed manually.

If you’ve got something to say or are willing to help – post it on the GitHub Issues page for the project and I’ll be in touch. Or if you just think I'm nuts then I guess you could mention that too.

I love Dynamics 365/CRM as a platform and want to see others adopt it. My contributions are really meant to take away some of the barriers for entry into the space and get people solving business problems faster.

Work in progress:
https://github.com/jlattimer/CrmDeveloperExtensions2